Find_each Vs Each In Rails: Which Should You Use?
Learn when to use find_each vs. each in Rails, how batching works, and how to process large datasets without memory issues.
• 18 min read
• 18 min read
Learn when to use find_each vs. each in Rails, how batching works, and how to process large datasets without memory issues.
• 18 min read
• 18 min read
Every Rails developer writes Model.all.each at some point. It works, it reads nicely, and on a development database with two hundred rows it is instant. Then the same code ships to production, the table has grown to four million rows, and the background job that used to finish in seconds eats all the memory on the box and gets killed.
That is the moment most developers meet find_each for the first time.
This guide explains what find_each in Rails actually does, how it differs from Ruby's each, how batching keeps memory flat, and how to decide between each, find_each, and find_in_batches. Everything here uses modern Active Record syntax and Rails 8 conventions, and every example is written so you can copy it into your own app and adapt it.
A Ruby array holds objects in memory. That is fine when the array is small. The trouble is that an Active Record model instance is not a cheap object. Each one carries:
user.orders or order.customertext columnsA rough rule of thumb is that a model instance with a handful of columns costs somewhere around one to three kilobytes of Ruby heap. Multiply that by a million rows, and you are asking the process to hold one to three gigabytes at once, before you do any actual work.
Two separate costs are hiding here, and it helps to name them:
The second cost is usually the one that kills the process. Your Postgres server is comfortable streaming a million rows. Your Puma worker or Sidekiq process is not comfortable holding a million Order objects.
find_each exists to solve exactly this problem.
each works on an Active Record relationHere is the important thing to understand: each is not an Active Record method. It is Ruby's Enumerable#each.
When you call it on a relation, Active Record has to hand Ruby something enumerable, so it loads the entire relation first.
User.where(active: true).each do |user|
UserMailer.weekly_digest(user).deliver_later
end
What happens under the hood:
SELECT "users".* FROM "users" WHERE "users"."active" = TRUE.User object per returned row.@records array.The block does not run for the first user until the last user has already been loaded. If the query returns 800,000 active users, you are holding 800,000 objects before a single email is queued.
You can see this in the Rails console. User.where(active: true) on its own is lazy and runs nothing. The moment you call each, map, size on a loaded relation, to_a, or anything else that needs real objects, the query fires and the whole result set materializes.
This distinction matters more than the method name.
If the records are already in memory, each is not doing anything expensive:
users = User.where(active: true).limit(50).to_a
users.each do |user| # plain Ruby, zero queries, nothing to optimize
puts user.email
end
That is an array. It has already been paid for. Iterating it with each is correct, and there is nothing to improve.
But this is different:
User.where(active: true).each { |user| ... } # a relation, one big query
Here each is triggering the query. Same method name, completely different cost profile. When people say "use find_each instead of each," what they really mean is "do not force a huge relation to load all at once."
One practical tell: if a relation is already loaded, calling find_each on it buys you nothing and can cause extra queries. Use each on loaded collections and find_each on relations you have not loaded yet.
find_each does in Railsfind_each is an Active Record method, not a Ruby one. It lives on ActiveRecord::Relation and it iterates records in batches instead of loading them all at once.
User.where(active: true).find_each do |user|
UserMailer.weekly_digest(user).deliver_later
end
From the outside, the block looks identical. It receives one record at a time, exactly like each. The difference is entirely in how those records reach you.
Internally, find_each delegates to find_in_batches, which delegates to in_batches. The loop works like this:
WHERE id > last_id, again limited to the batch size.Once a batch has been yielded, those objects become garbage and Ruby can reclaim them. Memory stays roughly flat no matter how big the table is.
For User.where(active: true).find_each, the queries are roughly:
SELECT "users".* FROM "users"
WHERE "users"."active" = TRUE
ORDER BY "users"."id" ASC
LIMIT 1000;
SELECT "users".* FROM "users"
WHERE "users"."active" = TRUE AND "users"."id" > 1204
ORDER BY "users"."id" ASC
LIMIT 1000;
SELECT "users".* FROM "users"
WHERE "users"."active" = TRUE AND "users"."id" > 2519
ORDER BY "users"."id" ASC
LIMIT 1000;
This technique is called keyset pagination, or cursor pagination. Notice what it does not use: OFFSET. That choice matters. OFFSET 900000 forces the database to walk and discard 900,000 rows before returning anything, so a naive offset-based loop gets slower with every page. A keyset loop stays fast on page one and page nine hundred, because the primary key index takes it straight to the right spot.
This is also a good reminder that your WHERE columns should be indexed. Batching removes the memory problem, not the scanning problem. If active has no index, every batch still scans. If you are not sure how to evaluate that, see the practical guide to database indexes in Rails.
The heart of find_each is a simple trade: more queries, far less memory.
With each on a relation of one million rows:
With find_each on the same relation:
The second column is the one that decides whether your job survives. A thousand small queries against an indexed column are cheap. A million live objects are not.
There is a subtlety worth knowing: memory stays flat only if your block does not accumulate anything. This quietly defeats the whole point:
# Wrong: you batched the query and then rebuilt the giant array anyway
results = []
Order.find_each do |order|
results << order.summary
end
You avoided holding a million Order objects and instead held a million summary objects. If you need aggregate output, write to a file, insert into another table, or stream the results as you go.
# Better: write as you go, hold nothing
CSV.open(Rails.root.join("tmp/orders.csv"), "w") do |csv|
csv << ["id", "email", "total"]
Order.includes(:user).find_each do |order|
csv << [order.id, order.user.email, order.total_cents]
end
end
Note the includes(:user) in that example. Eager loading works normally with find_each, and Rails preloads the associations for each batch rather than for the whole table. Without it, you would trigger one extra query per order, which is the classic N+1 problem described in the guide to optimizing Active Record queries in Rails.
batch_sizeBy default, find_each pulls 1,000 records per batch. You can change that with batch_size.
# Smaller batches: less memory per iteration, more round trips
Order.find_each(batch_size: 250) do |order|
order.recalculate_totals!
end
# Larger batches: fewer round trips, more memory per iteration
Subscriber.find_each(batch_size: 5_000) do |subscriber|
subscriber.update_column(:digest_sent_at, Time.current)
end
How to pick a number:
text columns, serialized JSON, large payloads) want a smaller batch, often 100 to 500. A thousand rows of a wide table can be surprisingly heavy.The default of 1,000 is a sensible starting point. Change it when you have a reason, and measure rather than guess. A quick way to see the difference is to wrap the loop and print GetProcessMem.new.mb or the output of ObjectSpace.count_objects before and after.
One thing batch_size does not do: it does not limit how many records are processed in total. It only controls how many arrive at a time. Every matching record is still visited.
start and finishfind_each accepts start and finish, which bound the iteration by primary key.
# Begin at id 50_000 and keep going to the end of the table
User.find_each(start: 50_000) do |user|
user.backfill_profile_slug!
end
# Process only ids 50_000 through 99_999
User.find_each(start: 50_000, finish: 99_999) do |user|
user.backfill_profile_slug!
end
Both bounds are inclusive, and both refer to the primary key, not to row position or to any other column.
These options are more useful than they look. Common uses:
start: 412_881 instead of reprocessing everything.finish: 1_000 in production to confirm the logic behaves before letting it loose on the full table.Here is the parallel pattern, which pairs naturally with Active Job:
class BackfillSlugsJob < ApplicationJob
queue_as :maintenance
def perform(start_id, finish_id)
User.find_each(start: start_id, finish: finish_id, batch_size: 500) do |user|
user.backfill_profile_slug!
end
end
end
# Enqueue four workers over four id ranges
[[1, 250_000], [250_001, 500_000], [500_001, 750_000], [750_001, 1_000_000]].each do |range|
BackfillSlugsJob.perform_later(*range)
end
If background jobs are new to you, the beginner's guide to Active Job in Rails covers the setup this pattern assumes.
find_each to specific recordsfind_each is a relation method, so every scope, where, join, and association chain you already use still applies. Batching is the last step, not a replacement for your query.
# Any scope chain works
Order.where(status: "pending")
.where(created_at: 6.months.ago..)
.find_each { |order| order.expire_if_stale! }
# Named scopes work
User.inactive.find_each { |user| user.send_winback_email }
# Associations work
account.invoices.unpaid.find_each { |invoice| invoice.send_reminder }
# Joins work
Order.joins(:user)
.where(users: { country: "HT" })
.find_each { |order| order.apply_regional_tax! }
Two habits worth building:
Select only what you need when the loop touches a couple of columns, and the table has heavy ones:
Article.select(:id, :title, :slug).find_each do |article|
RebuildSearchIndexJob.perform_later(article.id)
end
Just remember that touching an unselected attribute on those objects raises ActiveModel::MissingAttributeError, so keep the block narrow. Also keep :id in the select list, since the batching cursor depends on the primary key.
Pass IDs, not objects, to background jobs. Serializing full records into a queue payload is wasteful and can go stale before the job runs.
find_eachThis trips up a lot of people, so it deserves its own section.
find_each requires ordering by the primary key, because the primary key is the cursor that drives the batches. If you supply your own order, Rails discards it and logs a warning:
# The order is ignored; records still arrive by id ascending
User.order(created_at: :desc).find_each { |user| ... }
You can make this loud instead of silent by configuring config.active_record.error_on_ignored_order = true, or per call:
User.order(:created_at).find_each(error_on_ignore: true) { |user| ... }
# raises ArgumentError
Modern Rails does let you flip the direction of the primary key traversal with the order option, which accepts :asc or :desc:
# Newest ids first, still batched
Order.find_each(order: :desc, batch_size: 500) { |order| order.archive! }
What it will not do is batch by an arbitrary column. So if the work genuinely depends on custom ordering, you have three honest choices:
where plus limit), load it with to_a, and use plain each on a set small enough to hold.The important thing is to know that find_each overrode your order clause, rather than shipping a report that silently came out in the wrong sequence.
each vs find_each: memory and performance side by sideConcern |
|
|
|---|---|---|
Queries | One | One per batch |
Peak memory | Proportional to total rows | Proportional to batch size |
Time to first record | After the full result set loads | After the first batch loads |
Custom | Respected | Ignored; ID order is forced |
Risk on a large table | Process killed, timeouts | Low and predictable |
Best size range | Hundreds of rows | Thousands and up |
On raw speed, each can actually win on medium sets, because one query beats fifty. That is not the interesting comparison. find_each is not there to be faster; it is there to be safe at scale and to keep memory bounded. On a small table, the difference is noise. On a large one, each is not slower; it is a crash.
each is perfectly appropriateDo not reach for find_each everywhere. each is the right call when:
@posts.limit(20) or where(id: ids) with a short id list.map, sum, group_by, sort_by, or another Enumerable method that needs the whole collection at once. find_each yields one record at a time and does not build an array for you.find_each in a template. Query in the controller.Paginated pages are the clearest example. A page of 25 records is a loaded collection, and each is exactly right:
<% @articles.each do |article| %>
<%= render article %>
<% end %>
find_each is the better choiceReach for find_each when the set is unbounded or large, and the work is per record:
A simple rule: in a rake task or a job, default to find_each. In a controller or a view, default to each on an already bounded set.
A batch loop over a million rows might take twenty minutes. Your app is still live during those twenty minutes, and this is where find_each has behavior you should understand rather than discover.
Because batching uses WHERE id > last_id and not OFFSET:
id > last_id. Usually harmless. Occasionally dangerous.That last point deserves an example. If the block creates records that also match the scope, the loop can feed itself forever:
# Risky: each processed notification creates another matching notification
Notification.where(processed: false).find_each do |notification|
notification.process!
Notification.create!(user: notification.user, processed: false)
end
Guard against it with finish, which pins the upper bound before the run begins:
max_id = Notification.where(processed: false).maximum(:id)
Notification.where(processed: false).find_each(finish: max_id) do |notification|
notification.process!
end
A few more considerations for long runs:
lock_version column.start and you can resume from the last processed id.find_in_batches is the better fitfind_each yields one record. find_in_batches yields an array of records, one array per batch.
Subscriber.find_in_batches(batch_size: 500) do |subscribers|
MailchimpClient.bulk_upsert(subscribers.map(&:to_sync_payload))
end
Use find_in_batches when the work is naturally batch-shaped:
insert_all or upsert_all with an array of attribute hashes.There is a third member of this family, in_batches, which yields a relation instead of an array. That makes it the right tool for set-based updates, because you can call relation methods on each chunk:
# Update a million rows without instantiating a single object
User.where(onboarded: nil).in_batches(of: 5_000) do |relation|
relation.update_all(onboarded: false)
sleep(0.1) # be kind to the replica lag
end
That last example is worth internalizing. If your loop body is just record.update_column(...) with no logic, you probably do not want a loop at all. update_all inside in_batches does the same work in a fraction of the time, because it never builds Ruby objects. The trade-off is that it skips validations, callbacks, and timestamps, so use it deliberately.
Quick comparison:
find_each yields a record. Use it for per-record logic.find_in_batches yields an array. Use it for chunked work.in_batches yields a relation. Use it for bulk SQL such as update_all or delete_all.Calling each on an unbounded relation. The original sin. Order.all.each is fine in development and fatal in production.
Calling pluck on the whole table to "save memory." User.pluck(:id) avoids model instantiation but still builds an array of every ID in memory. Ten million integers is still a large array. Batch it, or use in_batches with pluck per chunk.
Accumulating inside the block. Batching the query and then appending every result to an array puts the memory right back.
Wrapping the entire run in one transaction. Long locks, replication lag, unhappy DBA.
Ignoring N+1 inside the loop. A million iterations that each fire an association query is a million extra queries. includes works with find_each, so use it.
Assuming your order survived. It did not. Turn on error_on_ignored_order so Rails tells you.
Using find_each for a handful of records. Twenty records do not need batching, and you have added queries and given up ordering for nothing.
Reaching for a loop when SQL would do. Counting, summing, and flat updates belong in the database. Order.sum(:total_cents) beats any Ruby loop you can write.
Forgetting indexes. Batching fixes memory, not table scans. If the scope filters on an unindexed column, every batch pays for it.
Work through these in order:
update_all, delete_all, sum, count, or upsert_all and skip iteration entirely.each.find_each.find_in_batches.in_batches with update_all.And a shorter version for code review: if the relation is not already loaded and you cannot state its maximum size, each is a bug waiting for growth.
each and find_each produce the same records in the same block. The difference is what your process holds in memory while that happens. each loads everything at once, which is perfect for a page of results and disastrous for a table that grows. find_each walks the table in batches of 1,000 by default, keeping memory flat and using the primary key index to stay fast from the first batch to the last.
Keep each for loaded collections and small bounded sets. Reach for find_each in jobs, migrations, and maintenance scripts. Step up to find_in_batches or in_batches when the work is chunked or when SQL can do it better than Ruby.
Pick one long-running loop in your codebase this week, check whether the relation is bounded, and switch it if it is not. Your future on-call self will appreciate it.