Put a progress meter in your long-running migrations

Uploaded with plasq's Skitch!

I'm working on a Rails project now that requires lots of database massaging and repair. This repair work needs to be tried and tested on a development workstation, reviewed on a staging server, then applied to the production system. Since the work needs to be repeatedly applied to several environments, I'm logically using migrations.

One nuisance I got sick of real quick is staring at a terminal running a long migration and wondering "is it doing anything?" and "how much longer is it going to take?" So I decided to add some progress indicators.

The simple yet effective method I've settled on is an on-screen count-down. Most of the migrations consist of the same pattern: 1) get a list of the records that need repaired, then 2) iterate over each record and repair it. I set the counter to the size of the record set I'll be iterating over, decrement it on each iteration, and print it to the screen. Seeing the numbers scroll across the screen lets me know the migration is working, and since the migrations count down to zero, I can gauge how long it's going to take to complete based on how fast the numbers are shrinking.

Here's an example:

class FixTheThingsWithTheStuff < ActiveRecord::Migration
  def self.up    
    query = <<-SQL
      select name
      from things
      where stuff = 1972
      and deleted_at is null
      group by name
      having count(id) > 1
      order by name
    SQL
    broken_rows = select_all(query)
    count = broken_rows.size
    broken_rows.each do |row|
      printf "[#{count-=1}]"
      # ... fix it! ...
    end
  end

  def self.down
    # ... re-break it! ...
  end
end