About UsServicesBlogContact
PhoneGithubTwitterLinkedinEmail
background

From DelayedJob to Solid Queue: Migrating a Legacy Rails App

A practical guide to migrating a legacy Ruby on Rails application from DelayedJob to Solid Queue, covering dependency changes, job class conversion, deployment strategy, and common pitfalls.

· 13 minute read
Share:
Share on Facebook Share on Twitter Share on LinkedIn Share by Email

DelayedJob has been a fixture in the Rails ecosystem since 2008. If your application has been around for any length of time, there’s a reasonable chance it’s still running DJ - perhaps because it works, perhaps because nobody has found the time to change it, perhaps both. It’s a pragmatic choice that made a lot of sense for a long time.

Solid Queue, which shipped as a default in Rails 8, represents a meaningfully different approach. It stores jobs in your primary relational database using a schema designed specifically for the purpose, avoids the polling-every-second approach that characterizes DJ, and integrates cleanly with Rails’ built-in ActiveJob interface. For applications already running PostgreSQL or MySQL, it eliminates the need for a separate Redis dependency that many teams added primarily to support Sidekiq.

This guide walks through migrating a legacy Rails application from DelayedJob to Solid Queue. The steps here assume Rails 7.1 or later - Solid Queue’s minimum supported version - and Ruby 3.1.6 or later. If you need to upgrade Rails first, see Ten Steps to Take Before Starting a Rails Upgrade.

Why Migrate at All

Before we get into the mechanics, it’s worth being honest about the trade-offs. DelayedJob isn’t broken, and if your background job load is modest and your team is comfortable with the current setup, there’s no emergency. That said, there are concrete reasons to make the move.

DelayedJob’s polling model - where workers repeatedly query the jobs table for new work - creates steady low-level database load even when the queue is empty. On a busy system, this is a measurable cost. The underlying query looks something like this:

SELECT * FROM delayed_jobs
WHERE (run_at IS NULL OR run_at <= NOW())
  AND (locked_at IS NULL OR locked_at < NOW() - interval '4 hours')
ORDER BY priority ASC, run_at ASC
LIMIT 1

Every worker runs this query on every polling cycle, whether there’s work to do or not. Workers also compete for the same rows, requiring an additional UPDATE to set locked_at before another worker can steal the job.

Solid Queue replaces this with SELECT ... FOR UPDATE SKIP LOCKED, supported in PostgreSQL 9.5+ and MySQL 8+. The operation is atomic: when a worker issues the query, the database acquires a row-level lock on the first available job and skips any rows already locked by other workers. There is no race condition window - two workers cannot claim the same job. Typical claim latency is under 5ms. The practical effect is that workers only touch the database when there is actual work to claim, and horizontal scaling (more workers) doesn’t increase contention.

DJ also lacks native support for several features that modern applications often need: recurring jobs, concurrency controls, job pausing, and a web UI for queue inspection. Solid Queue provides these, along with Mission Control - a Rails engine that gives operators a dashboard for monitoring and intervening in running queues.

Finally, if you’re working toward a Rails 8 upgrade, Solid Queue is the direction the framework is heading. Adopting it now reduces friction later.

Taking Stock Before You Start

The first step is understanding what you actually have. Run a search across your codebase for everything that touches DelayedJob:

$ grep -r "delayed_job\|delay\.\|handle_asynchronously\|Delayed::" app/ --include="*.rb" -l

This will surface:

Make a list. You’ll need to convert each of these patterns, and having a complete inventory prevents surprises late in the process.

Also check your database. DelayedJob stores pending and failed jobs in the delayed_jobs table. Before cutting over, you’ll need to decide what to do with any jobs sitting in that table - run them out with DJ before switching, discard them, or migrate them manually.

SELECT handler, attempts, failed_at, last_error
FROM delayed_jobs
ORDER BY created_at DESC
LIMIT 50;

Adding Solid Queue

Add Solid Queue to your Gemfile and remove the DJ gems:

# Remove these:
# gem 'delayed_job_active_record'
# gem 'delayed_job'

# Add this:
gem 'solid_queue'

Then install:

$ bundle install
$ bin/rails solid_queue:install
$ bin/rails db:migrate

The install task creates the Solid Queue tables (solid_queue_jobs, solid_queue_scheduled_executions, solid_queue_ready_executions, solid_queue_claimed_executions, solid_queue_blocked_executions, solid_queue_failed_executions, solid_queue_processes) and generates two configuration files: config/queue.yml for worker and dispatcher settings, and config/recurring.yml for recurring tasks. These tables live alongside your application tables in the same database, which is intentional - it’s part of the design.

If you’d prefer to use a separate database for queue storage (a reasonable choice for high-volume applications), Solid Queue supports this via Rails’ multi-database configuration. For most legacy apps making this transition, starting with a single database is the simpler path.

Configuring the Queue Adapter

Update config/application.rb (or the appropriate environment files) to use Solid Queue:

config.active_job.queue_adapter = :solid_queue

If you’ve been setting this per-environment, make sure all environments are updated - including test, where you likely want :test or :inline rather than :solid_queue.

Solid Queue’s behavior is controlled by config/queue.yml (generated by the installer). A typical production configuration looks like this:

production:
  dispatchers:
    - polling_interval: 1
      batch_size: 500
  workers:
    - queues: "default,mailers"
      threads: 5
      polling_interval: 0.1
      processes: 2
    - queues: "low_priority"
      threads: 2
      polling_interval: 1
      processes: 1

The workers section maps queue names to process configurations. If your legacy DJ setup used named queues, mirror that structure here. If everything ran on the default queue, the wildcard configuration above is sufficient to start.

Converting Job Classes

This is where most of the work happens. DelayedJob supports several patterns, and each has an ActiveJob equivalent.

Pattern 1: Plain Ruby objects with a perform method

The most common DJ pattern - a plain Ruby class with a perform method passed to Delayed::Job.enqueue:

# Before
class ProcessOrderJob
  def perform(order_id)
    order = Order.find(order_id)
    OrderProcessor.new(order).run
  end
end

Delayed::Job.enqueue(ProcessOrderJob.new(order.id))

Convert to an ActiveJob class:

# After
class ProcessOrderJob < ApplicationJob
  queue_as :default

  def perform(order_id)
    order = Order.find(order_id)
    OrderProcessor.new(order).run
  end
end

ProcessOrderJob.perform_later(order.id)

Note that perform_later passes arguments directly - you don’t instantiate the job object yourself.

Pattern 2: The .delay convenience method

DJ’s .delay method lets you background any method call without writing a job class:

# Before
UserMailer.delay.welcome_email(user.id)
SomeService.delay(queue: 'low_priority').sync_records

ActiveJob doesn’t have a direct equivalent for arbitrary method calls, but ActionMailer has had deliver_later for years:

# After (mailers)
UserMailer.welcome_email(user).deliver_later

For non-mailer .delay usage, you’ll need to create explicit job classes. This is more work, but it’s also more maintainable - implicit job creation from .delay calls makes it harder to understand what’s actually running in your queue.

Pattern 3: handle_asynchronously

# Before
class ReportGenerator
  def generate(report_id)
    # ...
  end
  handle_asynchronously :generate
end

This is another implicit pattern that needs an explicit job class. Extract the body of the method into a job:

# After
class GenerateReportJob < ApplicationJob
  queue_as :default

  def perform(report_id)
    # ...
  end
end

And update the call sites to use GenerateReportJob.perform_later(report_id).

Retries and error handling

DelayedJob automatically retries failed jobs with exponential backoff, up to 25 attempts by default. ActiveJob provides similar functionality, but you configure it explicitly per job class:

class ProcessOrderJob < ApplicationJob
  queue_as :default
  retry_on StandardError, wait: :polynomially_longer, attempts: 10
  discard_on ActiveRecord::RecordNotFound

  def perform(order_id)
    # ...
  end
end

retry_on’s default attempts is 5 - much lower than DJ’s 25 - so explicitly set a count for any jobs where you want DJ-like retry depth. The :polynomially_longer wait strategy uses the formula (executions**4) + jitter, where jitter is 15% by default. At 5 retries this gives waits of roughly 3s, 18s, 83s, 258s, and 627s. discard_on permanently drops jobs that encounter certain errors - useful for cases like a deleted record where retrying will never succeed.

When you need custom logic on discard (logging, alerting), use after_discard (Rails 7.1+):

class ProcessOrderJob < ApplicationJob
  discard_on ActiveRecord::RecordNotFound
  after_discard do |job, error|
    Rails.logger.warn "Discarded #{job.class} #{job.job_id}: #{error.message}"
  end
end

If you need to vary retry wait based on the error itself - for example, respecting a Retry-After header from an API - Rails 8 passes the error as a second argument to proc wait strategies:

retry_on RateLimitError, wait: ->(executions, error) { error.retry_after || executions**2 }

Review your existing DJ job classes for any custom reschedule_at or max_attempts overrides, and translate those into explicit retry_on configuration. The defaults are not equivalent.

Handling Scheduled Jobs

If you’re using run_at to schedule DJ jobs in the future:

# Before
Delayed::Job.enqueue(SomeJob.new, run_at: 1.hour.from_now)

ActiveJob handles this with set:

# After
SomeJob.set(wait: 1.hour).perform_later
# Or with a specific time:
SomeJob.set(wait_until: 1.hour.from_now).perform_later

Solid Queue stores scheduled jobs in solid_queue_scheduled_executions and moves them to the ready queue when their scheduled time arrives. The dispatcher process handles this - make sure your process configuration is starting a dispatcher alongside your workers.

Recurring Jobs

If you’re using a gem like delayed_cron_job or clockwork to handle periodic tasks, Solid Queue has native support for recurring jobs via config/recurring.yml (generated by the installer):

# config/recurring.yml
production:
  daily_report:
    class: DailyReportJob
    schedule: every day at 6am
    args: []
  weekly_digest:
    class: WeeklyDigestJob
    schedule: every monday at 9am

This eliminates the need for an external scheduler process for many common cases. For complex scheduling needs, gems like whenever can still write entries to cron that enqueue ActiveJob jobs.

Running Solid Queue

DelayedJob workers were typically started with:

$ bundle exec rake jobs:work

Solid Queue provides its own process supervisor:

$ bin/jobs start

This starts the supervisor, which then manages dispatcher and worker processes according to your configuration file. In production, you’ll want to run this under a process supervisor like systemd or whatever you’re already using to manage your DJ worker processes.

A minimal systemd unit for Solid Queue:

[Unit]
Description=Solid Queue
After=network.target

[Service]
Type=simple
WorkingDirectory=/var/www/myapp/current
ExecStart=/var/www/myapp/current/bin/jobs start
Restart=on-failure
RestartSec=5
User=deploy
Environment=RAILS_ENV=production

[Install]
WantedBy=multi-user.target

If you’re using Capistrano, update your deployment tasks to restart the Solid Queue service rather than the DJ worker process.

The Cutover

Draining an existing DJ queue cleanly before switching over avoids losing in-flight work. The simplest approach:

  1. Stop enqueueing new DJ jobs (deploy the code changes with DJ still running)
  2. Let the existing DJ workers drain the queue
  3. Verify the delayed_jobs table is empty (or contains only jobs you’re willing to discard)
  4. Stop the DJ workers
  5. Start Solid Queue
-- Verify the queue is empty
SELECT COUNT(*) FROM delayed_jobs WHERE failed_at IS NULL;

If you have long-running jobs or a large backlog, this drain period may take a while. Plan the cutover window accordingly. For applications that can’t afford downtime, you can run both systems simultaneously - new jobs go to Solid Queue, old jobs continue to drain through DJ - until the DJ queue is empty. This requires keeping both gems in your Gemfile temporarily.

Removing DelayedJob

Once Solid Queue is running cleanly in production, remove the DJ gems from your Gemfile and the delayed_jobs table from your database:

# Remove from Gemfile
# gem 'delayed_job_active_record'
# gem 'delayed_job'
$ bundle install

Create a migration to drop the old table:

class RemoveDelayedJobsTable < ActiveRecord::Migration[7.1]
  def up
    drop_table :delayed_jobs
  end

  def down
    raise ActiveRecord::IrreversibleMigration
  end
end

Don’t rush this step. Keep the table around until you’re confident the cutover was successful and there’s no remaining code that references Delayed::Job directly.

What to Watch After Cutover

Once Solid Queue is running in production, monitor a few things closely in the first days:

Queue depth - Are jobs being processed, or is the queue growing? Solid Queue’s Mission Control dashboard (if installed) shows this clearly. Alternatively, query directly:

SELECT queue_name, COUNT(*) as pending
FROM solid_queue_ready_executions
GROUP BY queue_name;

Failed jobs - Check solid_queue_failed_executions for any jobs that exhausted their retry attempts:

SELECT class_name, exception_executions, created_at
FROM solid_queue_failed_executions
ORDER BY created_at DESC
LIMIT 20;

Worker process health - solid_queue_processes tracks registered worker and dispatcher processes. If a process crashes, its entry will remain until the heartbeat expires. This table is useful for confirming your expected number of workers are online.

Database connection pool - each Solid Queue worker thread holds a database connection. With threads: 5 and processes: 2, that’s 10 connections from your worker process alone, on top of your web server’s pool. If you see ActiveRecord::ConnectionTimeoutError after cutover, increase pool: in config/database.yml. A rule of thumb: pool should be at least (worker_threads * worker_processes) + web_concurrency + a few for overhead.

A Note on Mission Control

Mission Control is a Rails engine that provides a web dashboard for Solid Queue. Adding it is optional, but worth considering if your team regularly needs to inspect or retry failed jobs:

gem 'mission_control-jobs'

Mount it in your routes:

mount MissionControl::Jobs::Engine, at: "/jobs"

Mission Control ships with HTTP basic authentication enabled by default. To wire up your application’s existing authentication instead, configure a custom base controller:

# config/application.rb
config.mission_control.jobs.base_controller_class = "AdminController"
config.mission_control.jobs.http_basic_auth_enabled = false

Mission Control will inherit any before_action filters on that controller - your existing authentication checks will apply without additional wiring. Mission Control provides visibility into queued, scheduled, failed, and in-progress jobs - functionality that previously required custom tooling or direct database queries. It requires Solid Queue 1.0.1 or later.

Practical Implications

The migration itself is straightforward for most applications. The effort scales with how heavily you’ve used DJ’s implicit patterns - .delay calls and handle_asynchronously scattered across a large codebase require more mechanical conversion work than explicit job classes do.

The payoff is a more transparent system. Every background operation becomes an explicit job class with documented retry behavior. The queue state lives in your application database, visible through standard tooling. And the dependency on Redis - if you were only using it for job processing - goes away.

For teams working toward Rails 8 compatibility, or simply looking to reduce operational complexity, Solid Queue is a reasonable destination.

Resources