When you have a long running block of code, you don’t want to run it inside a web application request cycle. A background processing queuing system is a good solution. There are a number of open source queuing systems available(delayed_job, beanstalk, etc) so you don’t need to write your own! This article will go over how to setup the resque queuing system in a Ruby on Rails application.
Resque setup:
Install redis
brew install redison mac.
or
http://code.google.com/p/redis/
Add resque to your gemfile:
gem "resque"
Install the new gem:
bundle install
Create a redis config file called redis.yml in config:
defaults: &defaults
host: localhost
port: 6379
development:
<<: *defaults
test:
<<: *defaults
staging:
<<: *defaults
production:
<<: *defaults
Add an initializer file called resque.rb in config/initializers:
Dir[File.join(Rails.root, 'app', 'jobs', '*.rb')].each { |file| require file }
config = YAML::load(File.open("#{Rails.root}/config/redis.yml"))[Rails.env]
Resque.redis = Redis.new(:host => config['host'], :port => config['port'])
Add resque.rake to lib/tasks
require 'resque/tasks'
task "resque:setup" => :environment
Running Resque:
start redis:
redis-server
start resque
COUNT=5 QUEUE=* rake resque:workers
see web UI:
resque-web
How to add resque jobs:
Create a job class
class NewsCollectionJob
@queue = :news_collection_job
def self.perform(start_date, end_date)
puts "from #{start_date} to #{end_date}"
#TODO your long running process here
end
end
Run it using:
Resque.enqueue(NewsCollectionJob, start_date, end_date)
This command will not block so you can embed this code in a model. There you go! A few simple steps to getting a faster performing ruby application using background processing on resque/redis.
Let's assume we have a shared Redis platform (e.g. RedisToGo). Meaning the Rails development, test, staging, and production all connect with the same redis-server. Is Resque smart enough to know to only process jobs that originate from that environment?
In the scenario you described, I would install redis locally and then configure the development and test environment blocks of the redis.yml to localhost and the staging/production to your RedisToGo host. This way, you segment your environments and resque will only work from the specified environment. On your production server, be sure to set your environment variable: RAILS_ENV=production
Wonderful post Tommy, many thanks !I've followed your instructions but when I run COUNT=5 QUEUE=* rake resque:workersit fails:(in /home/tex/rails-test/resque-test)(in /home/tex/rails-test/resque-test)(in /home/tex/rails-test/resque-test)(in /home/tex/rails-test/resque-test)(in /home/tex/rails-test/resque-test)(in /home/tex/rails-test/resque-test)rake aborted!You have a nil object when you didn't expect it!You might have expected an instance of Array.The error occurred while evaluating nil.[](See full trace by running task with –trace)rake aborted!You have a nil object when you didn't expect it!You might have expected an instance of Array.The error occurred while evaluating nil.[](See full trace by running task with –trace)rake aborted!You have a nil object when you didn't expect it!You might have expected an instance of Array.The error occurred while evaluating nil.[](See full trace by running task with –trace)rake aborted!You have a nil object when you didn't expect it!You might have expected an instance of Array.The error occurred while evaluating nil.[](See full trace by running task with –trace)rake aborted!You have a nil object when you didn't expect it!You might have expected an instance of Array.The error occurred while evaluating nil.[](See full trace by running task with –trace)running with –trace at the end of command the result is the same… (no stack trace at all..)this happens under:Linux Ubuntu 10.04rvm 1.0.21ruby 1.9.2p0rails 3.0.3resque 1.10.0Any idea ?
Did you create a redis.yml file?
Hi Tommy, your answer drive me into right direction, copying and pasting the file I've misssed the spaces and the redis.yml had a wrong format (my stupid mistake…):I have simply reformat from:defaults: &defaultshost: localhostport: 6379development:<<: *defaultstest:<<: *defaultsstaging:<<: *defaultsproduction:<<: *defaultsto “SPACE” stands form space character:defaults: &defaults”SPACE”host: localhost”SPACE”port: 6379development:”SPACE”<<: *defaultstest:”SPACE”<<: *defaultsstaging:”SPACE”<<: *defaultsproduction:”SPACE”<<: *defaultsNow it works, thank you very much !
Pingback: Llamarada » resque-scheduler, resque, rails integration, redis
Thank you very much for the detailed tutorial, I was wondering if it would be possible to pass a function in the model to be processed as background job.
This is it necessary to have the function written in another class.
Hi,
You can using by using ruby’s send method.
def example(method_name)
NameOfObject.send(method_name)
end