Module: Resque::Plugins::Retry

Included in:
ExponentialBackoff
Defined in:
lib/resque/plugins/retry.rb

Overview

If you want your job to retry on failure, simply extend your module/class with this module:

class DeliverWebHook
  extend Resque::Plugins::Retry # allows 1 retry by default.
  @queue = :web_hooks

  def self.perform(url, hook_id, hmac_key)
    heavy_lifting
  end
end

Easily do something custom:

class DeliverWebHook
  extend Resque::Plugins::Retry
  @queue = :web_hooks

  @retry_limit = 8  # default: 1
  @retry_delay = 60 # default: 0

  # used to build redis key, for counting job attempts.
  def self.retry_identifier(url, hook_id, hmac_key)
    "#{url}-#{hook_id}"
  end

  def self.perform(url, hook_id, hmac_key)
    heavy_lifting
  end
end

Instance Method Summary (collapse)

Instance Method Details

- (Object) after_perform_retry(*args)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Resque after_perform hook

Deletes retry attempt count from Redis.



295
296
297
 
# File 'lib/resque/plugins/retry.rb', line 295

def after_perform_retry(*args)
  clean_retry_key(*args)
end

- (Array) args_for_retry(*args)

This method is abstract.

Modify the arguments used to retry the job. Use this to do something other than try the exact same job again

Returns:

  • (Array) —

    new job arguments



141
142
143
 
# File 'lib/resque/plugins/retry.rb', line 141

def args_for_retry(*args)
  args
end

- (Object) before_perform_retry(*args)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Resque before_perform hook

Increments and sets the `@retry_attempt` count.



281
282
283
284
285
286
287
288
 
# File 'lib/resque/plugins/retry.rb', line 281

def before_perform_retry(*args)
  @on_failure_retry_hook_already_called = false

  # store number of retry attempts.
  retry_key = redis_retry_key(*args)
  Resque.redis.setnx(retry_key, -1)             # default to -1 if not set.
  @retry_attempt = Resque.redis.incr(retry_key) # increment by 1.
end

- (Object) clean_retry_key(*args)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Clean up retry state from redis once done



340
341
342
 
# File 'lib/resque/plugins/retry.rb', line 340

def clean_retry_key(*args)
  Resque.redis.del(redis_retry_key(*args))
end

- (Object) inherited(subclass)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Copy retry criteria checks on inheritance.



42
43
44
45
 
# File 'lib/resque/plugins/retry.rb', line 42

def inherited(subclass)
  super(subclass)
  subclass.instance_variable_set("@retry_criteria_checks", retry_criteria_checks.dup)
end

- (Object) instance_exec(*args, &block)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Used to perform retry criteria check blocks under the job instance's context

Returns:

  • (Object) —

    return value of the criteria check



326
327
328
329
330
331
332
333
334
335
 
# File 'lib/resque/plugins/retry.rb', line 326

def instance_exec(*args, &block)
  mname = "__instance_exec_#{Thread.current.object_id.abs}"
  class << self; self end.class_eval{ define_method(mname, &block) }
  begin
    ret = send(mname, *args)
  ensure
    class << self; self end.class_eval{ undef_method(mname) } rescue nil
  end
  ret
end

- (Object) on_failure_retry(exception, *args)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Note:

This hook will only allow execution once per job perform attempt. This was added because Resque v1.20.0 calls the hook twice. IMO; this isn't something resque-retry should have to worry about!

Resque on_failure hook

Checks if our retry criteria is valid, if it is we try again. Otherwise the retry attempt count is deleted from Redis.



309
310
311
312
313
314
315
316
317
318
319
 
# File 'lib/resque/plugins/retry.rb', line 309

def on_failure_retry(exception, *args)
  return if @on_failure_retry_hook_already_called

  if retry_criteria_valid?(exception, *args)
    try_again(exception, *args)
  else
    clean_retry_key(*args)
  end

  @on_failure_retry_hook_already_called = true
end

- (String) redis_retry_key(*args)

Builds the redis key to be used for keeping state of the job attempts.

Returns:

  • (String) —

    redis key



69
70
71
 
# File 'lib/resque/plugins/retry.rb', line 69

def redis_retry_key(*args)
  ['resque-retry', name, retry_identifier(*args)].compact.join(":").gsub(/\s/, '')
end

- (Fixnum) retry_attempt

Number of retry attempts used to try and perform the job

The real value is kept in Redis, it is accessed and incremented using a before_perform hook.

Returns:

  • (Fixnum) —

    number of attempts



93
94
95
 
# File 'lib/resque/plugins/retry.rb', line 93

def retry_attempt
  @retry_attempt ||= 0
end

- (Object) retry_criteria_check {|exception, *args| ... }

Register a retry criteria check callback to be run before retrying the job again

If any callback returns `true`, the job will be retried.

Examples:

Using a custom retry criteria check.


retry_criteria_check do |exception, *args|
  if exception.message =~ /InvalidJobId/
    # don't retry if we got passed a invalid job id.
    false
  else
    true
  end
end

Yields:

  • (exception, *args)

Yield Parameters:

  • exception (Exception) —

    the exception that was raised

  • args (Array) —

    job arguments

Yield Returns:

  • (Boolean) —

    false == dont retry, true = can retry



249
250
251
 
# File 'lib/resque/plugins/retry.rb', line 249

def retry_criteria_check(&block)
  retry_criteria_checks << block
end

- (Array) retry_criteria_checks

Retry criteria checks

Returns:

  • (Array) 


208
209
210
 
# File 'lib/resque/plugins/retry.rb', line 208

def retry_criteria_checks
  @retry_criteria_checks ||= []
end

- (Boolean) retry_criteria_valid?(exception, *args)

Test if the retry criteria is valid

Parameters:

  • exception (Exception)
  • args (Array) —

    job arguments

Returns:

  • (Boolean) 


188
189
190
191
192
193
194
195
196
197
198
199
200
201
 
# File 'lib/resque/plugins/retry.rb', line 188

def retry_criteria_valid?(exception, *args)
  # if the retry limit was reached, dont bother checking anything else.
  return false if retry_limit_reached?

  # We always want to retry if the exception matches.
  should_retry = retry_exception?(exception)

  # call user retry criteria check blocks.
  retry_criteria_checks.each do |criteria_check|
    should_retry ||= !!instance_exec(exception, *args, &criteria_check)
  end

  should_retry
end

- (Number) retry_delay(exception_class = nil)

This method is abstract.

Number of seconds to delay until the job is retried

Returns:

  • (Number) —

    number of seconds to delay



103
104
105
106
107
108
109
110
111
 
# File 'lib/resque/plugins/retry.rb', line 103

def retry_delay(exception_class = nil)
  if @retry_exceptions.is_a?(Hash)
    delay = @retry_exceptions[exception_class] || 0
    # allow an array of delays.
    delay.is_a?(Array) ? delay[retry_attempt] || delay.last : delay
  else
    @retry_delay ||= 0
  end
end

- (Boolean) retry_exception?(exception)

Convenience method to test whether you may retry on a given exception

also be a Class

Parameters:

  • an (Exception) —

    instance of Exception. Deprecated: can

Returns:

  • (Boolean) 


154
155
156
157
158
159
160
161
162
163
 
# File 'lib/resque/plugins/retry.rb', line 154

def retry_exception?(exception)
  return true if retry_exceptions.nil?
  !! retry_exceptions.any? do |ex|
    if exception.is_a?(Class)
      ex >= exception
    else
      ex === exception
    end
  end
end

- (Array?) retry_exceptions

This method is abstract.

Controls what exceptions may be retried

Default: `nil` - this will retry all exceptions.

Returns:

  • (Array, nil) 


173
174
175
176
177
178
179
 
# File 'lib/resque/plugins/retry.rb', line 173

def retry_exceptions
  if @retry_exceptions.is_a?(Hash)
    @retry_exceptions.keys
  else
    @retry_exceptions ||= nil
  end
end

- (String) retry_identifier(*args)

This method is abstract.

You may override to implement a custom retry identifier, you should consider doing this if your job arguments are many/long or may not cleanly convert to strings.

Builds a retry identifier using the job arguments. This identifier is used as part of the redis key

Parameters:

  • args (Array) —

    job arguments

Returns:

  • (String) —

    job identifier



58
59
60
61
 
# File 'lib/resque/plugins/retry.rb', line 58

def retry_identifier(*args)
  args_string = args.join('-')
  args_string.empty? ? nil : Digest::SHA1.hexdigest(args_string)
end

- (Object?) retry_job_delegate

This method is abstract.

Specify another resque job (module or class) to delegate retry duties to upon failure

Returns:

  • (Object, nil) —

    class or module if delegate on failure, otherwise nil



130
131
132
 
# File 'lib/resque/plugins/retry.rb', line 130

def retry_job_delegate
  @retry_job_delegate ||= nil
end

- (Fixnum) retry_limit

Maximum number of retrys we can attempt to successfully perform the job

A retry limit of 0 will never retry. A retry limit of -1 or below will retry forever.

Returns:

  • (Fixnum) 


81
82
83
 
# File 'lib/resque/plugins/retry.rb', line 81

def retry_limit
  @retry_limit ||= 1
end

- (Boolean) retry_limit_reached?

Test if the retry limit has been reached

Returns:

  • (Boolean) 


217
218
219
220
221
222
223
224
225
 
# File 'lib/resque/plugins/retry.rb', line 217

def retry_limit_reached?
  if retry_limit == 0
    true
  elsif retry_limit > 0
    true if retry_attempt >= retry_limit
  else
    false
  end
end

- (Number) sleep_after_requeue

This method is abstract.

Number of seconds to sleep after job is requeued

Returns:

  • (Number) —

    number of seconds to sleep



119
120
121
 
# File 'lib/resque/plugins/retry.rb', line 119

def sleep_after_requeue
  @sleep_after_requeue ||= 0
end

- (Object) try_again(exception, *args)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Retries the job



256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
 
# File 'lib/resque/plugins/retry.rb', line 256

def try_again(exception, *args)
  # some plugins define retry_delay and have it take no arguments, so rather than break those,
  # we'll just check here to see whether it takes the additional exception class argument or not
  temp_retry_delay = ([-1, 1].include?(method(:retry_delay).arity) ? retry_delay(exception.class) : retry_delay)

  retry_in_queue = retry_job_delegate ? retry_job_delegate : self
  if temp_retry_delay <= 0
    # If the delay is 0, no point passing it through the scheduler
    Resque.enqueue(retry_in_queue, *args_for_retry(*args))
  else
    Resque.enqueue_in(temp_retry_delay, retry_in_queue, *args_for_retry(*args))
  end

  # remove retry key from redis if we handed retry off to another queue.
  clean_retry_key(*args) if retry_job_delegate

  # sleep after requeue if enabled.
  sleep(sleep_after_requeue) if sleep_after_requeue > 0
end