Class: SiteFuel::External::AbstractExternalProgram

Inherits:
Object
  • Object
show all
Includes:
Logging
Defined in:
lib/sitefuel/external/AbstractExternalProgram.rb

Overview

lightweight abstraction around a program external to Ruby. The class is designed to make it easy to use an external program in a batch fashion. Note that the abstraction does not well support interacting back and forth with external programs.

Direct Known Subclasses

GIT, JPEGTran, PNGCrush, SVN

Constant Summary

VERSION_SEPARATOR = 
'.'
@@compatible_versions =

cache of whether compatible versions of programs exist

{}
@@program_exists =

cache of whether the actual programs that are abstracted exist

{}
@@program_binary = 
{}
@@program_options = 
{}
@@option_struct = 
Struct.new('ExternalProgramOption', 'name', 'template', 'default')

Instance Attribute Summary (collapse)

Class Method Summary (collapse)

Instance Method Summary (collapse)

Methods included from Logging

#debug, #error, #fatal, #info, #logger=, #warn

Constructor Details

- (AbstractExternalProgram) initialize

A new instance of AbstractExternalProgram



539
540
541
542
543
544
545
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 539

def initialize
  # check that a compatible version exists
  self.class.verify_compatible_version

  self.logger = SiteFuelLogger.instance
  @options = []
end

Instance Attribute Details

- (Object) options (readonly)

INSTANCE METHODS



537
538
539
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 537

def options
  @options
end

Class Method Details

+ (Boolean) allowed_option_name?(name)

tests whether a given option name is allowed

Returns:

  • (Boolean) 


396
397
398
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 396

def self.allowed_option_name?(name)
  not excluded_option_names.include?(name.to_sym)
end

+ (Object) call_option(option_name)

calls an option



364
365
366
367
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 364

def self.call_option(option_name)
  method_name = "option_"+option_name.to_s
  self.send(method_name.to_sym)
end

+ (Object) capture_output(command, *args)

Similar to Kernel#exec, but returns a string of the output



207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 207

def self.capture_output(command, *args)
  cli = command + ' ' + args.join(' ')


  # if we want to capture stderr we need to redirect to stdout
  if capture_stderr
    cli << ' 2>&1'
  end

  output_string = ""
  IO.popen(cli, 'r') do |io|
    output_string = io.read.chop
  end
  output_string
end

+ (Object) capture_stderr

generally we don't want to capture stderr since it helps users with finding out why things don't work. In certain cases we do need to capture it, however.



588
589
590
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 588

def self.capture_stderr
  false
end

+ (Boolean) compatible_version?

gives true if a binary with a compatible version exists

Returns:

  • (Boolean) 


243
244
245
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 243

def self.compatible_version?
  compatible_version_number?(program_version)
end

+ (Boolean) compatible_version_number?(version_number)

gives true if a given version number is compatible

Returns:

  • (Boolean) 


300
301
302
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 300

def self.compatible_version_number?(version_number)
  self.test_version_number(compatible_versions, version_number)
end

+ (Object) compatible_versions

gives a condition on the compatible versions. A version is considered compatible if it's greater than the given version. Eventually we'll probably need a way to give a version range and allow excluding particular versions.



237
238
239
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 237

def self.compatible_versions
  '> 0.0.0'
end

+ (Object) create_tmp_directory(keyword)

creates a temporary directory for sitefuel



522
523
524
525
526
527
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 522

def self.create_tmp_directory(keyword)
  dir_name = File.join(Dir.tmpdir, "sitefuel-#{keyword}-#{random_string}")
  Dir.mkdir(dir_name)

  dir_name
end

+ (Object) ensure_valid_option(name)

raises UnknownOption error if the given option isn't valid



422
423
424
425
426
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 422

def self.ensure_valid_option(name)
  if not option?(name)
    raise UnknownOption.new(self, name)
  end
end

+ (Object) excluded_option_names

list of excluded option names



390
391
392
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 390

def self.excluded_option_names
  [:default, :template]
end

+ (Object) execute(*options)

creates and executes an instance of this external program by taking a list of parameters and their values

self.execute :setflag,                       # set a flag
             :paramsetting, "param value",   # pass a single value
             :paramsetting2, "val1", "val2"  # pass multiple values


503
504
505
506
507
508
509
510
511
512
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 503

def self.execute(*options)
  instance = self.new
  organized = organize_options(*options)

  organized.each do |opt|
    instance.add_option(opt)
  end

  instance.execute
end

+ (Object) extract_program_version(version_output)

given the output of a program gives the version number or nil if not available



344
345
346
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 344

def self.extract_program_version(version_output)
  version_output[/(\d+\.\d+(\.\d+)?([a-zA-Z]+)?)/]
end

+ (Object) option(name, template = nil, default = nil)

declares an option for this program



430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 430

def self.option(name, template = nil, default = nil)
  unless name.kind_of? String
    name = name.to_s
  end

  unless allowed_option_name?(name)
    raise UnallowedOptionName.new(self, name, excluded_option_names)
  end

  # if a default is given but the template has no value slot...
  if default != nil and not template.include?('${value}')
    raise NoOptionValueSlot.new(self, name)
  end


  # give a method for the option
  method_name = "option_"+name
  struct = @@option_struct.new(name, template, default)
  define_class_method(method_name.to_sym) { struct }
end

+ (Boolean) option?(name)

gives true if given a known option

Returns:

  • (Boolean) 


416
417
418
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 416

def self.option?(name)
  self.options.include?(name)
end

+ (Object) option_default(option_name)

gives the default value for an option



402
403
404
405
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 402

def self.option_default(option_name)
  ensure_valid_option(option_name)
  self.call_option(option_name).default
end

+ (Object) option_template(option_name)

gives the template for an option



409
410
411
412
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 409

def self.option_template(option_name)
  ensure_valid_option(option_name)
  self.call_option(option_name).template
end

+ (Object) option_version

option for giving the version of the program



350
351
352
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 350

def self.option_version
  '--version'
end

+ (Object) options

gives the listing of declared options for the program



371
372
373
374
375
376
377
378
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 371

def self.options
  names = methods
  names = names.delete_if { |method| not method =~ /^option_.*$/ }
  names.sort!

  names = names.map { |option_name| option_name.sub(/^option_(.*)$/, '\1').to_sym }
  names - excluded_option_names
end

+ (Object) organize_options(*options)

organizes a list of options into a ragged array of arrays

organize_options(:setflag, :paramsetting, 'val1', 'val2')
# =>[[:setflag], [:paramsetting, 'val1', 'val2']]


456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 456

def self.organize_options(*options)
  organized = []
  i = 0

  while i < options.length
    # if we see a symbol are at a new option
    if options[i].kind_of? Symbol
      option_row = [options[i]]
      organized << option_row

      j = i+1
      while j < options.length
        case options[j]
          when String, Fixnum, Float
            # adds this value
            option_row << options[j]
            j += 1
          
          when Symbol
            # the zoom below will cause this spot to get looked at
            break
          
          else
            # the zoom below will force us to look at this spot again
            # and bail
            break
        end
      end

      # zoom i ahead to this spot
      i = j
    else
      raise MalformedOptions.new(self, options)
    end
  end

  return organized
end

+ (Object) output_handling

controls what happens with the output from the program

capture=:: output is captured into a string and returned from #execute

forward=:: output is forwarded to the terminal as normal



384
385
386
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 384

def self.output_handling
  :capture
end

+ (Object) program_binary

gives the location of the external program; uses the =which= unix command



187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 187

def self.program_binary

  # give the cached version if possible
  cached = @@program_binary[self]
  return cached if cached

  # otherwise try to find it:
  binary = capture_output("which", program_name)
  if binary.empty?
    raise ProgramNotFound.new(program_name)
  else
    # ensure the binary is resolved with respect to the root path
    binary = File.expand_path(binary, capture_output('pwd'))
    @@program_binary[self] = binary
    binary
  end
end

+ (Boolean) program_found?

gives true if the program can be found.

Returns:

  • (Boolean) 


225
226
227
228
229
230
231
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 225

def self.program_found?
  program_binary
rescue ProgramNotFound
  false
else
  true
end

+ (Object) program_version

gets the version of a program



356
357
358
359
360
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 356

def self.program_version
  extract_program_version(capture_output(program_binary, option_version))
rescue ProgramNotFound
  return nil
end

+ (Object) random_string(length = 12)

creates a random string by hashing the current time into hexadecimal



516
517
518
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 516

def self.random_string(length=12)
  Digest::SHA1.hexdigest(Time.now.to_f.to_s)[0, length]
end

+ (Object) test_version_number(compatible, version_number)

tests a version number against a list of compatible version specifications should be made into a Version class. We could also expand the Gem::Version class and use that....



277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 277

def self.test_version_number(compatible, version_number)
  # ensure we're dealing with an array
  version_scheme = compatible
  if not version_scheme.kind_of? Array
    version_scheme = [version_scheme]
  end

  version_scheme.each do |ver|
    case ver[0..0]
      when '>'
        return version_older?(ver[1..-1].strip, version_number)
      when '<'
        return !version_older?(ver[1..-1].strip, version_number)
      else
        # ignore this version spec
    end
  end

  return false
end

+ (Object) verify_compatible_version

raises the ProgramNotFound error if the program can't be found raises the VersionNotFound error if a compatible version isn't found. the verification is cached using a class variable so the verification only actually happens the first time.

Because of the caching this function is generally very fast and should be called by every method that actually will execute the program.



327
328
329
330
331
332
333
334
335
336
337
338
339
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 327

def self.verify_compatible_version
  verify_program_exists

  if @@compatible_versions[self] == nil
    @@compatible_versions[self] = compatible_version?
  end

  if @@compatible_versions[self] == true
    return true
  else
    raise VersionNotFound.new(self, self.compatible_versions, self.program_version)
  end
end

+ (Object) verify_program_exists

raises the ProgramNotFound error if the program can't be found See also AbstractExternalProgram.verify_compatible_version



307
308
309
310
311
312
313
314
315
316
317
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 307

def self.verify_program_exists
  if @@program_exists[self] == nil
    @@program_exists[self] = program_found?
  end
  
  if @@program_exists[self] == true
    return true
  else
    raise ProgramNotFound(self)
  end
end

+ (Boolean) version_older?(lhs, rhs)

gives true if the given version is newer. TODO this should be replaced by a proper version handling library (eg. versionometry (sp?))

Returns:

  • (Boolean) 


251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 251

def self.version_older? (lhs, rhs)
  return true if lhs == rhs

  # split into separate version chunks
  lhs = lhs.split(VERSION_SEPARATOR)
  rhs = rhs.split(VERSION_SEPARATOR)

  # if lhs is shorter than the rhs must be greater than or equal to the
  # lhs; but if the lhs is *longer* than the rhs must be greater than the
  # lhs.
  if lhs.length > rhs.length
    lhs = lhs[0...rhs.length]
    method = :<
  else
    method = :<=
    rhs = rhs[0...lhs.length]
  end

  # now compare
  lhs.join(VERSION_SEPARATOR).send(method, rhs.join(VERSION_SEPARATOR))
end

Instance Method Details

- (Object) add_option(option_row)

adds an option to be passed to this instance



569
570
571
572
573
574
575
576
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 569

def add_option(option_row)
  ensure_option_validity(option_row)

  case option_row
    when Array
      @options << option_row
  end
end

- (Object) apply_value(option_template, value)

applies a given value into an option template



594
595
596
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 594

def apply_value(option_template, value)
  option_template.gsub('${value}', value.to_s)
end

- (Object) build_command_line

builds the command line for a given program instance



619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 619

def build_command_line
  self.class.verify_compatible_version

  exec_string = self.class.program_binary.clone
  @options.each do |option_row|
    option_string = option_template(option_row.first)
    case option_row.length
      when 1
        if takes_value?(option_row.first)
          option_string = apply_value(option_string, self.class.option_default(option_row.first))
        end

      when 2
        option_string = apply_value(option_string, option_row[1])

      else
        option_string = ''
    end
    exec_string << ' ' << option_string
  end

  exec_string
end

- (Object) ensure_option_validity(option_row)

ensures the option specification makes sense



549
550
551
552
553
554
555
556
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 549

def ensure_option_validity(option_row)
  name = option_row.first
  if requires_value?(name) and option_row.length < 2
    raise NoValueForOption.new(self.class, name) 
  end

  true
end

- (Object) execute

executes the given AbstractExternalProgram instance



645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 645

def execute
  exec_string = build_command_line

  info '    Executing: '+exec_string
  output_handler = self.class.output_handling
  case output_handler
    when :capture
      output_string = self.class.capture_output(exec_string)

    when :forward
      output_string = exec(exec_string)

    else
      raise "Unknown output handler: #{output_handler}"
  end

  if $?.success?
    return output_string
  else
    raise ProgramExitedWithFailure.new(self.class, exec_string, $?.to_i)
  end
end

- (Boolean) has_default?(name)

gives true if an option has a default

Returns:

  • (Boolean) 


607
608
609
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 607

def has_default?(name)
  self.class.option_default(name) != nil
end

- (Object) option_template(name)

gives the template for an option



580
581
582
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 580

def option_template(name)
  self.class.option_template(name)
end

- (Boolean) requires_value?(name)

gives true if an option takes a value but has no default

Returns:

  • (Boolean) 


613
614
615
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 613

def requires_value?(name)
  takes_value?(name) and not has_default?(name)
end

- (Boolean) takes_value?(name)

returns true if a given option takes a value TODO this should be precomputed

Returns:

  • (Boolean) 


601
602
603
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 601

def takes_value? (name)
  option_template(name).include?('${value}')
end

- (Boolean) valid_option?(option_row)

gives true if the given option is valid

Returns:

  • (Boolean) 


560
561
562
563
564
565
 
# File 'lib/sitefuel/external/AbstractExternalProgram.rb', line 560

def valid_option?(option_row)
  ensure_option_validity(option_row)
  return true
rescue
  return false
end