Class: CloudCrowd::CommandLine

Inherits:
Object
  • Object
show all
Defined in:
lib/cloud_crowd/command_line.rb

Constant Summary

CONFIG_FILES =

Configuration files required for the `crowd` command to function.

['config.yml', 'config.ru', 'database.yml']
CC_ROOT =

Reference the absolute path to the root.

File.expand_path(File.dirname(__FILE__) + '/../..')
<<-EOS
CloudCrowd is a MapReduce-inspired Parallel Processing System for Ruby.

Wiki: http://wiki.github.com/documentcloud/cloud-crowd
Rdoc: http://rdoc.info/projects/documentcloud/cloud-crowd

Usage: crowd COMMAND OPTIONS

Commands:
  install       Install the CloudCrowd configuration files to the specified directory
  server        Start up the central server (requires a database)
  node          Start up a worker node (only one node per machine, please)
  console       Launch a CloudCrowd console, connected to the central database
  load_schema   Load the schema into the database specified by database.yml
  cleanup       Removes jobs that were finished over --days (7 by default) ago

  server -d [start | stop | restart]    Servers and nodes can be launched as
  node -d [start | stop | restart]      daemons, then stopped or restarted.

Options:
EOS

Instance Method Summary (collapse)

Constructor Details

- (CommandLine) initialize

Creating a CloudCrowd::CommandLine runs from the contents of ARGV.



36
37
38
39
40
41
42
43
44
45
46
47
48
49
 
# File 'lib/cloud_crowd/command_line.rb', line 36

def initialize
  parse_options
  command     = ARGV.shift
  subcommand  = ARGV.shift
  case command
  when 'console'      then run_console
  when 'server'       then run_server(subcommand)
  when 'node'         then run_node(subcommand)
  when 'load_schema'  then run_load_schema
  when 'install'      then run_install(subcommand)
  when 'cleanup'      then run_cleanup
  else                     usage
  end
end

Instance Method Details

- (Object) config_not_found (private)

Exit with an explanation if the configuration files couldn't be found.



228
229
230
231
 
# File 'lib/cloud_crowd/command_line.rb', line 228

def config_not_found
  puts "`crowd` can't find the CloudCrowd configuration directory. Please use `crowd -c path/to/config`, or run `crowd` from inside of the configuration directory itself."
  exit(1)
end

- (Object) connect_to_database(validate_schema) (private)

Establish a connection to the central server's database. Not all commands require this.



222
223
224
225
 
# File 'lib/cloud_crowd/command_line.rb', line 222

def connect_to_database(validate_schema)
  require 'cloud_crowd/models'
  CloudCrowd.configure_database("#{@options[:config_path]}/database.yml", validate_schema)
end

- (Object) ensure_config (private)

Check for configuration files, either in the current directory, or in the CLOUD_CROWD_CONFIG environment variable. Exit if they're not found.



169
170
171
172
173
 
# File 'lib/cloud_crowd/command_line.rb', line 169

def ensure_config
  return if @config_found
  found = CONFIG_FILES.all? {|f| File.exists? "#{@options[:config_path]}/#{f}" }
  found ? @config_dir = true : config_not_found
end

- (Object) install_file(source, dest, is_dir = false) (private)

Install a file and log the installation. If we're overwriting a file, offer a chance to back out.



235
236
237
238
239
240
241
242
 
# File 'lib/cloud_crowd/command_line.rb', line 235

def install_file(source, dest, is_dir=false)
  if File.exists?(dest)
    print "#{dest} already exists. Overwrite it? (yes/no) "
    return unless ['y', 'yes', 'ok'].include? gets.chomp.downcase
  end
  is_dir ? FileUtils.cp_r(source, dest) : FileUtils.cp(source, dest)
  puts "installed #{dest}" unless ENV['RACK_ENV'] == 'test'
end

- (Object) load_code (private)

Load in the CloudCrowd module code, dependencies, lib files and models. Not all commands require this.



214
215
216
217
218
 
# File 'lib/cloud_crowd/command_line.rb', line 214

def load_code
  ensure_config
  require "#{CC_ROOT}/lib/cloud-crowd"
  CloudCrowd.configure("#{@options[:config_path]}/config.yml")
end

- (Object) parse_options (private)

Parse all options for all commands. Valid options are: –config –port –environment –tag –daemonize –days.



177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
 
# File 'lib/cloud_crowd/command_line.rb', line 177

def parse_options
  @options = {
    :environment  => 'production',
    :config_path  => ENV['CLOUD_CROWD_CONFIG'] || '.',
    :daemonize    => false
  }
  @option_parser = OptionParser.new do |opts|
    opts.on('-c', '--config PATH', 'path to configuration directory') do |conf_path|
      @options[:config_path] = conf_path
    end
    opts.on('-p', '--port PORT', 'port number for server (central or node)') do |port_num|
      @options[:port] = port_num
    end
    opts.on('-e', '--environment ENV', 'server environment (defaults to production)') do |env|
      @options[:environment] = env
    end
    opts.on('-t', '--tag TAG', 'tag a node with a name') do |tag|
      @options[:tag] = tag
    end
    opts.on('-d', '--daemonize', 'run as a background daemon') do |daemonize|
      @options[:daemonize] = daemonize
    end
    opts.on('--days NUM_DAYS', 'grace period before cleanup (7 by default)') do |days|
      @options[:days] = days.to_i if days.match(/\A\d+\Z/)
    end
    opts.on_tail('-v', '--version', 'show version') do
      require "#{CC_ROOT}/lib/cloud-crowd"
      puts "CloudCrowd version #{VERSION}"
      exit
    end
  end
  @option_parser.banner = BANNER
  @option_parser.parse!(ARGV)
end

- (Object) restart_node

Restart the daemonized Node, if it exists.



127
128
129
130
131
 
# File 'lib/cloud_crowd/command_line.rb', line 127

def restart_node
  stop_node
  sleep 1
  start_node
end

- (Object) restart_server

Restart the daemonized central server.



96
97
98
99
100
 
# File 'lib/cloud_crowd/command_line.rb', line 96

def restart_server
  stop_server
  sleep 1
  start_server
end

- (Object) run_cleanup

Clean up all Jobs in the CloudCrowd database older than –days old.



153
154
155
156
157
 
# File 'lib/cloud_crowd/command_line.rb', line 153

def run_cleanup
  load_code
  connect_to_database(true)
  Job.cleanup_all(:days => @options[:days])
end

- (Object) run_console

Spin up an IRB session with the CloudCrowd code loaded in, and a database connection established. The equivalent of Rails' `script/console`.



53
54
55
56
57
58
59
60
61
62
 
# File 'lib/cloud_crowd/command_line.rb', line 53

def run_console
  require 'irb'
  require 'irb/completion'
  require 'pp'
  load_code
  connect_to_database true
  CloudCrowd::Server # Preload server to autoload classes.
  Object.send(:include, CloudCrowd)
  IRB.start
end

- (Object) run_install(install_path)

Install the required CloudCrowd configuration files into the specified directory, or the current one.



142
143
144
145
146
147
148
149
150
 
# File 'lib/cloud_crowd/command_line.rb', line 142

def run_install(install_path)
  require 'fileutils'
  install_path ||= '.'
  FileUtils.mkdir_p install_path unless File.exists?(install_path)
  install_file "#{CC_ROOT}/config/config.example.yml", "#{install_path}/config.yml"
  install_file "#{CC_ROOT}/config/config.example.ru", "#{install_path}/config.ru"
  install_file "#{CC_ROOT}/config/database.example.yml", "#{install_path}/database.yml"
  install_file "#{CC_ROOT}/actions", "#{install_path}/actions", true
end

- (Object) run_load_schema

Load in the database schema to the database specified in 'database.yml'.



134
135
136
137
138
 
# File 'lib/cloud_crowd/command_line.rb', line 134

def run_load_schema
  load_code
  connect_to_database(false)
  require 'cloud_crowd/schema.rb'
end

- (Object) run_node(subcommand)

`crowd node` can either 'start', 'stop', or 'restart'.



103
104
105
106
107
108
109
110
111
 
# File 'lib/cloud_crowd/command_line.rb', line 103

def run_node(subcommand)
  load_code
  ENV['RACK_ENV'] = @options[:environment]
  case (subcommand || 'start')
  when 'start'    then start_node
  when 'stop'     then stop_node
  when 'restart'  then restart_node
  end
end

- (Object) run_server(subcommand)

`crowd server` can either 'start', 'stop', or 'restart'.



65
66
67
68
69
70
71
72
73
 
# File 'lib/cloud_crowd/command_line.rb', line 65

def run_server(subcommand)
  load_code
  subcommand ||= 'start'
  case subcommand
  when 'start'    then start_server
  when 'stop'     then stop_server
  when 'restart'  then restart_server
  end
end

- (Object) start_node

Launch a Node. Please only run a single node per machine. The Node process will be long-lived, although its workers will come and go.



115
116
117
118
119
 
# File 'lib/cloud_crowd/command_line.rb', line 115

def start_node
  @options[:port] ||= Node::DEFAULT_PORT
  puts "Starting CloudCrowd Node on port #{@options[:port]}..."
  Node.new(@options)
end

- (Object) start_server

Convenience command for quickly spinning up the central server. More sophisticated deployments, load-balancing across multiple app servers, should use the config.ru rackup file directly. This method will start a single Thin server.



79
80
81
82
83
84
85
86
87
88
 
# File 'lib/cloud_crowd/command_line.rb', line 79

def start_server
  port        = @options[:port] || 9173
  daemonize   = @options[:daemonize] ? '-d' : ''
  log_path    = CloudCrowd.log_path('server.log')
  pid_path    = CloudCrowd.pid_path('server.pid')
  rackup_path = File.expand_path("#{@options[:config_path]}/config.ru")
  FileUtils.mkdir_p(CloudCrowd.log_path) if @options[:daemonize] && !File.exists?(CloudCrowd.log_path)
  puts "Starting CloudCrowd Central Server on port #{port}..."
  exec "thin -e #{@options[:environment]} -p #{port} #{daemonize} --tag cloud-crowd-server --log #{log_path} --pid #{pid_path} -R #{rackup_path} start"
end

- (Object) stop_node

If the daemonized Node is running, stop it.



122
123
124
 
# File 'lib/cloud_crowd/command_line.rb', line 122

def stop_node
  Thin::Server.kill CloudCrowd.pid_path('node.pid')
end

- (Object) stop_server

Stop the daemonized central server, if it exists.



91
92
93
 
# File 'lib/cloud_crowd/command_line.rb', line 91

def stop_server
  Thin::Server.kill(CloudCrowd.pid_path('server.pid'), 0)
end

- (Object) usage

Print `crowd` usage.



160
161
162
 
# File 'lib/cloud_crowd/command_line.rb', line 160

def usage
  puts "\n#{@option_parser}\n"
end