Class: ETL::Processor::FtpDownloaderProcessor

Inherits:

Processor

• Object
• Processor
• ETL::Processor::FtpDownloaderProcessor

show all

Defined in:

lib/etl/processor/ftp_downloader_processor.rb

Overview

Instance Attribute Summary (collapse)

• - (Object) files readonly

  Returns the value of attribute files.

• - (Object) host readonly

  Returns the value of attribute host.

• - (Object) local_dir readonly

  Returns the value of attribute local_dir.

• - (Object) port readonly

  Returns the value of attribute port.

• - (Object) remote_dir readonly

  Returns the value of attribute remote_dir.

• - (Object) username readonly

  Returns the value of attribute username.

Instance Method Summary (collapse)

• - (FtpDownloaderProcessor) initialize(control, configuration) constructor

  configuration options include:

  • host - hostname or IP address of FTP server (required)

  • port - port number for FTP server (default: 21)

  • remote_dir - remote path on FTP server (default: /)

  • files - list of files to download from FTP server (default: [])

  • username - username for FTP server authentication (default: anonymous)

  • password - password for FTP server authentication (default: nil)

  • local_dir - local output directory to save downloaded files (default: '').

• - (Object) process

pre_process :ftp_downloader, {
  :host => 'ftp.sec.gov',
  :path => 'edgar/Feed/2007/QTR2',
  :files => ['20070402.nc.tar.gz', '20070403.nc.tar.gz', '20070404.nc.tar.gz', 
             '20070405.nc.tar.gz', '20070406.nc.tar.gz'],
  :local_dir => '/data/sec/2007/04',
}

36
37
38
39
40
41
42
43
44

# File 'lib/etl/processor/ftp_downloader_processor.rb', line 36

def initialize(control, configuration)
  @host = configuration[:host]
  @port = configuration[:port] || 21
  @remote_dir = configuration[:remote_dir] || '/'
  @files = configuration[:files] || []
  @username = configuration[:username] || 'anonymous'
  @password = configuration[:password]
  @local_dir = configuration[:local_dir] || ''
end

13
14
15

# File 'lib/etl/processor/ftp_downloader_processor.rb', line 13

def files
  @files
end

10
11
12

# File 'lib/etl/processor/ftp_downloader_processor.rb', line 10

def host
  @host
end

15
16
17

# File 'lib/etl/processor/ftp_downloader_processor.rb', line 15

def local_dir
  @local_dir
end

11
12
13

# File 'lib/etl/processor/ftp_downloader_processor.rb', line 11

def port
  @port
end

12
13
14

# File 'lib/etl/processor/ftp_downloader_processor.rb', line 12

def remote_dir
  @remote_dir
end

14
15
16

# File 'lib/etl/processor/ftp_downloader_processor.rb', line 14

def username
  @username
end

46
47
48
49
50
51
52
53
54

# File 'lib/etl/processor/ftp_downloader_processor.rb', line 46

def process
  Net::FTP.open(@host) do |conn|
    conn.connect(@host, @port)
    conn.login(@username, @password)
    @files.each do |f|
      conn.getbinaryfile(remote_file(f), local_file(f))
    end
  end
end