Class: CSV

Inherits:
Object show all
Extended by:
Forwardable
Includes:
Enumerable
Defined in:
lib/csv.rb

Overview

This class provides a complete interface to CSV files and data. It offers tools to enable you to read and write to and from Strings or IO objects, as needed.

Reading

From a File

A Line at a Time

CSV.foreach("path/to/file.csv") do |row|
  # use row here...
end

All at Once

arr_of_arrs = CSV.read("path/to/file.csv")

From a String

A Line at a Time

CSV.parse("CSV,data,String") do |row|
  # use row here...
end

All at Once

arr_of_arrs = CSV.parse("CSV,data,String")

Writing

To a File

CSV.open("path/to/file.csv", "wb") do |csv|
  csv << ["row", "of", "CSV", "data"]
  csv << ["another", "row"]
  # ...
end

To a String

csv_string = CSV.generate do |csv|
  csv << ["row", "of", "CSV", "data"]
  csv << ["another", "row"]
  # ...
end

Convert a Single Line

csv_string = ["CSV", "data"].to_csv   # to CSV
csv_array  = "CSV,String".parse_csv   # from CSV

Shortcut Interface

CSV             { |csv_out| csv_out << %w{my data here} }  # to $stdout
CSV(csv = "")   { |csv_str| csv_str << %w{my data here} }  # to a String
CSV($stderr)    { |csv_err| csv_err << %w{my data here} }  # to $stderr

CSV and Character Encodings (M17n or Multilingualization)

This new CSV parser is m17n savvy. The parser works in the Encoding of the IO or String object being read from or written to. Your data is never transcoded (unless you ask Ruby to transcode it for you) and will literally be parsed in the Encoding it is in. Thus CSV will return Arrays or Rows of Strings in the Encoding of your data. This is accomplished by transcoding the parser itself into your Encoding.

Some transcoding must take place, of course, to accomplish this multiencoding support. For example, :col_sep, :row_sep, and :quote_char must be transcoded to match your data. Hopefully this makes the entire process feel transparent, since CSV's defaults should just magically work for you data. However, you can set these values manually in the target Encoding to avoid the translation.

It's also important to note that while all of CSV's core parser is now Encoding agnostic, some features are not. For example, the built-in converters will try to transcode data to UTF-8 before making conversions. Again, you can provide custom converters that are aware of your Encodings to avoid this translation. It's just too hard for me to support native conversions in all of Ruby's Encodings.

Anyway, the practical side of this is simple: make sure IO and String objects passed into CSV have the proper Encoding set and everything should just work. CSV methods that allow you to open IO objects (CSV::foreach(), CSV::open(), CSV::read(), and CSV::readlines()) do allow you to specify the Encoding.

One minor exception comes when generating CSV into a String with an Encoding that is not ASCII compatible. There's no existing data for CSV to use to prepare itself and thus you will probably need to manually specify the desired Encoding for most of those cases. It will try to guess using the fields in a row of output though, when using CSV::generate_line() or Array#to_csv().

I try to point out any other Encoding issues in the documentation of methods as they come up.

This has been tested to the best of my ability with all non-"dummy" Encodings Ruby ships with. However, it is brave new code and may have some bugs. Please feel free to report any issues you find with it.

Defined Under Namespace

Classes: FieldInfo, MalformedCSVError, Row, Table

Constant Summary

VERSION =

The version of the installed library.

"2.4.7".freeze
DateMatcher =

A Regexp used to find and convert some common Date formats.

/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2},?\s+\d{2,4} |
\d{4}-\d{2}-\d{2} )\z /x
DateTimeMatcher =

A Regexp used to find and convert some common DateTime formats.

/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2}\s+\d{1,2}:\d{1,2}:\d{1,2},?\s+\d{2,4} |
\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2} )\z /x
ConverterEncoding =

The encoding used by all converters.

Encoding.find("UTF-8")
Converters =

This Hash holds the built-in converters of CSV that can be accessed by name. You can select Converters with CSV.convert() or through the options Hash passed to CSV::new().

:integer

Converts any field Integer() accepts.

:float

Converts any field Float() accepts.

:numeric

A combination of :integer and :float.

:date

Converts any field Date::parse() accepts.

:date_time

Converts any field DateTime::parse() accepts.

:all

All built-in converters. A combination of :date_time and :numeric.

All built-in converters transcode field data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the field will remain unchanged.

This Hash is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all CSV objects.

To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

{ integer:   lambda { |f|
  Integer(f.encode(ConverterEncoding)) rescue f
},
float:     lambda { |f|
  Float(f.encode(ConverterEncoding)) rescue f
},
numeric:   [:integer, :float],
date:      lambda { |f|
  begin
    e = f.encode(ConverterEncoding)
    e =~ DateMatcher ? Date.parse(e) : f
  rescue  # encoding conversion or date parse errors
    f
  end
},
date_time: lambda { |f|
  begin
    e = f.encode(ConverterEncoding)
    e =~ DateTimeMatcher ? DateTime.parse(e) : f
  rescue  # encoding conversion or date parse errors
    f
  end
},
all:       [:date_time, :numeric] }
HeaderConverters =

This Hash holds the built-in header converters of CSV that can be accessed by name. You can select HeaderConverters with CSV.header_convert() or through the options Hash passed to CSV::new().

:downcase

Calls downcase() on the header String.

:symbol

The header String is downcased, spaces are replaced with underscores, non-word characters are dropped, and finally to_sym() is called.

All built-in header converters transcode header data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the header will remain unchanged.

This Hash is intetionally left unfrozen and users should feel free to add values to it that can be accessed by all CSV objects.

To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

{
  downcase: lambda { |h| h.encode(ConverterEncoding).downcase },
  symbol:   lambda { |h|
    h.encode(ConverterEncoding).downcase.gsub(/\s+/, "_").
                                         gsub(/\W+/, "").to_sym
  }
}
DEFAULT_OPTIONS =

The options used when no overrides are given by calling code. They are:

:col_sep

","

:row_sep

:auto

:quote_char

'"'

:field_size_limit

nil

:converters

nil

:unconverted_fields

nil

:headers

false

:return_headers

false

:header_converters

nil

:skip_blanks

false

:force_quotes

false

{ col_sep:            ",",
row_sep:            :auto,
quote_char:         '"',
field_size_limit:   nil,
converters:         nil,
unconverted_fields: nil,
headers:            false,
return_headers:     false,
header_converters:  nil,
skip_blanks:        false,
force_quotes:       false }.freeze

Constants included from Forwardable

Forwardable::FORWARDABLE_VERSION

Instance Attribute Summary (collapse)

Class Method Summary (collapse)

Instance Method Summary (collapse)

Methods included from Forwardable

def_instance_delegator, def_instance_delegators, instance_delegate

Methods included from Enumerable

#to_set

Constructor Details

- (CSV) initialize(data, options = Hash.new)

This constructor will wrap either a String or IO object passed in data for reading and/or writing. In addition to the CSV instance methods, several IO methods are delegated. (See CSV::open() for a complete list.) If you pass a String for data, you can later retrieve it (after writing to it, for example) with CSV.string().

Note that a wrapped String will be positioned at at the beginning (for reading). If you want it at the end (for writing), use CSV::generate(). If you want any other positioning, pass a preset StringIO object instead.

You may set any reading and/or writing preferences in the options Hash. Available options are:

:col_sep

The String placed between each field. This String will be transcoded into the data's Encoding before parsing.

:row_sep

The String appended to the end of each row. This can be set to the special :auto setting, which requests that CSV automatically discover this from the data. Auto-discovery reads ahead in the data looking for the next "\r\n", "\n", or "\r" sequence. A sequence will be selected even if it occurs in a quoted field, assuming that you would have the same line endings there. If none of those sequences is found, data is ARGF, STDIN, STDOUT, or STDERR, or the stream is only available for output, the default $INPUT_RECORD_SEPARATOR ($/) is used. Obviously, discovery takes a little time. Set manually if speed is important. Also note that IO objects should be opened in binary mode on Windows if this feature will be used as the line-ending translation can cause problems with resetting the document position to where it was before the read ahead. This String will be transcoded into the data's Encoding before parsing.

:quote_char

The character used to quote fields. This has to be a single character String. This is useful for application that incorrectly use ' as the quote character instead of the correct ". CSV will always consider a double sequence this character to be an escaped quote. This String will be transcoded into the data's Encoding before parsing.

:field_size_limit

This is a maximum size CSV will read ahead looking for the closing quote for a field. (In truth, it reads to the first line ending beyond this size.) If a quote cannot be found within the limit CSV will raise a MalformedCSVError, assuming the data is faulty. You can use this limit to prevent what are effectively DoS attacks on the parser. However, this limit can cause a legitimate parse to fail and thus is set to nil, or off, by default.

:converters

An Array of names from the Converters Hash and/or lambdas that handle custom conversion. A single converter doesn't have to be in an Array. All built-in converters try to transcode fields to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the field unchanged.

:unconverted_fields

If set to true, an unconverted_fields() method will be added to all returned rows (Array or CSV::Row) that will return the fields as they were before conversion. Note that :headers supplied by Array or String were not fields of the document and thus will have an empty Array attached.

:headers

If set to :first_row or true, the initial row of the CSV file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of CSV::parse_line() with the same :col_sep, :row_sep, and :quote_char as this instance to produce an Array of headers. This setting causes CSV#shift() to return rows as CSV::Row objects instead of Arrays and CSV#read() to return CSV::Table objects instead of an Array of Arrays.

:return_headers

When false, header rows are silently swallowed. If set to true, header rows are returned in a CSV::Row object with identical headers and fields (save that the fields do not go through the converters).

:write_headers

When true and :headers is set, a header row will be added to the output.

:header_converters

Identical in functionality to :converters save that the conversions are only made to header rows. All built-in converters try to transcode headers to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the header unchanged.

:skip_blanks

When set to a true value, CSV will skip over any rows with no content.

:force_quotes

When set to a true value, CSV will quote all CSV fields it creates.

See CSV::DEFAULT_OPTIONS for the default settings.

Options cannot be overriden in the instance methods for performance reasons, so be sure to set what you want here.



1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
 
# File 'lib/csv.rb', line 1552

def initialize(data, options = Hash.new)
  # build the options for this read/write
  options = DEFAULT_OPTIONS.merge(options)

  # create the IO object we will read from
  @io       =   if data.is_a? String then StringIO.new(data) else data end
  # honor the IO encoding if we can, otherwise default to ASCII-8BIT
  @encoding = raw_encoding || Encoding.default_internal || Encoding.default_external
  #
  # prepare for building safe regular expressions in the target encoding,
  # if we can transcode the needed characters
  #
  @re_esc   =   "\\".encode(@encoding) rescue ""
  @re_chars =   %w[ \\ .  [  ]  -  ^  $  ?
                    *  +  {  }  (  )  |  #
                    \  \r \n \t \f \v ].
                map { |s| s.encode(@encoding) rescue nil }.compact

  init_separators(options)
  init_parsers(options)
  init_converters(options)
  init_headers(options)

  unless options.empty?
    raise ArgumentError, "Unknown options:  #{options.keys.join(', ')}."
  end

  # track our own lineno since IO gets confused about line-ends is CSV fields
  @lineno = 0
end

Instance Attribute Details

- (Object) col_sep (readonly)

The encoded :col_sep used in parsing and writing. See CSV::new for details.



1587
1588
1589
 
# File 'lib/csv.rb', line 1587

def col_sep
  @col_sep
end

- (Object) encoding (readonly)

The Encoding CSV is parsing or writing in. This will be the Encoding you receive parsed data in and/or the Encoding data will be written in.



1654
1655
1656
 
# File 'lib/csv.rb', line 1654

def encoding
  @encoding
end

- (Object) field_size_limit (readonly)

The limit for field size, if any. See CSV::new for details.



1599
1600
1601
 
# File 'lib/csv.rb', line 1599

def field_size_limit
  @field_size_limit
end

- (Object) lineno (readonly)

The line number of the last row read from this file. Fields with nested line-end characters will not affect this count.



1660
1661
1662
 
# File 'lib/csv.rb', line 1660

def lineno
  @lineno
end

- (Object) quote_char (readonly)

The encoded :quote_char used in parsing and writing. See CSV::new for details.



1597
1598
1599
 
# File 'lib/csv.rb', line 1597

def quote_char
  @quote_char
end

- (Object) row_sep (readonly)

The encoded :row_sep used in parsing and writing. See CSV::new for details.



1592
1593
1594
 
# File 'lib/csv.rb', line 1592

def row_sep
  @row_sep
end

Class Method Details

+ (Object) dump(ary_of_objs, io = "", options = Hash.new)

This method allows you to serialize an Array of Ruby objects to a String or File of CSV data. This is not as powerful as Marshal or YAML, but perhaps useful for spreadsheet and database interaction.

Out of the box, this method is intended to work with simple data objects or Structs. It will serialize a list of instance variables and/or Struct.members().

If you need need more complicated serialization, you can control the process by adding methods to the class to be serialized.

A class method csv_meta() is responsible for returning the first row of the document (as an Array). This row is considered to be a Hash of the form key_1,value_1,key_2,value_2,... CSV::load() expects to find a class key with a value of the stringified class name and CSV::dump() will create this, if you do not define this method. This method is only called on the first object of the Array.

The next method you can provide is an instance method called csv_headers(). This method is expected to return the second line of the document (again as an Array), which is to be used to give each column a header. By default, CSV::load() will set an instance variable if the field header starts with an @ character or call send() passing the header as the method name and the field value as an argument. This method is only called on the first object of the Array.

Finally, you can provide an instance method called csv_dump(), which will be passed the headers. This should return an Array of fields that can be serialized for this object. This method is called once for every object in the Array.

The io parameter can be used to serialize to a File, and options can be anything CSV::new() accepts.



1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
 
# File 'lib/csv.rb', line 1040

def self.dump(ary_of_objs, io = "", options = Hash.new)
  obj_template = ary_of_objs.first

  csv = new(io, options)

  # write meta information
  begin
    csv << obj_template.class.csv_meta
  rescue NoMethodError
    csv << [:class, obj_template.class]
  end

  # write headers
  begin
    headers = obj_template.csv_headers
  rescue NoMethodError
    headers = obj_template.instance_variables.sort
    if obj_template.class.ancestors.find { |cls| cls.to_s =~ /\AStruct\b/ }
      headers += obj_template.members.map { |mem| "#{mem}=" }.sort
    end
  end
  csv << headers

  # serialize each object
  ary_of_objs.each do |obj|
    begin
      csv << obj.csv_dump(headers)
    rescue NoMethodError
      csv << headers.map do |var|
        if var[0] == ?@
          obj.instance_variable_get(var)
        else
          obj[var[0..-2]]
        end
      end
    end
  end

  if io.is_a? String
    csv.string
  else
    csv.close
  end
end

+ (Object) filter(*args)

:call-seq:

filter( options = Hash.new ) { |row| ... }
filter( input, options = Hash.new ) { |row| ... }
filter( input, output, options = Hash.new ) { |row| ... }

This method is a convenience for building Unix-like filters for CSV data. Each row is yielded to the provided block which can alter it as needed. After the block returns, the row is appended to output altered or not.

The input and output arguments can be anything CSV::new() accepts (generally String or IO objects). If not given, they default to ARGF and $stdout.

The options parameter is also filtered down to CSV::new() after some clever key parsing. Any key beginning with :in_ or :input_ will have that leading identifier stripped and will only be used in the options Hash for the input object. Keys starting with :out_ or :output_ affect only output. All other keys are assigned to both objects.

The :output_row_sep option defaults to $INPUT_RECORD_SEPARATOR ($/).



1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
 
# File 'lib/csv.rb', line 1156

def self.filter(*args)
  # parse options for input, output, or both
  in_options, out_options = Hash.new, {row_sep: $INPUT_RECORD_SEPARATOR}
  if args.last.is_a? Hash
    args.pop.each do |key, value|
      case key.to_s
      when /\Ain(?:put)?_(.+)\Z/
        in_options[$1.to_sym] = value
      when /\Aout(?:put)?_(.+)\Z/
        out_options[$1.to_sym] = value
      else
        in_options[key]  = value
        out_options[key] = value
      end
    end
  end
  # build input and output wrappers
  input  = new(args.shift || ARGF,    in_options)
  output = new(args.shift || $stdout, out_options)

  # read, yield, write
  input.each do |row|
    yield row
    output << row
  end
end

+ (Object) foreach(path, options = Hash.new, &block)

This method is intended as the primary interface for reading CSV files. You pass a path and any options you wish to set for the read. Each row of file will be passed to the provided block in turn.

The options parameter can be anything CSV::new() understands. This method also understands an additional :encoding parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding::default_external(). CSV will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8" would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.



1197
1198
1199
1200
1201
1202
1203
1204
 
# File 'lib/csv.rb', line 1197

def self.foreach(path, options = Hash.new, &block)
  encoding =  options.delete(:encoding)
  mode     =  "rb"
  mode     << ":#{encoding}" if encoding
  open(path, mode, options) do |csv|
    csv.each(&block)
  end
end

+ (Object) generate(*args) {|csv| ... }

:call-seq:

generate( str, options = Hash.new ) { |csv| ... }
generate( options = Hash.new ) { |csv| ... }

This method wraps a String you provide, or an empty default String, in a CSV object which is passed to the provided block. You can use the block to append CSV rows to the String and when the block exits, the final String will be returned.

Note that a passed String is modfied by this method. Call dup() before passing if you need a new String.

The options parameter can be anything CSV::new() understands. This method understands an additional :encoding parameter when not passed a String to set the base Encoding for the output. CSV needs this hint if you plan to output non-ASCII compatible data.

Yields:

  • (csv) 


1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
 
# File 'lib/csv.rb', line 1224

def self.generate(*args)
  # add a default empty String, if none was given
  if args.first.is_a? String
    io = StringIO.new(args.shift)
    io.seek(0, IO::SEEK_END)
    args.unshift(io)
  else
    encoding = args.last.is_a?(Hash) ? args.last.delete(:encoding) : nil
    str      = ""
    str.encode!(encoding) if encoding
    args.unshift(str)
  end
  csv = new(*args)  # wrap
  yield csv         # yield for appending
  csv.string        # return final String
end

+ (Object) generate_line(row, options = Hash.new)

This method is a shortcut for converting a single row (Array) into a CSV String.

The options parameter can be anything CSV::new() understands. This method understands an additional :encoding parameter to set the base Encoding for the output. This method will try to guess your Encoding from the first non-nil field in row, if possible, but you may need to use this parameter as a backup plan.

The :row_sep option defaults to $INPUT_RECORD_SEPARATOR ($/) when calling this method.



1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
 
# File 'lib/csv.rb', line 1254

def self.generate_line(row, options = Hash.new)
  options  = {row_sep: $INPUT_RECORD_SEPARATOR}.merge(options)
  encoding = options.delete(:encoding)
  str      = ""
  if encoding
    str.force_encoding(encoding)
  elsif field = row.find { |f| not f.nil? }
    str.force_encoding(String(field).encoding)
  end
  (new(str, options) << row).string
end

+ (Object) instance(data = $stdout, options = Hash.new)

This method will return a CSV instance, just like CSV::new(), but the instance will be cached and returned for all future calls to this method for the same data object (tested by Object#object_id()) with the same options.

If a block is given, the instance is passed to the block and the return value becomes the return value of the block.



989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
 
# File 'lib/csv.rb', line 989

def self.instance(data = $stdout, options = Hash.new)
  # create a _signature_ for this method call, data object and options
  sig = [data.object_id] +
        options.values_at(*DEFAULT_OPTIONS.keys.sort_by { |sym| sym.to_s })

  # fetch or create the instance for this signature
  @@instances ||= Hash.new
  instance    =   (@@instances[sig] ||= new(data, options))

  if block_given?
    yield instance  # run block, if given, returning result
  else
    instance        # or return the instance
  end
end

+ (Object) load(io_or_str, options = Hash.new)

This method is the reading counterpart to CSV::dump(). See that method for a detailed description of the process.

You can customize loading by adding a class method called csv_load() which will be passed a Hash of meta information, an Array of headers, and an Array of fields for the object the method is expected to return.

Remember that all fields will be Strings after this load. If you need something else, use options to setup converters or provide a custom csv_load() implementation.



1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
 
# File 'lib/csv.rb', line 1097

def self.load(io_or_str, options = Hash.new)
  csv = new(io_or_str, options)

  # load meta information
  meta = Hash[*csv.shift]
  cls  = meta["class".encode(csv.encoding)].split("::".encode(csv.encoding)).
                                            inject(Object) do |c, const|
    c.const_get(const)
  end

  # load headers
  headers = csv.shift

  # unserialize each object stored in the file
  results = csv.inject(Array.new) do |all, row|
    begin
      obj = cls.csv_load(meta, headers, row)
    rescue NoMethodError
      obj = cls.allocate
      headers.zip(row) do |name, value|
        if name[0] == ?@
          obj.instance_variable_set(name, value)
        else
          obj.send(name, value)
        end
      end
    end
    all << obj
  end

  csv.close unless io_or_str.is_a? String

  results
end

+ (Object) open(*args)

:call-seq:

open( filename, mode = "rb", options = Hash.new ) { |faster_csv| ... }
open( filename, options = Hash.new ) { |faster_csv| ... }
open( filename, mode = "rb", options = Hash.new )
open( filename, options = Hash.new )

This method opens an IO object, and wraps that with CSV. This is intended as the primary interface for writing a CSV file.

You must pass a filename and may optionally add a mode for Ruby's open(). You may also pass an optional Hash containing any options CSV::new() understands as the final argument.

This method works like Ruby's open() call, in that it will pass a CSV object to a provided block and close it when the block terminates, or it will return the CSV object when no block is provided. (Note: This is different from the Ruby 1.8 CSV library which passed rows to the block. Use CSV::foreach() for that behavior.)

You must provide a mode with an embedded Encoding designator unless your data is in Encoding::default_external(). CSV will check the Encoding of the underlying IO object (set by the mode you pass) to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read just as you can with a normal call to IO::open(). For example, "rb:UTF-32BE:UTF-8" would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.

An opened CSV object will delegate to many IO methods for convenience. You may call:

  • binmode()

  • binmode?()

  • close()

  • close_read()

  • close_write()

  • closed?()

  • eof()

  • eof?()

  • external_encoding()

  • fcntl()

  • fileno()

  • flock()

  • flush()

  • fsync()

  • internal_encoding()

  • ioctl()

  • isatty()

  • path()

  • pid()

  • pos()

  • pos=()

  • reopen()

  • seek()

  • stat()

  • sync()

  • sync=()

  • tell()

  • to_i()

  • to_io()

  • truncate()

  • tty?()



1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
 
# File 'lib/csv.rb', line 1329

def self.open(*args)
  # find the +options+ Hash
  options = if args.last.is_a? Hash then args.pop else Hash.new end
  # default to a binary open mode
  args << "rb" if args.size == 1
  # wrap a File opened with the remaining +args+
  csv     = new(File.open(*args), options)

  # handle blocks like Ruby's open(), not like the CSV library
  if block_given?
    begin
      yield csv
    ensure
      csv.close
    end
  else
    csv
  end
end

+ (Object) parse(*args, &block)

:call-seq:

parse( str, options = Hash.new ) { |row| ... }
parse( str, options = Hash.new )

This method can be used to easily parse CSV out of a String. You may either provide a block which will be called with each row of the String in turn, or just use the returned Array of Arrays (when no block is given).

You pass your str to read from, and an optional options Hash containing anything CSV::new() understands.



1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
 
# File 'lib/csv.rb', line 1361

def self.parse(*args, &block)
  csv = new(*args)
  if block.nil?  # slurp contents, if no block is given
    begin
      csv.read
    ensure
      csv.close
    end
  else           # or pass each row to a provided block
    csv.each(&block)
  end
end

+ (Object) parse_line(line, options = Hash.new)

This method is a shortcut for converting a single line of a CSV String into a into an Array. Note that if line contains multiple rows, anything beyond the first row is ignored.

The options parameter can be anything CSV::new() understands.



1381
1382
1383
 
# File 'lib/csv.rb', line 1381

def self.parse_line(line, options = Hash.new)
  new(line, options).shift
end

+ (Object) read(path, options = Hash.new)

Use to slurp a CSV file into an Array of Arrays. Pass the path to the file and any options CSV::new() understands. This method also understands an additional :encoding parameter that you can use to specify the Encoding of the data in the file to be read. You must provide this unless your data is in Encoding::default_external(). CSV will use this to determine how to parse the data. You may provide a second Encoding to have the data transcoded as it is read. For example, encoding: "UTF-32BE:UTF-8" would read UTF-32BE data from the file but transcode it to UTF-8 before CSV parses it.



1396
1397
1398
1399
1400
1401
 
# File 'lib/csv.rb', line 1396

def self.read(path, options = Hash.new)
  encoding =  options.delete(:encoding)
  mode     =  "rb"
  mode     << ":#{encoding}" if encoding
  open(path, mode, options) { |csv| csv.read }
end

+ (Object) readlines(*args)

Alias for CSV::read().



1404
1405
1406
 
# File 'lib/csv.rb', line 1404

def self.readlines(*args)
  read(*args)
end

+ (Object) table(path, options = Hash.new)

A shortcut for:

CSV.read( path, { headers:           true,
                  converters:        :numeric,
                  header_converters: :symbol }.merge(options) )


1415
1416
1417
1418
1419
 
# File 'lib/csv.rb', line 1415

def self.table(path, options = Hash.new)
  read( path, { headers:           true,
                converters:        :numeric,
                header_converters: :symbol }.merge(options) )
end

Instance Method Details

- (Object) <<(row) Also known as: add_row, puts

The primary write method for wrapped Strings and IOs, row (an Array or CSV::Row) is converted to CSV and appended to the data source. When a CSV::Row is passed, only the row's fields() are appended to the output.

The data source must be open for writing.



1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
 
# File 'lib/csv.rb', line 1689

def <<(row)
  # make sure headers have been assigned
  if header_row? and [Array, String].include? @use_headers.class
    parse_headers  # won't read data for Array or String
    self << @headers if @write_headers
  end

  # handle CSV::Row objects and Hashes
  row = case row
        when self.class::Row then row.fields
        when Hash            then @headers.map { |header| row[header] }
        else                      row
        end

  @headers =  row if header_row?
  @lineno  += 1

  output = row.map(&@quote).join(@col_sep) + @row_sep  # quote and separate
  if @io.is_a?(StringIO)             and
     output.encoding != raw_encoding and
     (compatible_encoding = Encoding.compatible?(@io.string, output))
    @io = StringIO.new(@io.string.force_encoding(compatible_encoding))
    @io.seek(0, IO::SEEK_END)
  end
  @io << output

  self  # for chaining
end

- (Object) convert(name = nil, &converter)

:call-seq:

convert( name )
convert { |field| ... }
convert { |field, field_info| ... }

You can use this method to install a CSV::Converters built-in, or provide a block that handles a custom conversion.

If you provide a block that takes one argument, it will be passed the field and is expected to return the converted value or the field itself. If your block takes two arguments, it will also be passed a CSV::FieldInfo Struct, containing details about the field. Again, the block should return a converted field or the field itself.



1735
1736
1737
 
# File 'lib/csv.rb', line 1735

def convert(name = nil, &converter)
  add_converter(:converters, self.class::Converters, name, &converter)
end

- (Object) converters

Returns the current list of converters in effect. See CSV::new for details. Built-in converters will be returned by name, while others will be returned as is.



1605
1606
1607
1608
1609
1610
 
# File 'lib/csv.rb', line 1605

def converters
  @converters.map do |converter|
    name = Converters.rassoc(converter)
    name ? name.first : converter
  end
end

- (Object) each

Yields each row of the data source in turn.

Support for Enumerable.

The data source must be open for reading.



1766
1767
1768
1769
1770
 
# File 'lib/csv.rb', line 1766

def each
  while row = shift
    yield row
  end
end

- (Boolean) force_quotes?

Returns true if all output fields are quoted. See CSV::new for details.

Returns:

  • (Boolean) 


1648
 
# File 'lib/csv.rb', line 1648

def force_quotes?()       @force_quotes       end

- (Object) header_convert(name = nil, &converter)

:call-seq:

header_convert( name )
header_convert { |field| ... }
header_convert { |field, field_info| ... }

Identical to CSV#convert(), but for header rows.

Note that this method must be called before header rows are read to have any effect.



1750
1751
1752
1753
1754
1755
 
# File 'lib/csv.rb', line 1750

def header_convert(name = nil, &converter)
  add_converter( :header_converters,
                 self.class::HeaderConverters,
                 name,
                 &converter )
end

- (Object) header_converters

Returns the current list of converters in effect for headers. See CSV::new for details. Built-in converters will be returned by name, while others will be returned as is.



1636
1637
1638
1639
1640
1641
 
# File 'lib/csv.rb', line 1636

def header_converters
  @header_converters.map do |converter|
    name = HeaderConverters.rassoc(converter)
    name ? name.first : converter
  end
end

- (Boolean) header_row?

Returns true if the next row read will be a header row.

Returns:

  • (Boolean) 


1788
1789
1790
 
# File 'lib/csv.rb', line 1788

def header_row?
  @use_headers and @headers.nil?
end

- (Object) headers

Returns nil if headers will not be used, true if they will but have not yet been read, or the actual headers after they have been read. See CSV::new for details.



1621
1622
1623
 
# File 'lib/csv.rb', line 1621

def headers
  @headers || true if @use_headers
end

- (Object) inspect

Returns a simplified description of the key CSV attributes in an ASCII compatible String.



1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
 
# File 'lib/csv.rb', line 1944

def inspect
  str = ["<#", self.class.to_s, " io_type:"]
  # show type of wrapped IO
  if    @io == $stdout then str << "$stdout"
  elsif @io == $stdin  then str << "$stdin"
  elsif @io == $stderr then str << "$stderr"
  else                      str << @io.class.to_s
  end
  # show IO.path(), if available
  if @io.respond_to?(:path) and (p = @io.path)
    str << " io_path:" << p.inspect
  end
  # show encoding
  str << " encoding:" << @encoding.name
  # show other attributes
  %w[ lineno     col_sep     row_sep
      quote_char skip_blanks ].each do |attr_name|
    if a = instance_variable_get("@#{attr_name}")
      str << " " << attr_name << ":" << a.inspect
    end
  end
  if @use_headers
    str << " headers:" << headers.inspect
  end
  str << ">"
  begin
    str.join
  rescue  # any encoding error
    str.map do |s|
      e = Encoding::Converter.asciicompat_encoding(s.encoding)
      e ? s.encode(e) : s.force_encoding("ASCII-8BIT")
    end.join
  end
end

- (Object) read Also known as: readlines

Slurps the remaining rows and returns an Array of Arrays.

The data source must be open for reading.



1777
1778
1779
1780
1781
1782
1783
1784
 
# File 'lib/csv.rb', line 1777

def read
  rows = to_a
  if @use_headers
    Table.new(rows)
  else
    rows
  end
end

- (Boolean) return_headers?

Returns true if headers will be returned as a row of results. See CSV::new for details.

Returns:

  • (Boolean) 


1628
 
# File 'lib/csv.rb', line 1628

def return_headers?()     @return_headers     end

- (Object) rewind

Rewinds the underlying IO object and resets CSV's lineno() counter.



1673
1674
1675
1676
1677
1678
 
# File 'lib/csv.rb', line 1673

def rewind
  @headers = nil
  @lineno  = 0

  @io.rewind
end

- (Object) shift Also known as: gets, readline

The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an Array of fields (if header rows are not used) or a CSV::Row (when header rows are used).

The data source must be open for reading.



1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
 
# File 'lib/csv.rb', line 1799

def shift
  #########################################################################
  ### This method is purposefully kept a bit long as simple conditional ###
  ### checks are faster than numerous (expensive) method calls.         ###
  #########################################################################

  # handle headers not based on document content
  if header_row? and @return_headers and
     [Array, String].include? @use_headers.class
    if @unconverted_fields
      return add_unconverted_fields(parse_headers, Array.new)
    else
      return parse_headers
    end
  end

  # begin with a blank line, so we can always add to it
  line = ""

  #
  # it can take multiple calls to <tt>@io.gets()</tt> to get a full line,
  # because of \r and/or \n characters embedded in quoted fields
  #
  in_extended_col = false
  csv             = Array.new

  loop do
    # add another read to the line
    unless parse = @io.gets(@row_sep)
      return nil
    end

    parse.sub!(@parsers[:line_end], "")

    if csv.empty?
      #
      # I believe a blank line should be an <tt>Array.new</tt>, not Ruby 1.8
      # CSV's <tt>[nil]</tt>
      #
      if parse.empty?
        @lineno += 1
        if @skip_blanks
          next
        elsif @unconverted_fields
          return add_unconverted_fields(Array.new, Array.new)
        elsif @use_headers
          return self.class::Row.new(Array.new, Array.new)
        else
          return Array.new
        end
      end
    end

    parts =  parse.split(@col_sep, -1)
    if parts.empty?
      if in_extended_col
        csv[-1] << @col_sep   # will be replaced with a @row_sep after the parts.each loop
      else
        csv << nil
      end
    end

    # This loop is the hot path of csv parsing. Some things may be non-dry
    # for a reason. Make sure to benchmark when refactoring.
    parts.each do |part|
      if in_extended_col
        # If we are continuing a previous column
        if part[-1] == @quote_char && part.count(@quote_char) % 2 != 0
          # extended column ends
          csv.last << part[0..-2]
          raise MalformedCSVError if csv.last =~ @parsers[:stray_quote]
          csv.last.gsub!(@quote_char * 2, @quote_char)
          in_extended_col = false
        else
          csv.last << part
          csv.last << @col_sep
        end
      elsif part[0] == @quote_char
        # If we are staring a new quoted column
        if part[-1] != @quote_char || part.count(@quote_char) % 2 != 0
          # start an extended column
          csv             << part[1..-1]
          csv.last        << @col_sep
          in_extended_col =  true
        else
          # regular quoted column
          csv << part[1..-2]
          raise MalformedCSVError if csv.last =~ @parsers[:stray_quote]
          csv.last.gsub!(@quote_char * 2, @quote_char)
        end
      elsif part =~ @parsers[:quote_or_nl]
        # Unquoted field with bad characters.
        if part =~ @parsers[:nl_or_lf]
          raise MalformedCSVError, "Unquoted fields do not allow " +
                                   "\\r or \\n (line #{lineno + 1})."
        else
          raise MalformedCSVError, "Illegal quoting on line #{lineno + 1}."
        end
      else
        # Regular ole unquoted field.
        csv << (part.empty? ? nil : part)
      end
    end

    # Replace tacked on @col_sep with @row_sep if we are still in an extended
    # column.
    csv[-1][-1] = @row_sep if in_extended_col

    if in_extended_col
      # if we're at eof?(), a quoted field wasn't closed...
      if @io.eof?
        raise MalformedCSVError,
              "Unclosed quoted field on line #{lineno + 1}."
      elsif @field_size_limit and csv.last.size >= @field_size_limit
        raise MalformedCSVError, "Field size exceeded on line #{lineno + 1}."
      end
      # otherwise, we need to loop and pull some more data to complete the row
    else
      @lineno += 1

      # save fields unconverted fields, if needed...
      unconverted = csv.dup if @unconverted_fields

      # convert fields, if needed...
      csv = convert_fields(csv) unless @use_headers or @converters.empty?
      # parse out header rows and handle CSV::Row conversions...
      csv = parse_headers(csv)  if     @use_headers

      # inject unconverted fields and accessor, if requested...
      if @unconverted_fields and not csv.respond_to? :unconverted_fields
        add_unconverted_fields(csv, unconverted)
      end

      # return the results
      break csv
    end
  end
end

- (Boolean) skip_blanks?

Returns true blank lines are skipped by the parser. See CSV::new for details.

Returns:

  • (Boolean) 


1646
 
# File 'lib/csv.rb', line 1646

def skip_blanks?()        @skip_blanks        end

- (Boolean) unconverted_fields?

Returns true if unconverted_fields() to parsed results. See CSV::new for details.

Returns:

  • (Boolean) 


1615
 
# File 'lib/csv.rb', line 1615

def unconverted_fields?() @unconverted_fields end

- (Boolean) write_headers?

Returns true if headers are written in output. See CSV::new for details.

Returns:

  • (Boolean) 


1630
 
# File 'lib/csv.rb', line 1630

def write_headers?()      @write_headers      end