Class: IO::LikeHelpers::CharacterIO::ConverterReader Private

Inherits:
BasicReader
  • Object
show all
Defined in:
lib/io/like_helpers/character_io/converter_reader.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

This class is a character reader that converts bytes from one character encoding to another. It is also used when simply handling universal newline conversion.

Constant Summary collapse

MIN_BUFFER_SIZE =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

This comes from MRI.

128 * 1024

Instance Method Summary collapse

Methods inherited from BasicReader

#encoding

Constructor Details

#initialize(buffered_io, buffer_size: MIN_BUFFER_SIZE, encoding_opts: {}, external_encoding: nil, internal_encoding: nil) ⇒ ConverterReader

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Creates a new intance of this class.

Note that ‘MIN_BUFFER_SIZE` is used for the buffer size when `buffer_size` is `nil` or less than `MIN_BUFFER_SIZE`.

When ‘internal_encoding` is `nil`, character conversion is not performed, but newline conversion will still be performed if `encoding_opts` specifies such.

Parameters:

  • buffered_io (LikeHelpers::BufferedIO) —

    a readable stream that always blocks

  • buffer_size (Integer, nil) (defaults to: MIN_BUFFER_SIZE) —

    the size of the internal buffer

  • encoding_opts (Hash) (defaults to: {})
  • external_encoding (Encoding, nil) (defaults to: nil) —

    the encoding to apply to the content provided by the underlying stream

  • internal_encoding (Encoding, nil) (defaults to: nil) —

    the encoding into which characters from the underlying stream are converted



34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
 
# File 'lib/io/like_helpers/character_io/converter_reader.rb', line 34

def initialize(
  buffered_io,
  buffer_size: MIN_BUFFER_SIZE,
  encoding_opts: {},
  external_encoding: nil,
  internal_encoding: nil
)
  super(buffered_io, encoding: internal_encoding || external_encoding)

  if ! buffer_size || buffer_size < MIN_BUFFER_SIZE
    buffer_size = MIN_BUFFER_SIZE
  end
  @buffer_size = buffer_size
  @start_idx = @end_idx = @buffer_size
  @buffer = "\0".b * @buffer_size
  @converter = internal_encoding ?
    Encoding::Converter.new(
     external_encoding, internal_encoding, **encoding_opts
    ) :
    Encoding::Converter.new('', '', **encoding_opts)
end

Instance Method Details

#clear ⇒ nil

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Clears the state of this reader.

Returns:

  • (nil) 


60
61
62
63
64
 
# File 'lib/io/like_helpers/character_io/converter_reader.rb', line 60

def clear
  super
  @start_idx = @end_idx = @buffer_size
  nil
end

#consume(length) ⇒ nil

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Consumes bytes from the front of the buffer.

Parameters:

  • length (Integer) —

    the number of bytes to consume

Returns:

  • (nil) 


86
87
88
89
90
91
 
# File 'lib/io/like_helpers/character_io/converter_reader.rb', line 86

def consume(length)
  existing_content_size = @end_idx - @start_idx
  length = existing_content_size if length > existing_content_size
  @start_idx += length
  nil
end

#content ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the bytes of the buffer as a binary encoded String.

The returned bytes should be encoded using the value of BasicReader#encoding and ‘String#force_encoding`. Bytes are returned rather than characters because CharacterIO#read_line works on bytes for compatibility with the MRI implementation and working with characters would be inefficient in that case.

Returns:

  • (String) —

    the bytes of the buffer



76
77
78
 
# File 'lib/io/like_helpers/character_io/converter_reader.rb', line 76

def content
  @buffer[@start_idx..@end_idx-1]
end

#empty? ⇒ Boolean

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns ‘true` if the read buffer is empty and `false` otherwise.

Returns:

  • (Boolean) 


97
98
99
 
# File 'lib/io/like_helpers/character_io/converter_reader.rb', line 97

def empty?
  @start_idx >= @end_idx
end

#refill(many = true) ⇒ nil

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Refills the buffer from the stream.

Parameters:

  • many (Boolean) (defaults to: true) —

    read and convert only 1 character when ‘false`; otherwise, read and convert many characters

Returns:

  • (nil)

Raises:

  • (Encoding::InvalidByteSequenceError) —

    if character conversion is being performed and the next sequence of bytes are invalid in the external encoding

  • (Encoding::UndefinedConversionError) —

    if character conversion is being performed and the character read from the stream cannot be converted to a character in the target encoding

  • (EOFError) —

    when reading at the end of the stream

  • (IOError) —

    if the stream is not readable

  • (IOError) —

    if the buffer is already full



118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
 
# File 'lib/io/like_helpers/character_io/converter_reader.rb', line 118

def refill(many = true)
  existing_content_size = @end_idx - @start_idx
  # Nothing to do if the character buffer is already full.
  return nil if existing_content_size >= @buffer_size

  conversion_buffer = ''.b

  conversion_options = Encoding::Converter::PARTIAL_INPUT
  conversion_options |= Encoding::Converter::AFTER_OUTPUT unless many

  begin
    loop do
      buffered_io.refill if buffered_io.read_buffer_empty?
      input = buffered_io.peek
      input_count = input.bytesize

      result = @converter.primitive_convert(
        input,
        conversion_buffer,
        0,
        @buffer_size - existing_content_size,
        conversion_options
      )

      case result
      when :after_output, :source_buffer_empty, :destination_buffer_full
        consumed = input_count - input.bytesize
        buffered_io.skip(consumed)
        break unless conversion_buffer.empty?
        next
      when :invalid_byte_sequence
        putback = @converter.putback
        consumed = input_count - input.bytesize - putback.bytesize
        buffered_io.skip(consumed)
      end

      raise @converter.last_error
    end
  rescue EOFError
    conversion_buffer << @converter.finish
    # Ignore this if there is still buffered data.
    raise if conversion_buffer.empty?
  end

  append_to_buffer(conversion_buffer.b)

  nil
end

#unread(buffer, length: buffer.bytesize) ⇒ nil

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Places bytes at the beginning of the read buffer.

Parameters:

  • buffer (String) —

    the bytes to insert into the read buffer

  • length (Integer) (defaults to: buffer.bytesize) —

    the number of bytes from the beginning of ‘buffer` to insert into the read buffer

Returns:

  • (nil)

Raises:

  • (IOError) —

    if the remaining space in the internal buffer is insufficient to contain the given data



178
179
180
181
182
183
184
185
186
187
 
# File 'lib/io/like_helpers/character_io/converter_reader.rb', line 178

def unread(buffer, length: buffer.bytesize)
  existing_content_size = @end_idx - @start_idx
  if length > @buffer_size - existing_content_size
    raise IOError, 'insufficient buffer space for unread'
  end

  prepend_to_buffer(buffer.b[0, length])

  nil
end