AZ::Splitter - Splitting and extracting data from file or string, with loading into memory in parts

Reading configuration file:
Find first one of substrings in file without case sensitive:

Special thanks too:
AUTHOR

NAME

AZ::Splitter - Splitting and extracting data from file or string, with loading into memory in parts

SYNOPSIS

  use AZ::Splitter;    # With flags importing into current namespace
  use AZ::Splitter (); # Access to flags from AZ:: namespace only

  $s = AZ::Splitter->new( \*IN );
  $s = AZ::Splitter->new( \*IN, $limit );
  $s = AZ::Splitter->new( \*IN, $limit, $buffsize );
  $s = AZ::Splitter->new( \*IN, $limit, $buffsize, s_2BUFF|s_UTFOK );
  $s = AZ::Splitter->new( \$SCALAR, ... ); # Also for scalar variables

  $s->read_to( \*OUT, $separator );
  $s->read_to( \*OUT, $separator, $limit );
  $s->read_to( \*OUT, $separator, $limit, s_APPEND|s_WEND|s_WCASE );
  $s->read_to( \*OUT, \@separators, ... ) # Many separators
  $s->read_to( \$SCALAR, ... ); # Also for scalar variables
  $s->read_to( undef, ... );    # Only rewind

  $s->read_some( \*OUT );
  $s->read_some( \*OUT, $limit );
  $s->read_some( \*OUT, $limit, s_APPEND );
  $s->read_some( \$SCALAR, ... ); # Also for scalar variables
  $s->read_some( undef, ... );    # Only rewind

  $s->stat_rsize();
  $s->stat_wsize();
  $s->stat_match();
  $s->stat_error();

DESCRIPTION

This is an object-oriented class for fast and simple splitting of big files or strings using little memory. Also can be used for searching data in files, but the main fuction is parsing big volume data.

METHODS

new(INPUT[,LIMIT[,BUFFSIZE[,FLAGS]]])

The constructor method instantiates a new AZ::Splitter object.

INPUT - is a reference to file stream, opened for reading, or reference to string.

LIMIT - limit size of input stream data in characters. If this argument is not defined or equal to -1, then all input stream data will be available for read.

BUFFSIZE - size of buffer in characters. If this argument is not defined or equal to -1, then will be used default buffer size 32768 characters.

FLAGS - 2 modificators are available in this method:

s_2BUFF - use second buffer. Can really speed up search in case insensitive mode.

s_UTFOK - disable UTF-8 data check in UTF-8 mode. Use this flag if you are absolutely sure, that your UTF-8 data is valid.

read_to(OUTPUT,SEPARATOR[,LIMIT[,FLAGS]])

This method reads all data from input stream before first found separator, beginning from current position. Returns true value if found separator or end of stream expected at first time. Otherwise, or in case of reading error, will return false value.

OUTPUT - is a reference to file stream, opened for writing, or reference to string. If another type of value (ex: undef), then data will be not stored.

SEPARATOR - is a string-separator or reference to array with many separators.

Remember! In case of many separators, left separator alltimes have more priority then right!

LIMIT - size in characters. Defines, the maximum number of characters that must be stored in OUTPUT. If this argument not defined or equal to -1, then this method will be trying to store all readed data.

FLAGS - 3 modificators are available in this method:

s_APPEND - append data to OUTPUT if OUTPUT is a reference to string.

s_WCASE - search in case insensitive mode.

s_WEND - at the end of input stream returns only false value. Without this modificator, if end of stream expected at first time, then will be returned true value.

read_some(OUTPUT[,LIMIT[,FLAGS]])

This method reads fixed number of characters from input stream beginning from current position. Returns true value, if any characters were read or end of input stream is not expected yet. Otherwise, or in case of reading error, will return false value.

OUTPUT - same as in read_to() method.

LIMIT - limit size in characters, how many it is necessary to read. If this argument is not defined or equal to -1, then will be read all available data from input stream.

FLAGS - only one modificator available:

s_APPEND - same as in read_to() method.

stat_wsize()

This method returns quantity of characters from previous position and to found separator(or end of stream) after using method read_to(). After using method read_some() this method returns quantity of readed characters.

stat_rsize()

The same as previous method. Only returns quantity of successfuly stored characters.

stat_match()

After using method read_to() returns found separator. Otherwise returns empty string.

stat_error()

Returns error status. In any reading errors methods read_to() and read_some() immediately stops any operations, returns false status and sets up this parameter to true value.

UTF-8 SUPPORT

Fully supported when using perl version 5.8.1, or higher. Input stream, separators, and result reference(if this is file stream), must be in UTF-8 mode. If input stream will has malformed UTF-8 data, then reading from stream will be immediately stopped.

WARNINGS

Remember! This class is using block reading and before destruct class-object, you should work with input stream only through these class methods.

In UTF-8 mode search without case sensitive is very slowly.. It is because operation of changing case on UTF-8 data has slow speed.

Remember, in UTF-8 mode all sizes of this module contain characters, not bytes!

All flags exist in AZ:: namespace and you can disable importing them to your namespace.

EXAMPLES

Reading configuration file:

  open( my $fh, '<', 'config.txt' ) or die $!;
  my $stream = AZ::Splitter->new($fh);

  my $line;
  while( $stream->read_to( \$line, "\r\n" ) ) {
    # Do something with $line
  }
  close($fh) or die $!;

Find first one of substrings in file without case sensitive:

  # Initialize array of strings
  my @strings = ( 'word1', 'word2', 'phrase 1', 'word3' );

  open( my $fh, '<', 'file.txt' ) or die $!;
  my $stream = AZ::Splitter->new($fh);

  # Now, let trying to find one of substrings
  my $result = $stream->read_to( undef, \@strings, -1, s_WCASE|s_WEND );

  if($result) {
    print "Found substring '", $stream->stat_match(), "'\n";
  }
  elsif( $stream->stat_error() ) {
    print "Fatal error during reading file!\n";
  }
  else {
    print "Nothing found..\n";
  }
  close($fh) or die $!;

Special thanks too:

Andrey Fimushkin, <plohaja@mail.ru>

AUTHOR

Andrian Zubko aka Ondr, <ondr@cpan.org>

AZ::Splitter - Splitting and extracting data from file or string, with loading into memory in parts