imstrsplit {iotools}

Create an iterator for splitting binary or character input into a matrix

Description

imstrsplit takes a binary connection or character vector (which is interpreted as a file name) and splits it into a character matrix according to the separator.

Usage

imstrsplit(x, sep="|", nsep=NA, strict=TRUE, ncol = NA,
          type=c("character", "numeric", "logical", "integer",  "complex", 
                 "raw"), max.line = 65536L, max.size = 33554432L)

Arguments

x

character vector (each element is treated as a row) or a raw vector (LF characters '\n' separate rows) to split

sep

single character: field (column) separator. Set to NA for no seperator; in other words, a single column.

nsep

row name separator (single character) or NA if no row names are included

strict

logical, if FALSE then mstrsplit will not fail on parsing errors, otherwise input not matching the format (e.g. more columns than expected) will cause an error.

ncol

number of columns to expect. If NA then the number of columns is guessed from the first line.

type

a character string representing one of the 6 atomic types: 'character', 'numeric', 'logical', 'integer', 'complex', or 'raw'. The output matrix will use this as its storage mode and the input will be parsed directly into this format without using intermediate strings.

max.line

maximum length of one line (in byets) - determines the size of the read buffer, default is 64kb

max.size

maximum size of the chunk (in bytes), default is 32Mb

Details

If the input is a raw vector, then it is interpreted as ASCII/UTF-8 content with LF ('\n') characters separating lines. If the input is a character vector then each element is treated as a line.

If nsep is specified then all characters up to (but excluding) the occurrence of nsep are treated as the row name. The remaining characters are split using the sep character into fields (columns). If ncol is NA then the first line of the input determines the number of columns. mstrsplit will fail with an error if any line contains more columns then expected unless strict is FALSE. Excessive columns are ignored in that case. Lines may contain fewer columns in which case they are set to NA.

The processing is geared towards efficiency - no string re-coding is performed and raw input vector is processed directly, avoiding the creation of intermediate string representations.

Note that it is legal to use the same separator for sep and nsep in which case the first field is treated as a row name and subsequent fields as data columns.

Value

A matrix with as many rows as they are lines in the input and as many columns as there are fields in the first line. The storage mode of the matrix will be determined by the input to type.

Author(s)

Michael Kane

Examples

  mm <- model.matrix(~., iris)
  f <- file("iris_mm.io", "wb")
  writeBin(as.output(mm), f)
  close(f)
  it <- imstrsplit("iris_mm.io", type="numeric", nsep="\t")
  iris_mm <- it$nextElem()
  print(head(iris_mm))

  ## remove iterator, connections and files
  rm("it")
  gc(FALSE)
  unlink("iris_mm.io")