new_stream {mrgsim.parallel}

Create a stream of outputs and inputs

Description

By stream we mean a list that pre-specifies the output file names, replicate numbers and possibly input objects for a simulation. Passing locker initiates a call to setup_locker(), which sets up or resets the output directories.

For the data.frame method, the data are chunked into a list by columns listed in cols. Ideally, this is a singlel column that operates as a unique ID across the data set and is used by chunk_by_id() to form the chunks. Alternatively, cols can be multiple column names which are pasted together to form a unique ID that is used for splitting via chunk_by_cols().

Usage

new_stream(x, ...)

## S3 method for class 'list'
new_stream(x, locker = NULL, format = NULL, ...)

## S3 method for class 'data.frame'
new_stream(x, nchunk, cols = "ID", locker = NULL, format = NULL, ...)

## S3 method for class 'numeric'
new_stream(x, ...)

## S3 method for class 'character'
new_stream(x, ...)

Arguments

x

A list or vector to template the stream; for the numeric method, passing a single number will fill x with a sequence of that length.

...

Additional arguments passed to file_set().

locker

Passed to setup_locker() as dir; important to note that the directory will be unlinked if it exists and is an established locker directory.

format

Passed to format_stream().

nchunk

The number of chunks.

cols

The name(s) of the column(s) specifying unique IDs to use to split the data.frame into chunks; this could be a unique ID or a combination of columns that when pasted together form a unique ID.

Value

A list with the following elements:

• i the position number

• file the output file name

• x the input object.

The list has class file_stream as well as locker_stream (if locker was passed) and a class attribute for the output if format was passed.

See Also

format_stream(), locate_stream(), ext_stream(), file_stream(), file_set()

Examples

x <- new_stream(3)
x[[1]]

new_stream(2, locker = file.path(tempdir(), "foo"))

df <- data.frame(ID = c(1,2,3,4))
x <- new_stream(df, nchunk = 2)
x[[2]]

format_is_set(x[[2]])

x <- new_stream(3, format = "fst")
format_is_set(x[[2]])