DStorage {DSL}

Virtual Distributed Storage Class

Description

When using class DList the underlying ‘virtual’ storage plays an important role. It defines how to use the given storage (read/write methods, etc.), where the data is to be stored (i.e., the base directory on the file system), and how DMap as well as DReduce have to be applied.

Usage

DStorage(type = c("LFS", "HDFS"), base_dir, chunksize = 1024^2)
is.DStorage( ds )

Arguments

type

the type of the storage to be created. Currently only "LFS" and "HDFS" storage types are supported.

base_dir

specifies the base directory where data is to be stored.

chunksize

defines the size of each chunk written to the virtual storage.

ds

a virtual possibly distributed storage.

Value

An object which inherits from class DStorage, or, in case of is.DStorage() a logical indicating whether it inherits from "DStorage" or not.

Examples

## creating a new virtual storage using 50MB chunks
ds <- DStorage(type = "LFS", base_dir = tempdir(),
chunksize = 50 * 1024^2)
is.DStorage( ds )