filearray_bind {filearray} | R Documentation |
Merge and bind homogeneous file arrays
Description
The file arrays to be merged must be homogeneous: same data type, partition size, and partition length
Usage
filearray_bind(
...,
.list = list(),
filebase = tempfile(),
symlink = FALSE,
overwrite = FALSE,
cache_ok = FALSE
)
Arguments
... , .list |
file array instances |
filebase |
where to create merged array |
symlink |
whether to use |
overwrite |
whether to overwrite when |
cache_ok |
see 'Details', only used if |
Details
The input arrays must share the same data type and partition size.
The dimension for each partition should also be the same. For example
an array x1
has dimension 100x20x30
with partition size
1
, then each partition dimension is 100x20x1
, and there are
30
partitions. x1
can bind with another array of the same
partition size. This means if x2
has dimension
100x20x40
and each partition size is 1
, then x1
and
x2
can be merged.
If filebase
exists and overwrite
is FALSE
, an error will
always raise. If overwrite=TRUE
and cache_ok=FALSE
, then
the existing filebase
will be erased and any data stored within will
be lost.
If both overwrite
and cache_ok
are TRUE
, then
, before erasing filebase
, the function validates the existing
array header and compare the header signatures. If the existing header
signature is the same as the array to be created, then the existing array
will be returned. This cache_ok
could be extremely useful when
binding large arrays with symlink=FALSE
as the cache might avoid
moving files around. However, cache_ok
should be enabled with caution.
This is because only the header information will be compared, but the
partition data will not be compared. If the existing array was generated from
an old versions of the source arrays, but the data from the source arrays
has been altered, then the cache_ok=TRUE
is rarely proper as the cache
is outdated.
The symlink
option should be used with extra caution. Creating
symbolic links is definitely faster than copying partition files. However,
since the partition files are simply linked to the original partition files,
changing to the input arrays will also affect the merged arrays, and
vice versa; see 'Examples'. Also for arrays created from symbolic links, if
the original
arrays are deleted, while the merged arrays will not be invalidated,
the corresponding partitions will no longer be accessible. Attempts to
set deleted partitions will likely result in failure. Therefore
symlink
should be set to true when creating merged arrays are
temporary for read-only purpose, and when speed and disk space is in
consideration. For extended reading, please check files
for details.
Value
A bound array in 'FileArray'
class.
Examples
partition_size <- 1
type <- "double"
x1 <- filearray_create(
tempfile(), c(2,2), type = type,
partition_size = partition_size)
x1[] <- 1:4
x2 <- filearray_create(
tempfile(), c(2,1), type = type,
partition_size = partition_size)
x2[] <- 5:6
y1 <- filearray_bind(x1, x2, symlink = FALSE)
y2 <- filearray_bind(x1, x2)
# y1 copies partition files, and y2 simply creates links
# if symlink is supported
y1[] - y2[]
# change x1
x1[1,1] <- NA
# y1 is not affected
y1[]
# y2 changes
y2[]