hdd_setkey {hdd} | R Documentation |
Sorts HDD objects
Description
This function sets a key to a HDD file. It creates a copy of the HDD file sorted by the key. Note that the sorting process is very time consuming.
Usage
hdd_setkey(x, key, newfile, chunkMB = 500, replace = FALSE, verbose = 1)
Arguments
x |
A hdd file. |
key |
A character vector of the keys. |
newfile |
Destination of the result, i.e., a destination folder that will receive the HDD data. |
chunkMB |
The size of chunks used to sort the data. Default is 500MB. The bigger this number the faster the sorting is (depends on your memory available though). |
replace |
Default is |
verbose |
Numeric, default is 1. Whether to display information on the advancement of the algorithm. If equal to 0, nothing is displayed. |
Details
This function is provided for convenience reason: it does the job of sorting the data and ensuring consistency across files, but it is very slow since it involves copying several times the entire data set. To be used parsimoniously.
Value
This functions does not return anything in R, instead its result is a new
folder populated with .fst
files which represent a data set that can be loaded
with the function hdd()
.
Author(s)
Laurent Berge
See Also
See hdd
, sub-.hdd
and cash-.hdd
for the extraction and manipulation of out of memory data. For importation of
HDD data sets from text files: see txt2hdd
.
See hdd_slice
to apply functions to chunks of data (and create
HDD objects) and hdd_merge
to merge large files.
To create/reshape HDD objects from memory or from other HDD objects, see
write_hdd
.
To display general information from HDD objects: origin
,
summary.hdd
, print.hdd
,
dim.hdd
and names.hdd
.
Examples
# Toy example with iris data
# Creating HDD data to be sorted
hdd_path = tempfile() # => folder where the data will be saved
write_hdd(iris, hdd_path)
# Let's add data to it
for(i in 1:5) write_hdd(iris, hdd_path, add = TRUE)
base_hdd = hdd(hdd_path)
summary(base_hdd)
# Sorting by Sepal.Width
hdd_sorted = tempfile()
# we use a very small chunkMB to show how the function works
hdd_setkey(base_hdd, key = "Sepal.Width",
newfile = hdd_sorted, chunkMB = 0.010)
base_hdd_sorted = hdd(hdd_sorted)
summary(base_hdd_sorted) # => additional line "Sorted by:"
print(base_hdd_sorted)
# Sort with two keys:
hdd_sorted = tempfile()
# we use a very small chunkMB to show how the function works
hdd_setkey(base_hdd, key = c("Species", "Sepal.Width"),
newfile = hdd_sorted, chunkMB = 0.010)
base_hdd_sorted = hdd(hdd_sorted)
summary(base_hdd_sorted)
print(base_hdd_sorted)