import_files {assertable}

Given a vector of filenames, append all files and return as one data.table using a user-defined function

Description

Given a character vector of filenames, check how many of them currently exist. Optionally, can keep checking for a specified amount of time, at a given frequency

Usage

import_files(filenames, folder = "", FUN = fread, warn_only = FALSE,
  multicore = FALSE, use.names = TRUE, fill = TRUE,
  mc.preschedule = FALSE, mc.cores = getOption("mc.cores", 2L), ...)

Arguments

filenames

A character vector of filenames (specify full paths if you are checking files that are not in present working directory)

folder

An optional character containing the folder name that contains the files you want to check (if used, do not include folderpath in the filenames characters). If not specified, will look in present working directory.

FUN

function: The function that you want to use to import your data, e.g. read.csv, fread, read_dta, etc.

warn_only

Boolean (T/F), whether to send a warning message as opposed to an error message if files are missing prior to import. Will only import the files that do exist.

multicore

boolean, use lapply or mclapply (multicore = T) to loop over files in filenames for import. Default=F.

use.names

boolean, pass to the use.names option for rbindlist

fill

boolean, pass to the fill option for rbindlist

mc.preschedule

boolean, pass to the mc.preschedule option for mclapply if multicore = T. Default = F.

mc.cores

pass to the mc.preschedule option for mclapply if multicore = T. Default = mclapply default.

...

named arguments of FUN to pass to FUN

Value

One data.table that contains all files in filenames, combined together using rbindlist. Returns an error if any file in filenames does not exist

Examples

## Not run: 
 for(i in 1:3) {
   data <- CO2
   data$id_var <- i
   write.csv(data,file=paste0("file_",i,".csv"),row.names=FALSE)
 }
 filenames <- paste0("file_",c(1:3),".csv")
 import_files(filenames, FUN=fread)
 import_files(filenames, FUN=read.csv, stringsAsFactors=FALSE)
 import_files(filenames, FUN=fread, multicore=T, mc.cores=1) # Only if you have a multi-core system

## End(Not run)