create_qsub_config {qsub}

Create a qsub configuration object.

Description

Create a qsub configuration object.

Usage

create_qsub_config(
  remote,
  local_tmp_path,
  remote_tmp_path,
  name = "r2qsub",
  num_cores = 1,
  memory = "4G",
  max_running_tasks = NULL,
  max_wall_time = "01:00:00",
  batch_tasks = 1,
  compress = c("gz", "bz2", "xz", "none"),
  modules = "R",
  execute_before = NULL,
  verbose = FALSE,
  wait = TRUE,
  remove_tmp_folder = TRUE,
  stop_on_error = TRUE
)

override_qsub_config(
  qsub_config = get_default_qsub_config(),
  remote = qsub_config$remote,
  local_tmp_path = qsub_config$local_tmp_path,
  remote_tmp_path = qsub_config$remote_tmp_path,
  name = qsub_config$name,
  num_cores = qsub_config$num_cores,
  memory = qsub_config$memory,
  max_running_tasks = qsub_config$max_running_tasks,
  max_wall_time = qsub_config$max_wall_time,
  batch_tasks = qsub_config$batch_tasks,
  compress = qsub_config$compress,
  modules = qsub_config$modules,
  execute_before = qsub_config$execute_before,
  verbose = qsub_config$verbose,
  wait = qsub_config$wait,
  remove_tmp_folder = qsub_config$remove_tmp_folder,
  stop_on_error = qsub_config$stop_on_error
)

Arguments

remote

Remote machine specification for ssh, in format such as user@server:port that does not require interactive password entry.

local_tmp_path

A directory on the local machine in which to store temporary files. Should not contain a tilde ('~').

remote_tmp_path

A directory on the remote machine in which to store temporary files. Should not contain a tilde ('~').

name

The name of the execution. This will show up, for instance, in qstat.

num_cores

The number of cores to allocate per element in X in a qsub_lapply (default: 1).

memory

The memory to allocate per core (default: "4G"). If this is set too high without it being required, you might not be able to make optimal use of the remote cluster.

max_running_tasks

limit concurrent array job task execution (default: NULL, infinite). If you have long jobs and there are many other users on the cluster, it is recommended you set this value to a reasonable number, such as 1/4th the total number of nodes * number of cores per node.

max_wall_time

The maximum time each task is allowed to run (default: "01:00:00", 1 hour). If set to NULL, the job will be allowed to run indefinitely. Mind you, this might annoy other users of the cluster.

batch_tasks

How many values in X should be processed per task. Useful for when the 'length(X)' is very large (> 10000).

compress

Compression method to use: "none", "gz" (default), "bz2", or "xz".

modules

Which modules to load (default: "R"). If set to NULL, it will be assumed Rscript will be available in the path through other means.

execute_before

Commands to execute in the bash shell before running R.

verbose

Whether or not to print out any ssh commands.

wait

If TRUE, will wait until the execution has finished by periodically checking the job status.

remove_tmp_folder

If TRUE, will remove everything that was created related to this execution at the end.

stop_on_error

If TRUE, will stop when an error occurs, else returns a NA for errored instances.

qsub_config

A qsub_config to be overridden

Value

A qsub configuration object.

See Also

qsub_lapply, set_default_qsub_config

Examples

## Not run: 
qsub_config <- create_qsub_config(
  remote = "[e-mail]:22",
  local_tmp_path = "/home/myuser/workspace/.r2gridengine",
  remote_tmp_path = "/scratch/myuser/.r2gridengine"
)
qsub_lapply(1:10, function(x) x + 1, qsub_config = qsub_config)

set_default_qsub_config(qsub_config, permanent = TRUE)
qsub_lapply(1:10, function(x) x + 1)

qsub_lapply(
  X = 1:10,
  FUN = function(x) x + 1,
  qsub_config = override_qsub_config(verbose = TRUE)
)

## End(Not run)