batchReduceResults {BatchJobs}

Reduces results via a binary function and adds jobs for this to a registry.

Description

Each jobs reduces a certain number of results on one slave. You can then submit these jobs to the batch system. Later, you can do a final reduction with reduceResults on the master.

Usage

batchReduceResults(
  reg,
  reg2,
  fun,
  ids,
  part = NA_character_,
  init,
  block.size,
  more.args = list()
)

Arguments

reg

[Registry]
Registry whose results should be reduced by fun.

reg2

[Registry]
Empty registry that should store the job for the mapping.

fun

[function(aggr, job, res, ...)]
Function to reduce results with.

ids

[integer]
Ids of jobs whose results should be reduced with fun. Default is all jobs.

part

[character] Only useful for multiple result files, then defines which result file part(s) should be loaded. NA means all parts are loaded, which is the default.

init

[any]
Initial object for reducing.

block.size

[integer(1)]
Number of results reduced in one job.

more.args

[list]
A list of other arguments passed to fun. Default is empty list.

Value

Vector of type integer with job ids.

Examples

# generating example results:
reg1 = makeRegistry(id = "BatchJobsExample1", file.dir = tempfile(), seed = 123)
f = function(x) x^2
batchMap(reg1, f, 1:20)
submitJobs(reg1)
waitForJobs(reg1)

# define function to reduce on slave, we want to sum the squares
myreduce = function(aggr, job, res) aggr + res

# sum 5 results on each slave process, i.e. 4 jobs
reg2 = makeRegistry(id = "BatchJobsExample2", file.dir = tempfile(), seed = 123)
batchReduceResults(reg1, reg2, fun = myreduce, init = 0, block.size = 5)
submitJobs(reg2)
waitForJobs(reg2)

# now reduce one final time on master
reduceResults(reg2, fun = myreduce)