split_parapply {bigparallelr}

Split-parApply-Combine

Description

A Split-Apply-Combine strategy to parallelize the evaluation of a function.

Usage

split_parapply(
  FUN,
  ind,
  ...,
  .combine = NULL,
  ncores = nb_cores(),
  nb_split = ncores,
  opts_cluster = list(),
  .costs = NULL
)

Arguments

FUN

The function to be applied to each subset matrix.

ind

Initial vector of indices that will be splitted in nb_split.

...

Extra arguments to be passed to FUN.

.combine

Function to combine the results with do.call. This function should accept multiple arguments (using ...). For example, you can use c, cbind and rbind. This package also provides function plus to add multiple arguments together. The default is NULL, in which case the results are not combined and are returned as a list, each element being the result of a block.

ncores

Number of cores to use. Default uses nb_cores().

nb_split

Number of blocks. Default uses ncores.

opts_cluster

Optional parameters for clusters passed as a named list. E.g., you can use type = "FORK" to use forks instead of clusters. You can also use outfile = "" to redirect printing to the console.

.costs

Vector of costs (e.g. proportional to computation time) associated with each element of ind. Default is NULL (same cost).

Details

This function splits indices in parts, then apply a given function to each part and finally combine the results.

Value

Return a list of ncores elements, each element being the result of one of the cores, computed on a block. The elements of this list are then combined with do.call(.combine, .) if .combined is not NULL.

Examples

## Not run: 

str(
  split_parapply(function(ind) {
    sqrt(ind)
  }, ind = 1:10000, ncores = 2)
)

## End(Not run)