wrap_parallel {spaMM} | R Documentation |
Selecting interfaces for parallelisation
Description
spaMM implements three interfaces for parallelisation. Depending on their arguments, either serial computation (default), a socket cluster (parallelisation default), or a fork cluster (available in linux and alike operating systems) can be used by all interfaces.
dopar
is called by default by its bootstrap procedures, and dofuture
has been developed as an alternative, whose use is controlled by spaMM.options(wrap_parallel="dofuture")
(versus the default, spaMM.options(wrap_parallel="dopar")
. combinepar
is the third and more recent interface; it is not a formally supported wrap_parallel
option because its additional functionalities are of no use in spaMM's bootstrap procedures.
dopar
is based on a patchwork of backends: for socket clusters, depending whether the doSNOW package is attached, foreach
or pbapply
is called (doSNOW allows more efficient load balancing than pbapply
); for fork clusters, parallel::mclapply
is used. This makes it impossible to ensure consistency of options accross computation environments, notably of enforcing the .combine
control of foreach
; and this makes it error-prone to ensure identical control of random number generators in all cases (although dopar
and combinepar
still aim to ensure the latter control).
By contrast, dofuture
is based only on the future and future.apply packages, in principle allowing a single syntax to control of random number generator across the different cases, hence repeatable results across them. This does not make a difference for bootstrap computations in spaMM as the bootstrap samples are never simulated in parallel: only refitting the models is performed in parallel, and fit results do not depend on random numbers. Further, the future-based code for socket clusters appears significantly slower than the one used by dopar
. For these reasons, the latter function is used by default by spaMM.
combinepar
is a third and more recent approach designed to address the other issue: it always uses foreach so that the .combine
control is consistently enforced. It uses future only when no alternative is available to produce a progress bar (namely, for socket clusters when doSNOW is not available).