autopar {automl}

parameters for automatic hyperparameters optimization

Description

List of parameters to allow multi deep neural network automatic hyperparameters tuning with Particle Swarm Optimization
Not mandatory (the list is preset and all arguments are initialized with default value) but it is advisable to adjust some important arguments for performance reasons (including processing time)

Arguments

psopartpopsize

number of particles in swarm, the main argument that should be tuned (default value 8, which is quite low)
#tuning priority 1

psoxxx

see pso for other PSO specific arguments details

numiterations

number of convergence steps between particles (hyperparameters), default value 3)
#tuning priority 1

auto_modexec

if ‘TRUE’ the type of Neural Net optimization will be randomly choosen between ‘trainwgrad’ and ‘trainwpso’ for each particle
default value is ‘FALSE’ (so default value of argument ‘modexec’ in automl_train_manual function, actually ‘trainwgrad’ as default is more suited to large data volume)
the value can be forced if defined in hpar list

auto_runtype

if ‘2steps’ the 2 following steps will be run automatically (default value is ‘normal’):
1st overfitting, the goal is performance
2nd regularization, the goal is generalization
nb: ‘overfitting’ or ‘regularization’ may be directly specified to avoid the 2 steps

auto_minibatchsize

see below

auto_minibatchsize_min

see below

auto_minibatchsize_max

‘auto_minibatch’ default value ‘TRUE’ for automatic adjustment of ‘minibatchsize’ argument in automl_train_manual function
the minimum and maximum value for ‘minibatchsize’ corespond to 2 to the power value (default 0 for ‘auto_minibatchsize_min’ and 9 for ‘auto_minibatchsize_max’)

auto_learningrate

see below

auto_learningrate_min

see below

auto_learningrate_max

‘auto_learningrate’ default value ‘TRUE’ for automatic adjustment of ‘learningrate’ argument in automl_train_manual function
the minimum and maximum value for ‘learningrate’ correspond to 10 to the power negative value (default -5 for ‘auto_learningrate_min’ and -2 for ‘auto_learningrate_max’)

auto_beta1

see below

auto_beta2

‘auto_beta1’ and ‘auto_beta2’ default value ‘TRUE’ for automatic adjustment of ‘beta1’ and ‘beta2’ argument in automl_train_manual function

auto_psopartpopsize

see below

auto_psopartpopsize_min

see below

auto_psopartpopsize_max

‘auto_psopartpopsize’ default value ‘TRUE’ for automatic adjustment of ‘psopartpopsize’ argument in automl_train_manual function (concern only ‘modexec’ set to ‘trainwpso’)
the minimum and maximum value for ‘psopartpopsize’ ; default 2 for ‘auto_psopartpopsize_min’ and 50 for ‘auto_psopartpopsize_max’)

auto_lambda

see below

auto_lambda_min

see below

auto_lambda_max

‘auto_lambda’ default value ‘FALSE’ for automatic adjustment of ‘lambda’ regularization argument in automl_train_manual function
the minimum and maximum value for ‘lambda’ correspond to 10 to the power value (default -2) for ‘auto_lambda_min’ and (default 4) for ‘auto_lambda_max’)

auto_psovelocitymaxratio

see below

auto_psovelocitymaxratio_min

see below

auto_psovelocitymaxratio_max

‘auto_psovelocitymaxratio’ default value ‘TRUE’ for automatic adjustment of ‘psovelocitymaxratio’ PSO velocity max ratio argument in automl_train_manual function
the minimum and maximum value for ‘psovelocitymaxratio’; default 0.01 for ‘auto_psovelocitymaxratio_min’ and 0.5 for ‘auto_psovelocitymaxratio_max’

auto_layers

see below (‘auto_layers’ default value ‘TRUE’ for automatic adjustment of layers shape in automl_train_manual function)

auto_layers_min

(linked to ‘auto_layers’ above, set hpar ‘layersshape’ and ‘layersacttype’) the minimum number of hidden layers (default 1 no hidden layer)

auto_layers_max

(linked to ‘auto_layers’ above, set hpar ‘layersshape’ and ‘layersacttype’) the maximum number of hidden layers (default 2)

auto_layersnodes_min

(linked to ‘auto_layers’ above, set hpar ‘layersshape’ and ‘layersacttype’) the minimum number of nodes per layer (default 3)

auto_layersnodes_max

(linked to ‘auto_layers’ above, set hpar ‘layersshape’ and ‘layersacttype’) the maximum number of nodes per layer (default 33)

auto_layersdropo

see below

auto_layersdropoprob_min

see below

auto_layersdropoprob_max

‘auto_layersdropo’ default value ‘FALSE’ for automatic adjustment of hpar ‘layersdropoprob’ in automl_train_manual function)
the minimum and maximum value for ‘layersdropoprob’; default 0.05 for ‘auto_layersdropoprob_min’ and 0.75 for ‘auto_layersdropoprob_max’

seed

seed for reproductibility (default 4)

nbcores

number of cores used to parallelize particles optimization, not available on Windows (default 1, automatically reduced if not enough cores)

verbose

to display or not the costs at each iteration for each particle (default TRUE)

subtimelimit

time limit in seconds for sub modelizations to avoid waiting too long for a specific particle to finish its modelization (default 3600)

back to automl_train