tfb_softplus {tfprobability}

Computes Y = g(X) = Log[1 + exp(X)]

Description

The softplus Bijector has the following two useful properties:

• The domain is the positive real numbers

• softplus(x) approx x, for large x, so it does not overflow as easily as the Exp Bijector.

Usage

tfb_softplus(
  hinge_softness = NULL,
  low = NULL,
  validate_args = FALSE,
  name = "softplus"
)

Arguments

hinge_softness

Nonzero floating point Tensor. Controls the softness of what would otherwise be a kink at the origin. Default is 1.0.

low

Nonzero floating point tensor, lower bound on output values. Implicitly zero if NULL. Otherwise, the transformation y = softplus(x) + low is implemented. This is equivalent to a tfb_chain(list(tfb_shift(low), tfb_softplus())) bijector and is provided for convenience.

validate_args

Logical, default FALSE. Whether to validate input with asserts. If validate_args is FALSE, and the inputs are invalid, correct behavior is not guaranteed.

name

name prefixed to Ops created by this class.

Details

The optional nonzero hinge_softness parameter changes the transition at zero. With hinge_softness = c, the bijector is:

f_c(x) := c * g(x / c) = c * Log[1 + exp(x / c)].
```

For large x >> 1,

```
c * Log[1 + exp(x / c)] approx c * Log[exp(x / c)] = x
```

so the behavior for large x is the same as the standard softplus.
As c > 0 approaches 0 from the right, f_c(x) becomes less and less soft,
approaching max(0, x).
* c = 1 is the default.
* c > 0 but small means f(x) approx ReLu(x) = max(0, x).
* c < 0 flips sign and reflects around the y-axis: f_{-c}(x) = -f_c(-x).
* c = 0 results in a non-bijective transformation and triggers an exception.
Note: log(.) and exp(.) are applied element-wise but the Jacobian is a reduction over the event space.

[1 + exp(x / c)]: R:1%20+%20exp(x%20/%20c)
[1 + exp(x / c)]: R:1%20+%20exp(x%20/%20c)
[exp(x / c)]: R:exp(x%20/%20c)

Value

a bijector instance.

See Also

For usage examples see tfb_forward(), tfb_inverse(), tfb_inverse_log_det_jacobian().

Other bijectors: tfb_absolute_value(), tfb_affine_linear_operator(), tfb_affine_scalar(), tfb_affine(), tfb_ascending(), tfb_batch_normalization(), tfb_blockwise(), tfb_chain(), tfb_cholesky_outer_product(), tfb_cholesky_to_inv_cholesky(), tfb_correlation_cholesky(), tfb_cumsum(), tfb_discrete_cosine_transform(), tfb_expm1(), tfb_exp(), tfb_ffjord(), tfb_fill_scale_tri_l(), tfb_fill_triangular(), tfb_glow(), tfb_gompertz_cdf(), tfb_gumbel_cdf(), tfb_gumbel(), tfb_identity(), tfb_inline(), tfb_invert(), tfb_iterated_sigmoid_centered(), tfb_kumaraswamy_cdf(), tfb_kumaraswamy(), tfb_lambert_w_tail(), tfb_masked_autoregressive_default_template(), tfb_masked_autoregressive_flow(), tfb_masked_dense(), tfb_matrix_inverse_tri_l(), tfb_matvec_lu(), tfb_normal_cdf(), tfb_ordered(), tfb_pad(), tfb_permute(), tfb_power_transform(), tfb_rational_quadratic_spline(), tfb_rayleigh_cdf(), tfb_real_nvp_default_template(), tfb_real_nvp(), tfb_reciprocal(), tfb_reshape(), tfb_scale_matvec_diag(), tfb_scale_matvec_linear_operator(), tfb_scale_matvec_lu(), tfb_scale_matvec_tri_l(), tfb_scale_tri_l(), tfb_scale(), tfb_shifted_gompertz_cdf(), tfb_shift(), tfb_sigmoid(), tfb_sinh_arcsinh(), tfb_sinh(), tfb_softmax_centered(), tfb_softsign(), tfb_split(), tfb_square(), tfb_tanh(), tfb_transform_diagonal(), tfb_transpose(), tfb_weibull_cdf(), tfb_weibull()