estimRoundingError {incubate}

Estimate rounding error based on given sample of metric values The idea is to check at which level of rounding the sample values do not change.

Description

Estimate rounding error based on given sample of metric values The idea is to check at which level of rounding the sample values do not change.

Usage

estimRoundingError(obs, roundDigits = seq.int(-4L, 6L), maxObs = 100L)

Arguments

obs

numeric. Metric values from a sample to estimate the corresponding rounding error

roundDigits

integer. Which level of rounding to test? Negative numbers round to corresponding powers of 10

maxObs

integer. How many observations to consider at most? If the provided sample has more observations a sub-sample is used.

Value

estimated rounding error