dltCalibrateCameras {StereoMorph} | R Documentation |
Finds the optimized DLT coefficients for a stereo camera setup
Description
This function uses the corners from a grid positioned in several different orientations within a stereo camera setup to estimate the DLT calibration coefficients that minimize reconstruction error.
Usage
dltCalibrateCameras(coor.2d, nx, grid.size, c.run = FALSE, reduce.grid.dim = 3,
fit.min.break = 1, nlm.iter.max.init = 100, objective.min.init = 10,
nlm.eval.max = 350, nlm.iter.max = 250, nlm.calls.max = 100,
min.views = 'max', objective.min = 1, grid.incl.min=2,
objective.min.break = NULL, start.param=NULL,
sx = NULL, sy = NULL, print.progress = FALSE, print.tab = '')
## S3 method for class 'dltCalibrateCameras'
summary(object, ...)
Arguments
coor.2d |
a four-dimensional array of grid points. The first two dimensions correspond to each matrix of grid points, the third corresponds to each grid position/orientation and the fourth corresponds to each camera view. Can be read from file by the function |
nx |
the number of points along the first dimension (e.g. this would be the number of points in each row if points in |
grid.size |
the size of the grid squares in real-world units (e.g. millimeters). |
c.run |
a logical indicating whether a second optimization should be performed on the calibration coefficients. |
reduce.grid.dim |
a numeric indicating the number of grid points along each dimension for each grid after resampling. The total number of resampled points is |
fit.min.break |
passed to |
nlm.iter.max.init |
The maximum number of iterations to be performed by |
objective.min.init |
The objective used during the initial coefficient optimization, passed as a control parameter to |
nlm.eval.max |
The maximum number of evaluations to be performed by |
nlm.iter.max |
The maximum number of iterations to be performed by |
nlm.calls.max |
The maximum number of different sets of random starting parameters to use during coefficient optimization. This parameter cannot exceed 576. |
min.views |
The minimum views in which corners must be detected in order to use in coefficient estimation. If set to 'max' (default) this will be equal to the number of input views. |
objective.min |
The expected mean reconstruction error when optimizing the calibration coefficients (the minimum, or |
grid.incl.min |
The minimum number of grids to include during coefficient optimization. |
objective.min.break |
During coefficient optimization if the error (in pixels) always exceeds this value the estimation will stop estimating the position of additional checkerboards. |
start.param |
An set of fixed starting parameters to be used during coefficient optimization. This parameter is intended primarily for debugging. |
sx |
Used for de-bugging. |
sy |
Used for de-bugging. |
print.progress |
a logical indicating whether the progress of the function should be printed while running. This includes the error in grid re-sampling, an iteration count during optimization and other outputs relating to the optimization. |
print.tab |
Tabs preceding lines printed to console. |
object |
a list of class |
... |
further arguments passed to or from other methods. |
Details
Calibration is the most challenging step in stereo camera data collection. Most fundamentally, DLT calibration requires a set of 3D coordinates and their corresponding 2D pixel coordinates in each camera view in order to derive calibration coefficients (see dltCoefficients
). These coefficients can then be used to reconstruct any point in 3D given its 2D pixel coordinates in two or more camera views. DLT calibration has traditionally been done using a "calibration object", typically a 3D box-shaped structure filled with markers at known 3D positions. Such objects require the use of high precision machining in order to achieve an accurate calibration and the calibration points are usually digitized manually.
The dltCalibrateCameras()
function provides a camera calibration routine that is easier to implement and potentially more accurate. This function uses the corners from a grid positioned in several different orientations within the calibration space to estimate the DLT calibration coefficients that minimize reconstruction error. The easiest method for obtaining these corner points is to print a checkerboard pattern (using drawCheckerboard
), attach the pattern to flat, hard surface and use findCheckerboardCorners
to automatically extract the pixel coordinates of the internal corners.
The grid pattern should be photographed in at least four different positions and orientations spanning the volume to be calibrated (the tutorial files loaded with StereoMorph include eight different positions). Using only a couple of positions will result in uneven sampling of the calibration volume causing larger errors in some regions relative to others. Additionally, using only a single orientation of the checkerboard will produce higher errors along a particular dimension relative to the others. Once the pixel coordinates of the grid points (e.g. the internal corners of the checkerboard pattern) have been extracted from all of the calibration images, they should be read into an array using readCheckerboardsToArray
. This function allows for the point order to be reversed along rows, columns or both. If one of the cameras views the pattern upside down relative to another camera or if the pattern is in a different orientation, the grid points may be extracted in a different order. This can be fixed using the row.reverse
and col.reverse
arguments in readCheckerboardsToArray
. It is essential that the grid points extracted from each camera view correspond to each other row-by-row or else the calibration will not work.
dltCalibrateCameras()
first calls resampleGridImagePoints
to downsample the number of grid points. reduce.grid.dim
is the downsample number (the default is 3, meaning 3x3 or nine points per grid). Downsampling can be turned off by setting reduce.grid.dim
to 0
, although this is not recommended as it will increase run-time substantially without increasing accuracy. A camera perspective model is fit to the full point set such that the number of points input to the coefficient optimization can be reduced (thereby reducing run-time) without losing any relevant information (see resampleGridImagePoints
). If print.progress
is set to TRUE
, the mean and maximum fit error is printed for each input grid. As the fitting does not take into account lens distortion, high fit errors may indicate large distortional effects.
Since each checkerboard grid has been photographed in an arbitrary position and orientation, the 3D coordinates of the grid points are unknown. However, if the first grid is fixed, each additional grid can be described by applying six transformation parameters relative to the first (three translational and three rotational). Using the reduced grid point set, dltCalibrateCameras()
uses nlminb()
to search for the six transformation parameters per grid that minimizes the RMS error when the 3D coordinates are input (with the corresponding 2D coordinates) to dltCoefficients
. In effect, dltCalibrateCameras()
solves for the position of each grid in 3D space using the error from dltCoefficients
as an optimality criterion. Since the first grid is fixed, the optimization will search for 6*(n-1) parameters, where n is the number of separate grid orientations. nlminb()
calls the function dltTransformationParameterRMSError
.
In order to fully explore the parameter space, dltCalibrateCameras()
calls nlminb()
several times with a different set of randomly generated starting parameters to estimate the transformation parameters for each additional grid. The number of different sets of starting parameters is determined by nlm.calls.max
. An initial optimization run is intended to quickly determine whether a particular set of starting parameters is likely to lead to convergence. The number of iterations for this initial optimization is determined by nlm.iter.max.init
and the objective used in determining likely convergence is objective.min.init
. If it is determined that the starting parameters are likely to lead to convergence below objective.min
, nlminb()
is allowed to continue optimizing. For each grid, the solution yielding the lowest error, or the first solution below the objective.min
threshold, is retained for the next grid optimization.
These optimal transformation parameters are then used to obtain the 3D coordinates of the original grid points (not downsampled). Once these 3D coordinates are known, the 3D and 2D pixel coordinates are input to dltCoefficients
to obtain the 11 calibration coefficients per camera. For this reason, the calibration coefficient RMS Error (coefficient.rmse
) returned will differ slightly from the reported final nlminb()
minimum (t.min
).
If c.run
is set to TRUE
, dltCalibrateCameras()
performs a second optimization on the calibration coefficients themselves. nlminb()
is used, this time calling dltCoefficientRMSError
, to find the 11 calibration coefficients per view that minimizes the reconstruction RMS error. Note that dltCoefficients
cannot be used as with the previous optimization because the coefficients must be an input. Running this second optimization seems to have little effect in increasing the accuracy of the calibration but is included as this may be useful for some stereo setups.
Value
a list of class "dltCalibrateCameras"
with the following elements:
cal.coeff |
a matrix of 11 optimized DLT calibration coefficients per camera view. |
coor.3d |
the optimized 3D coordinates of the input grid points in |
mean.reconstruct.rmse |
the RMS error when |
coefficient.rmse |
the RMS error when |
t.param.final |
the final transformation parameters reported by |
t.min |
the minimum reported by |
t.runtime |
the run-time (in seconds) for the first optimization. |
if c.run
is FALSE
, the following are NA
. Otherwise,
c.param.init |
the initial parameters for the second optimization. |
c.param.final |
the final parameters reported by |
c.min |
the minimum reported by |
c.iter |
the number of iterations reported by |
c.runtime |
the run-time (in seconds) for the second optimization. |
Author(s)
Aaron Olsen
References
For a general overview of DLT: http://kwon3d.com/theory/dlt/dlt.html
See Also
dltTestCalibration
,
dltCoefficients
,
readCheckerboardsToArray
,
transformPlanarCalibrationCoordinates
,
dltTransformationParameterRMSError
,
Examples
## SET NUMBER OF INTERNAL CORNERS FOR CALIBRATION GRIDS
nx <- 21
ny <- 14
## GET THE FILE DIRECTORY FOR EXTRA R PACKAGE FILES
fdir <- paste0(path.package("StereoMorph"), "/extdata/")
## SET FILE PATH TO CHECKERBOARD CORNERS FROM CALIBRATION IMAGE SET
## THE TUTORIAL INCLUDES 8 CALIBRATION IMAGES FROM TWO CAMERA VIEWS
file <- matrix(c(paste0(fdir, "cal_a", 1:8, "_v1.txt"),
paste0(fdir, "cal_a", 1:8, "_v2.txt")), ncol=2)
## READ IN CHECKERBOARD CORNERS
## NOTE THAT col.reverse IS USED TO MAKE POINTS CORRESPOND
coor.2d <- readCheckerboardsToArray(file=file, nx=nx, ny=ny, col.reverse=TRUE)
## SET GRID SIZE (IN MM)
grid.size <- 6.347889
## Not run:
## CALIBRATE CAMERAS
## TO REDUCE RUN-TIME, WE JUST USE CORNERS FROM TWO IMAGES (1 AND 5)
dlt_calibrate_cameras <- dltCalibrateCameras(coor.2d=coor.2d[, , c(1, 5), ], nx=nx,
grid.size=grid.size, c.run=FALSE, print.progress=TRUE)
## RUN CALIBRATION ON ALL IMAGES, ACCURACY IS GREATLY IMPROVED
dlt_calibrate_cameras <- dltCalibrateCameras(coor.2d=coor.2d, nx=nx,
grid.size=grid.size, c.run=FALSE, print.progress=TRUE)
## PRINT SUMMARY
summary(dlt_calibrate_cameras)
## End(Not run)