svmDP {DPpack} R Documentation

## Privacy-preserving Support Vector Machine

### Description

This class implements differentially private support vector machine (SVM) (Chaudhuri et al. 2011). Either the output or the objective perturbation method can be used.

### Details

To use this class for SVM, first use the new method to construct an object of this class with the desired function values and hyperparameters, including a choice of the desired kernel. After constructing the object, the fit method can be applied with a provided dataset and data bounds to fit the model. In fitting, the model stores a vector of coefficients coeff which satisfy differential privacy. Additionally, if a nonlinear kernel is chosen, the models stores a mapping function from the input data X to a higher dimensional embedding V in the form of a method XtoV as required (Chaudhuri et al. 2011). These can be released directly, or used in conjunction with the predict method to privately predict the label of new datapoints. Note that the mapping function XtoV is based on an approximation method via Fourier transforms (Rahimi and Recht 2007; Rahimi and Recht 2008).

Note that in order to guarantee differential privacy for the SVM model, certain constraints must be satisfied for the values used to construct the object, as well as for the data used to fit. These conditions depend on the chosen perturbation method. First, the loss function is assumed to be differentiable (and doubly differentiable if the objective perturbation method is used). The hinge loss, which is typically used for SVM, is not differentiable at 1. Thus, to satisfy this constraint, this class utilizes the Huber loss, a smooth approximation to the hinge loss (Chapelle 2007). The level of approximation to the hinge loss is determined by a user-specified constant, h, which defaults to 0.5, a typical value. Additionally, the regularizer must be 1-strongly convex and differentiable. It also must be doubly differentiable if objective perturbation is chosen. Finally, it is assumed that if x represents a single row of the dataset X, then the l2-norm of x is at most 1 for all x. In order to ensure this constraint is satisfied, the dataset is preprocessed and scaled, and the resulting coefficients are postprocessed and un-scaled so that the stored coefficients correspond to the original data. Due to this constraint on x, it is best to avoid using a bias term in the model whenever possible. If a bias term must be used, the issue can be partially circumvented by adding a constant column to X before fitting the model, which will be scaled along with the rest of X. The fit method contains functionality to add a column of constant 1s to X before scaling, if desired.

### Methods

#### Method new()

Create a new svmDP object.

##### Arguments
X

Dataframe of data to be fit.

y

Vector or matrix of true labels for each row of X.

upper.bounds

Numeric vector of length ncol(X) giving upper bounds on the values in each column of X. The ncol(X) values are assumed to be in the same order as the corresponding columns of X. Any value in the columns of X larger than the corresponding upper bound is clipped at the bound.

lower.bounds

Numeric vector of length ncol(X) giving lower bounds on the values in each column of X. The ncol(X) values are assumed to be in the same order as the corresponding columns of X. Any value in the columns of X larger than the corresponding upper bound is clipped at the bound.

Boolean indicating whether to add a bias term to X. Defaults to FALSE.

#### Method XtoV()

Convert input data X into transformed data V. Uses sampled pre-filter values and a mapping function based on the chosen kernel to produce D-dimensional data V on which to train the model or predict future values. This method is only used if the kernel is nonlinear. See Chaudhuri et al. (2011) for more details.

##### Arguments
X

Dataframe of data on which to make predictions. Must be of same form as X used to fit coefficients.

Boolean indicating whether to add a bias term to X. Defaults to FALSE. If add.bias was set to TRUE when fitting the coefficients, add.bias should be set to TRUE for predictions.

raw.value

Boolean indicating whether to return the raw predicted value or the rounded class label. If FALSE (default), outputs the predicted labels 0 or 1. If TRUE, returns the raw score from the SVM model.

##### Returns

Matrix of predicted labels or scores corresponding to each row of X.

#### Method clone()

The objects of this class are cloneable with this method.

svmDP$clone(deep = FALSE) ##### Arguments deep Whether to make a deep clone. ### References Chaudhuri K, Monteleoni C, Sarwate AD (2011). “Differentially Private Empirical Risk Minimization.” Journal of Machine Learning Research, 12(29), 1069-1109. https://jmlr.org/papers/v12/chaudhuri11a.html. Chapelle O (2007). “Training a Support Vector Machine in the Primal.” Neural Computation, 19(5), 1155-1178. doi:10.1162/neco.2007.19.5.1155. Rahimi A, Recht B (2007). “Random Features for Large-Scale Kernel Machines.” In Platt J, Koller D, Singer Y, Roweis S (eds.), Advances in Neural Information Processing Systems, volume 20. https://proceedings.neurips.cc/paper/2007/file/013a006f03dbc5392effeb8f18fda755-Paper.pdf. Rahimi A, Recht B (2008). “Weighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning.” In Koller D, Schuurmans D, Bengio Y, Bottou L (eds.), Advances in Neural Information Processing Systems, volume 21. https://proceedings.neurips.cc/paper/2008/file/0efe32849d230d7f53049ddc4a4b0c60-Paper.pdf. ### Examples # Build train dataset X and y, and test dataset Xtest and ytest N <- 400 X <- data.frame() y <- data.frame() for (i in (1:N)){ Xtemp <- data.frame(x1 = stats::rnorm(1,sd=.28) , x2 = stats::rnorm(1,sd=.28)) if (sum(Xtemp^2)<.15) ytemp <- data.frame(y=0) else ytemp <- data.frame(y=1) X <- rbind(X, Xtemp) y <- rbind(y, ytemp) } Xtest <- X[seq(1,N,10),] ytest <- y[seq(1,N,10),,drop=FALSE] X <- X[-seq(1,N,10),] y <- y[-seq(1,N,10),,drop=FALSE] # Construct object for SVM regularizer <- 'l2' # Alternatively, function(coeff) coeff%*%coeff/2 eps <- 1 gamma <- 0.1 kernel <- 'Gaussian' D <- 20 svmdp <- svmDP$new(regularizer, eps, gamma, kernel=kernel, D=D)

# Fit with data
# Bounds for X based on construction
upper.bounds <- c( 1, 1)
lower.bounds <- c(-1,-1)
svmdp$fit(X, y, upper.bounds, lower.bounds) # No bias term # Predict new data points predicted.y <- svmdp$predict(Xtest)
n.errors <- sum(predicted.y!=ytest)

[Package DPpack version 0.0.11 Index]