MDplot {DataVisualizations}

Mirrored Density plot (MD-plot)

Description

This function creates a MD-plot for each variable of the data matrix. The MD-plot is a visualization for a boxplot-like shape of the PDF published in [Thrun et al., 2020] with the default ordering by shape. It is an improvement of violin or so-called bean plots and posses advantages in comparison to the conventional well-known box plot [Thrun et al., 2020].

A complete guide about the MDplot can be found in https://md-plot.readthedocs.io/en/latest/index.html.

Usage

MDplot(Data, Names, Ordering='Default', Scaling="None",

Fill='darkblue', RobustGaussian=TRUE, GaussianColor='magenta',

Gaussian_lwd=1.5, BoxPlot=FALSE,BoxColor='darkred', 

MDscaling='width', LineColor='black', LineSize=0.01,

QuantityThreshold=50, UniqueValuesThreshold=12,

SampleSize=5e+05,SizeOfJitteredPoints=1,OnlyPlotOutput=TRUE,

main="MD-plot",ylab="Range of values in which PDE is estimated",

BW=FALSE,ForceNames=FALSE)

Arguments

Data

[1:n,1:d] Numerical Matrix containing the n cases of d variables. Each column is one variable. A data.frame is automatically transformed to a numerical matrix.

Names

Optional: [1:d] Names of the variables. If missing, the columnnames of data are used. If not missing, than the names can be cleaned or not (see ForceNames).

Ordering

Optional: string, either Default, Columnwise or AsIs, Alphabetical, Average, Bimodal, Variance or Statistics. Please see details for explanation.

Scaling

Optional, Default is None, Percentalize, CompleteRobust, Robust or Log, Please see details for explanation.

Fill

Optional: String or Vector, which gives the color(s) with which MDs are to be filled with.

RobustGaussian

Optional: If TRUE: each MDplot of a variable is overlayed with a roubustly estimated unimodal Gaussian distribution in the range of this variable, if statistical testing does not yield a significant p.value. In this case the packages moments, diptest and signal are required.

GaussianColor

Optional: string, color of robustly estimated gaussian, only for RobustGaussian=TRUE.

Gaussian_lwd

Optional: numerical, line width of robustly estimated gaussian, only for RobustGaussian=TRUE.

BoxPlot

Optional: If TRUE: each MDplot is overlayed with a Box-Whisker Diagram.

BoxColor

Optional: string, color of Boxplot, only for BoxPlot=TRUE.

MDscaling

Optional: if "area", all violins have the same area (before trimming the tails). If "count", areas are scaled proportionally to the number of observations. If "width" (default), all MDs have the same maximum width.

LineColor

Optional: string, color of line around the mirrored densities. NA disables this features which is usefull if ones wants to avoid vertical lines leading to outliers.

LineSize

Optional: numerical, linewidth of line around the mirrored densities.

QuantityThreshold

Optional: numeric value defining the threshold of the minimal amount of values in data. Below this threshold no density estimation is performed and a 1D scatter plot with jittered points is drawn. Only Data Science experts should change this value after they understand how the density is estimated (see [Ultsch, 2005]).

UniqueValuesThreshold

Optional: numeric value defining the threshold of the minimal amount of unique values in data. Below this threshold no density estimation and statistical testing is performed and a 1D scatter plot with jittered points drawn. Only Data Science experts should change this value after they understand how the density is estimated (see [Ultsch, 2005]).

SampleSize

Optional: numeric value defining a threshold. Above this threshold uniform sampling of finite cases is performed in order to shorten computation time.If rowr is not installed, uniform sampling of all cases is performed. If required, SampleSize=n can be set to omit this procedure.

SizeOfJitteredPoints

Optional: scalar. If not enough unique values for density estimation are given, data points are jittered. This parameter defines the size of the points.

OnlyPlotOutput

Optional: Default TRUE only a ggplot object is given back, if FALSE: Additinally, scaled data and ordering are the output of this function in a list.

main

string defining the (centered) title of the plot

ylab

string defining the y label, PDE= pareto density estimation (see [Ultsch, 2005])

BW

FALSE: usual ggplot2 background and style which is good for screen visualizations

TRUE: theme_bw() is used which is more appropriate for publications

ForceNames

FALSE: Per Default column names are cleaned for propper plotting

TRUE: forces to set the column names as given. Beware, this can result in plotting errors.

Details

In short, the MD-plot can be described as a PDE optimized violin plot. The Pareto Density Estimation (PDE) is an approach to estimate the probability density function (pdf) [Ultsch, 2005].

The MD-plot is in the process of beeing peer-reviewed [Thrun/Ultsch, 2019].

Statistical testing is performed with dip.test and agostino.test.

For the paramter Ordering the following options are possible:

Default

Ordering of plots by convex/concav/unimodal/nonunimodal shapes using statistical criteria. In this case the signal is required.

Columnwise

Ordering of plots by the order of columns of Data.

AsIs

Synonym of Columnwise: Ordering of plots by the order of columns of Data.

Alphabetical

Ordering of plots by the order of columns of Data sorted in alphabetical order by column names.

Average

Ordering of plots by the order of columns of Data sorted in order of increasing column-wise average

Bimodal

Ordering of plots by the order of columns of Data sorted in order of decreasing bimodality amplitude[Zhang et al., 2003]

Variance

Ordering of plots by the order of columns of Data sorted in order of increasing inter-quartile range

Statistics

Ordering of plots depending on the logarithm of the p-vlaues of statistical testing. In this case the packages moments, diptest and signal are required.

For the paramter Scaling the following options are possible:

None

No Scaling of data is done.

Percentalize

Data is scaled between zero and 100.

CompleteRobust

Data is first robustly scaled between zero and 1, then centered to zero and outliers are capped by a robustly formula described in RobustNormalization.

Robust

Data is robustly scaled between zero and 1 by a formula described in the RobustNormalization.

Log

Data is transformed with a sgined log allowing for negative values to be transformed with a logarithm of base 10, please see SignedLog for details.

Value

In the default case of OnlyPlotOutput==TRUE: The ggplot object of the MD-plot.

Otherwise for OnlyPlotOutput==FALSE: A list of

ggplotObj

The ggplot object of the MD-plot.

Ordering

The ordering of columns of data defined by Ordering.

DataOrdered

[1:n,1:d] matrix of ordered and scaled data defined by Ordering and Scaling.

Note that the package ggExtra is not necessarily required but if given the feature names are automatically rotated.

Note

1.) One would assume that in the first of the two following cases ggplot2 only adjusts the plotting region but:

MDplot(MTY)+ylim(c(0,7000)) is equal to MDplot(MTY[MTY<7000]).

This means in both cases the data is clipped and AFTERWARDS the density estimation is performed.

2.) Because of a (sometimes) strange behavior of either ggplot2 or reshape2, numerical column names are changed to character by adding 'C_' which can disabled using ForceNames=TRUE.

3.) Columnnames will be automatically deblanked and cleaned. To force specific columnnames the input Names can be used in combination with ForceNames=TRUE. However, this can result in plotting errors or other strange behavior.

4.) Overlaying MD-plots with robustly estimated gaussians seldomly will yield magenta (or other GaussianColor) lines overlaying more than the violin plot they should overlay, because the width of the two plots is not the same (but I am unable to set it strictly in ggplot). In such a case just call the function again.

Author(s)

Michael Thrun, Felix Pape contributed with the idea to use ggplot2 as the basic framework.

References

[Thrun et al., 2020] Thrun, M. C., Gehlert, T. & Ultsch, A.: Analyzing the Fine Structure of Distributions, PLoS ONE, Vol. 15(10), pp. 1-66, DOI 10.1371/journal.pone.0238835, 2020.

[Ultsch, 2005] Ultsch, A.: Pareto density estimation: A density estimation for knowledge discovery, in Baier, D.; Werrnecke, K. D., (Eds), Innovations in classification, data science, and information systems, Proc Gfkl 2003, pp 91-100, Springer, Berlin, 2005.

[Zhang et al., 2003] Zhang, C., Mapes, B., & Soden, B.: Bimodality in tropical water vapour, Quarterly Journalof the Royal Meteorological Society, 129(594), 2847-2866, 2003.

See Also

https://md-plot.readthedocs.io/en/latest/index.html

ClassMDplot

https://pypi.org/project/md-plot/

Examples

x = cbind(
    A = runif(2000, 1, 5),
    B = c(rnorm(1000, 0, 1), rnorm(1000, 2.6, 1)),
    C = c(rnorm(2000, 2.5, 1)),
    D = rpois(2000, 5)
  )
MDplot(x)