significance_analysis {bandit} R Documentation

## significance_analysis

### Description

A convenience function to perform overall proportion comparison using prop.test, before doing pairwise comparisons, to see what outcomes seem to be better than others.

### Usage

significance_analysis(x, n)


### Arguments

 x as in prop.test, a vector of the number of successes n as in prop.test, a vector of the number of trials

### Value

a data frame with the following columns:

 successes x totals n estimated_proportion x/n lower 0.95 confidence interval on the estimated amount by which this alternative outperforms the next-lower alternative upper 0.95 confidence interval on the estimated amount by which this alternative outperforms the next-lower alternative significance p-value for the test that this alternative outperforms the next-lower alternative order order, by highest success proportion best 1 if it is part of the 'highest performing group' – those groups which were not significantly different from the best group p_best Bayesian posterior probability that this alternative is the best binomial bandit

### Note

This is intended for use in A/B split testing – so sizes of n should be roughly equal. Also, note that alternatives which have the same rank are grouped together for analysis with the 'next-lower' alternative, so you may want to check to see if ranks are equal.

### Author(s)

Thomas Lotze <thomaslotze@thomaslotze.com>

prop.test

### Examples

x = c(10,20,30,50)
n = c(100,102,120,130)
sa = significance_analysis(x,n)
sa[rev(order(sa$estimated_proportion)), ] x = c(37,41,30,43,39,30,31,35,50,30) n = rep(50, length(x)) sa = significance_analysis(x,n) sa[rev(order(sa$estimated_proportion)), ]

x = c(37,41,30,43,39,30,31,37,50,30)
n = rep(50, length(x))
sa = significance_analysis(x,n)
sa[rev(order(sa\$estimated_proportion)), ]


[Package bandit version 0.5.1 Index]