significance_analysis {bandit} R Documentation

## significance_analysis

### Description

A convenience function to perform overall proportion comparison using prop.test, before doing pairwise comparisons, to see what outcomes seem to be better than others.

### Usage

```significance_analysis(x, n)
```

### Arguments

 `x` as in prop.test, a vector of the number of successes `n` as in prop.test, a vector of the number of trials

### Value

a data frame with the following columns:

 `successes` x `totals` n `estimated_proportion` x/n `lower` 0.95 confidence interval on the estimated amount by which this alternative outperforms the next-lower alternative `upper` 0.95 confidence interval on the estimated amount by which this alternative outperforms the next-lower alternative `significance` p-value for the test that this alternative outperforms the next-lower alternative `order` order, by highest success proportion `best` 1 if it is part of the 'highest performing group' – those groups which were not significantly different from the best group `p_best` Bayesian posterior probability that this alternative is the best binomial bandit

### Note

This is intended for use in A/B split testing – so sizes of n should be roughly equal. Also, note that alternatives which have the same rank are grouped together for analysis with the 'next-lower' alternative, so you may want to check to see if ranks are equal.

### Author(s)

Thomas Lotze <thomaslotze@thomaslotze.com>

`prop.test`

### Examples

```x = c(10,20,30,50)
n = c(100,102,120,130)
sa = significance_analysis(x,n)
sa[rev(order(sa\$estimated_proportion)), ]

x = c(37,41,30,43,39,30,31,35,50,30)
n = rep(50, length(x))
sa = significance_analysis(x,n)
sa[rev(order(sa\$estimated_proportion)), ]

x = c(37,41,30,43,39,30,31,37,50,30)
n = rep(50, length(x))
sa = significance_analysis(x,n)
sa[rev(order(sa\$estimated_proportion)), ]
```

[Package bandit version 0.5.0 Index]