significance_analysis {bandit} | R Documentation |

## significance_analysis

### Description

A convenience function to perform overall proportion comparison using prop.test, before doing pairwise comparisons, to see what outcomes seem to be better than others.

### Usage

```
significance_analysis(x, n)
```

### Arguments

`x` |
as in prop.test, a vector of the number of successes |

`n` |
as in prop.test, a vector of the number of trials |

### Value

a data frame with the following columns:

`successes` |
x |

`totals` |
n |

`estimated_proportion` |
x/n |

`lower` |
0.95 confidence interval on the estimated amount by which this alternative outperforms the next-lower alternative |

`upper` |
0.95 confidence interval on the estimated amount by which this alternative outperforms the next-lower alternative |

`significance` |
p-value for the test that this alternative outperforms the next-lower alternative |

`order` |
order, by highest success proportion |

`best` |
1 if it is part of the 'highest performing group' – those groups which were not significantly different from the best group |

`p_best` |
Bayesian posterior probability that this alternative is the best binomial bandit |

### Note

This is intended for use in A/B split testing – so sizes of n should be roughly equal. Also, note that alternatives which have the same rank are grouped together for analysis with the 'next-lower' alternative, so you may want to check to see if ranks are equal.

### Author(s)

Thomas Lotze <thomaslotze@thomaslotze.com>

### See Also

### Examples

```
x = c(10,20,30,50)
n = c(100,102,120,130)
sa = significance_analysis(x,n)
sa[rev(order(sa$estimated_proportion)), ]
x = c(37,41,30,43,39,30,31,35,50,30)
n = rep(50, length(x))
sa = significance_analysis(x,n)
sa[rev(order(sa$estimated_proportion)), ]
x = c(37,41,30,43,39,30,31,37,50,30)
n = rep(50, length(x))
sa = significance_analysis(x,n)
sa[rev(order(sa$estimated_proportion)), ]
```

*bandit*version 0.5.1 Index]