get.asymp.power {LiftTest}

A Bootstrap Proportion Test for Brand Lift Testing (Liu et al., 2023)

Description

This function generates the asymptotic power of the proposed bootstrap test. Two methods are provided: the asymptotic power based on the relative lift and the asymptotic power the absolute lift. For more details, please refer to the paper Liu et al., (2023).

Usage

get.asymp.power(n1, n2, p1, p2, method='relative', alpha=0.05)

Arguments

n1

sample size of the control group

n2

sample size of the treatment group

p1

success probability of the control group

p2

success probability of the treatment group

method

two methods are provided: method = c(\texttt{`relative'}, \texttt{`absolute'}). \texttt{`relative'} means min sample size based on the relative lift. \texttt{`absolute'} means min sample size based on the absolute lift.

alpha

significance level. By default alpha = 0.05.

Details

Let N = n_1 + n_2 and \kappa = n_1/N. We define

\sigma_{a,n} = \sqrt{n_1^{-1}p_1(1-p_1) + n_2^{-1}p_2(1-p_2)},

\bar\sigma_{a,n} = \sqrt{(n_1^{-1} + n_2^{-1})\bar p(1-\bar p)}.

where \bar p = \kappa p_1 + (1-\kappa) p_2. \sigma_{a,n} is the standard deviation of the absolute lift and \bar\sigma_{a,n} can be viewed as the standard deviation of the combined sample of the control and treatment groups. Let \delta_a = p_2 - p_1 be the absolute lift. The asymptotic power function based on the absolute lift is given by

\beta_{Absolute}(\delta_a) \approx \Phi\left( -cz_{\alpha/2} + \frac{\delta_a}{\sigma_{a,n}} \right) + \Phi\left( -cz_{\alpha/2} - \frac{\delta_a}{\sigma_{a,n}} \right).

The asymptotic power function based on the relative lift is given by

\beta_{Relative}(\delta_a) \approx \Phi \left( -cz_{\alpha/2} \frac{p_0}{\bar p} + \frac{\delta_a}{\sigma_{a,n}} \right) + \Phi \left( -cz_{\alpha/2} \frac{p_0}{\bar p} - \frac{\delta_a}{\sigma_{a,n}} \right),

where \Phi(\cdot) is the CDF of the standard normal distribution N(0,1), z_{\alpha/2} is the upper (1-\alpha/2) quantile of N(0,1), and c = {\bar\sigma_{a,n}}/\sigma_{a,n}.

Value

Return the asymptotic power

References

Wanjun Liu, Xiufan Yu, Jialiang Mao, Xiaoxu Wu, and Justin Dyer. 2023. Quantifying the Effectiveness of Advertising: A Bootstrap Proportion Test for Brand Lift Testing. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM ’23)

Examples

n1 <- 100; n2 <- 100; p1 <- 0.1; p2 <- 0.2
get.asymp.power(n1, n2, p1, p2, method='relative')