27 McNemar Test

This chapter only represents work that needed to be done for a specific application. Some of the formulas and equations provided are not necessarily coherent with the articles originally published on the topic. The majority of the work was derived from Connor’s 1987 paper. This chapter could benefit from a great deal of improvement and additional explanation.

The McNemar Test compares proportions of related samples in which the outcome for each sample is a binary response. Th response is the same in each sample. Related samples may mean subjects from one sample are matched with subjects with similiar qualities (subjects are related, but outcomes are not); or it may mean that subjects are paired with themselves, as in a pre-post design (outcomes are related because they are taken on the same subject).

Trial 2
1 0
Trial 1 1 \(p_{1,1}\) \(p_{1,0}\) \(p_1\)
0 \(p_{0,1}\) \(p_{0,0}\) \(1 - p_1\)
\(p_2\) \(1 - p_2\)



The table demonstrates the possible outcomes of such a experiment. Suppose \(Y_i\) deontes the outcome of Trial 1 and \(Y_j\) denotes the outcome of Trial 2. \(Y_{ij}\) denotes the outcome of the first and second trials, that is \(Y_{ij} = Y_i \cap Y_j\). Then:

\[\begin{aligned} p_{11} &= P(Y_i = 1 \cap Y_j = 1) \\ p_{10} &= P(Y_i = 1 \cap Y_j = 0) \\ p_{01} &= P(Y_i = 0 \cap Y_j = 1) \\ p_{00} &= P(Y_i = 0 \cap Y_j = 0) \\ \end{aligned}\]

Furthermore

\[\begin{aligned} p_1 &= p_{11} + p_{10} &= P(Y_i = 1) \\ p_2 &= p_{11} + p_{01} &= P(Y_j = 1) \end{aligned}\]

In upcoming sections, the values of the difference and sum of \(p_1\) and \(p_2\) will be important, so we define

\[\begin{aligned} \delta &= p_1 - p_2 &= (p_{11} + p_{10}) - (p_{11} + p_{01}) &= p_{10} - p_{01} \\ \\ \\ \psi &= p_1 + p_2 &= (p_{11} + p_{10}) + (p_{11} + p_{01}) &= 2p_{11} + p_{10} + p_{01} \end{aligned}\]

27.1 Sample Size Calculations for Paired Design

Three methods of calculating power for McNemar’s Test have been presented. Miettenen proposed a method of estimating the power in 1968. Duffy provided the exact power in 1984. Connor provided an additional method of estimating the power in 1987.

27.1.1 Miettenen’s Sample Size Calculation

Miettenen was the first to provide a popular power calculation for McNemar’s test with a paired-design. Duffy would later show that this calculation tends to under-estimate the power. Subsequently, sample sizes derived from this calculation are generally lower than is needed to obtain the designed power.

Let \(\alpha\) be the probability of Type I Error, and let \(\beta\) be the probability of Type II Error. Furthermore, let \(Z_\alpha = \Phi(1-\alpha)\) and \(Z_\beta = \Phi(1-\beta)\). Now suppose we wish to determine the sample size \(n_m\) (for Miettenen method) required to find a change in proportion from \(p_1\) to \(p_2\) with significance \(\alpha\) and power \(1-\beta\). The required sample size is calculated by:

\[ n_m = \frac{\Big( Z_\alpha \psi^{1/2} + Z_\beta \big(\psi - \delta^2 (3+\psi) / (4\psi) \big)^{1/2} \Big)^2}{\delta^2} \]

27.1.2 Connor’s Sample Size Calculation

Connor proposed a method for sample size calculation in addition to Miettenen’s. Connor’s method tends to over-estimate the power. Subsequently, sample sizes derived from this calculation are generally higher than is actually needed to obtain the designed power.

Let \(\alpha\) be the probability of Type I Error, and let \(\beta\) be the probability of Type II Error. Furthermore, let \(Z_\alpha = \Phi(1-\alpha)\) and \(Z_\beta = \Phi(1-\beta)\). Now suppose we wish to determine the sample size \(n_c\) (for Miettenen method) required to find a change in proportion from \(p_1\) to \(p_2\) with significance \(\alpha\) and power \(1-\beta\). The required sample size is calculated by:

\[ n_c = \frac{\big( Z_\alpha \psi^{1/2} + Z_\beta (\psi - \delta^2)^{1/2} \big)^2}{\delta^2} \]

27.2 Power Calculation for Paired Design

The following power calculations are derived in a backward fashion. In the application I had at the time, I needed to calculate sample sizes, and also wanted to allow functionality in my R function to obtain power with a supplied sample size. Since I had the sample size equations, I solved for the power. Normally this would be done the other way around, ie, take the power function and solve for \(n\). In the future, this should be revised appropriately.

27.2.1 Power Calculation for Miettenen Method

Let \(\alpha\) be the probability of Type I Error, and let \(\beta\) be the probability of Type II Error. Furthermore, let \(Z_\alpha = \Phi(1-\alpha)\) and \(Z_\beta = \Phi(1-\beta)\). The power function can be found from the sample size equation by:

\[\begin{aligned} n_m &= \frac{\Big( Z_\alpha \psi^{1/2} + Z_\beta \big(\psi - \delta^2 (3+\psi) / (4\psi) \big)^{1/2} \Big)^2} {\delta^2} \\ \Rightarrow n_m\delta^2 &= \Big( Z_\alpha \psi^{1/2} + Z_\beta \big(\psi - \delta^2 (3+\psi) / (4\psi) \big)^{1/2} \Big)^2 \\ \Rightarrow \sqrt{n_m}\delta &= Z_\alpha \psi^{1/2} + Z_\beta \big(\psi - \delta^2 (3+\psi) / (4\psi) \big)^{1/2} \\ \Rightarrow \sqrt{n_m}\delta - Z_\alpha \psi^{1/2} &= Z_\beta \big(\psi - \delta^2 (3+\psi) / (4\psi) \big)^{1/2} \\ \Rightarrow \frac{\sqrt{n_m}\delta - Z_\alpha \psi^{1/2}}{\big(\psi - \delta^2 (3+\psi) / (4\psi) \big)^{1/2}} &= Z_\beta \\ \Rightarrow \Phi^{-1}\bigg(\frac{\sqrt{n_m}\delta - Z_\alpha \psi^{1/2}} {\big(\psi - \delta^2 (3+\psi) / (4\psi) \big)^{1/2} }\bigg) &= 1 - \beta \end{aligned}\]

27.2.2 Power Calculation for Connor Method

Let \(\alpha\) be the probability of Type I Error, and let \(\beta\) be the probability of Type II Error. Furthermore, let \(Z_\alpha = \Phi(1-\alpha)\) and \(Z_\beta = \Phi(1-\beta)\). The power function can be found from the sample size equation by:

\[\begin{aligned} n_c &= \frac{\big( Z_\alpha \psi^{1/2} + Z_\beta (\psi - \delta^2)^{1/2} \big)^2}{\delta^2} \\ \Rightarrow n_c\delta^2 &= \big( Z_\alpha \psi^{1/2} + Z_\beta (\psi - \delta^2)^{1/2} \big)^2 \\ \Rightarrow \sqrt{n_c\delta} &= Z_\alpha \psi^{1/2} + Z_\beta (\psi - \delta^2)^{1/2} \\ \Rightarrow \sqrt{n_c\delta} - Z_\alpha \psi^{1/2} &= Z_\beta (\psi - \delta^2)^{1/2} \\ \Rightarrow \frac{\sqrt{n_c\delta} - Z_\alpha \psi^{1/2}}{(\psi - \delta^2)^{1/2}} &= Z_\beta \\ \Rightarrow \Phi^{-1}\bigg(\frac{\sqrt{n_c\delta} - Z_\alpha \psi^{1/2}}{(\psi - \delta^2)^{1/2}}\bigg) &= 1 - \beta \end{aligned}\]