Table 3.

Comparison of Multiple-Test Methods by Simulation

Method
Type I Errors
Type II Errors
Observed FDR
Observed PER
Observed FWER
Bonferroni 0.171 ± 0.01 68.4 ± 0.1 0.014 ± 0.001 0.012 ± 0.001 0.154 ± 0.01
Holm 0.174 ± 0.01 68.2 ± 0.1 0.014 ± 0.001 0.012 ± 0.001 0.157 ± 0.01
Hochberg 0.174 ± 0.01 68.2 ± 0.1 0.014 ± 0.001 0.012 ± 0.001 0.157 ± 0.01
Hommel 0.179 ± 0.01 68.0 ± 0.1 0.014 ± 0.001 0.012 ± 0.001 0.162 ± 0.01
Benjamin and Hochberg 12.1 ± 0.14 21.0 ± 0.1 0.167 ± 0.002 0.174 ± 0.001 1.000
Storey and Tibshirani
16.3 ± 0.20
17.9 ± 0.1
0.202 ± 0.002
0.215 ± 0.001
1.000