What can these Simulations tell you?
• Notice how the app provides a way to understand power.
• A very useful concept that emerges is the rectangular shape of the distribution when the mu chosen (the alternative population mean), matches the null value of 100. When the null is true, all pvalues are equally likely! This fact holds for all NHST tests.
• When sample size changes, power also changes, but cohen's d does not.
• Cohen's d is known to be upward biased, especially for smaller sample sizes. This is why the average simulated value is probably larger than the theoretical value, except sometimes for when the number of simulations chosen is very small (which gives an imprecise estimate). The Hedges g statistic should give a more precise estimate of d. Hedges g cannot be calculated when sample size is less than three.