Confidence Levels in MarketSight
MarketSight can run statistical tests at confidence levels from 99.9% to 70%.
A confidence level of 95% sets the probability of erroneously observing statistically significant results to .05 (1 - .95). For this example, .05 is known as the Type I error. It represents the likelihood of observing statistically significant results when no significant relationship actually exists.
Lowering the confidence level increases the likelihood of seeing a significant result when none exists. Conversely, raising the confidence level decreases the likelihood of seeing a significant result when none exists.
Note that a higher Confidence Level is not necessarily ”better” because it introduces a different type of error known as Type II error. Type II error is the probability that no statistically significant result will be observed when in fact a statistically significant relationship does indeed exist between variables. Stated differently, raising the Confidence Level reduces the power of a test, where power refers to the ability of a test to correctly identify significant differences when they actually exist.