3/10/2012

Frequentism, which is one way of defining and understanding probability, is incredible in its simplicity and power. This is an article to share the main ideas of frequentism. Also,
be sure to read work by John Venn (__The Logic of Chance__), Richard von Mises (__Probability, Statistics, and Truth__), and more recently, Deborah Mayo.

What do you believe will happen to the frequency of Heads if we simulate flipping a coin 7,000 times? Actually, your beliefs of what may happen don't matter. Let's just do the experiment and see what happens.

And how about 1,000,000 times?

How about 5 coins 10,000 times?

And how about 100 coins 10,000 times?

Much like the epsilon-delta proofs in calculus, one could ask someone "How close do you want to get?" to p. We could get them to within a small enough e that their measurement tool could not tell the difference, and this applies whether or not we know what the real p is.

The Strong Law of Large Numbers (SLLN) says that it is almost certain that between the mth and nth observations in a group of length n, the relative frequency of Heads will remain near the fixed value p and be within the interval [p-e, p+e], for any e > 0, provided that m and n are sufficiently large numbers. That is,

As an example, if we are going to flip a coin 1,000,000 times, and want to get within .01 of the true p (say .5 in this case), if we look at the last 100,000 flips (ie. flips 900,000 to 1,000,000) this tells us that

Could P(Heads) lie outside of this interval? Yes. Is that scenario likely? No.

Also, we shouldn't find frequentist p-values or confidence intervals confusing. Consider this graph

From the book __Statistical Sleuth__. Data originally from "General Relativity at 75: How Right was Einstein?" (Will, 1990)

Often, non-frequentist statistics rely on frequentism. For example, in some forms of Bayesian statistics, prior distributions often come from previous experiments. Also, sampling from the posterior distribution using Monte Carlo Markov Chains (MCMC) is frequentist. For example

- Use a burn-in period?

(make coin flips > some small number, since relative frequency is “rough” for a small number of flips) - Use more iterations?

(flip the coin more times, you know it will have a better chance of convergence) - Use more chains?

(flip more coins, multiple evidence of converging is better evidence of existence) - Starting with a different seed?

(and if still converges with different seeds, this is like entering a ‘collective’ randomly and still getting the same relative frequency)

If you enjoyed *any* of my content, please consider supporting it in a variety of ways:

- Check out a random article at http://statisticool.com/random.htm
- Buy what you need on Amazon from my affiliate link
- Share my Shutterstock photo gallery
- Sign up to be a Shutterstock contributor