Testing fairness of a die

Have you ever wondered if your dice, as you use them in board games, are fair?

This page suggests a statistical test to answer that question. The test critiria is the maximum number of occurences of one or more faces in a given number of throws/rolls.

Firstly we model the die and briefly discuss the model, then make some simplifications and finally develop the test. This includes determining a number of throws/rolls needed to check for fairness.

Modelisation, simplifications

For example in rolling a die kk=7 times, C(6+7-1,7) different combinations can be obtained, analogues to sampling with replacing, not taking the order of the throws into account. Meaning of C(nn,kk) is nn!/(kk!(nn-kk)!)

						Sum of occurences
2	0	3	1	1	0	7
0	1	1	3	0	2	7
Summarize: Sort all rows in descending order, for example the above two ones summarize into twice the follwing one:
3	2	1	1	0	0	7

For simplification of the annotation in the graph, we adapt the following:

Calculated probabilities for throwing a fair die, Test

6 throws

The blue stair like function shows the probability of all possible combinations. The combinations are annotated above the step, summarized as described above. For a fair die the probabilities of all the summarized combinations are the same. The probabilities are sorted in ascending order. At the left the graph starts with 6 points at (1/6)^6 which is for each face appearing 6 times. These combinations 600000 060000 ... summarize into 600000.

Test criteria

The green lines indicate the test criteria at a significance level of at the step closest to 5%. That means at ≈5% risk of a Type I error: rejecting a true hypothesis. Looking at the plot: In this case the hypothesis of a fair die is accepted if no face occurs more than 3 times (below 411000) and if two different faces do not appear more than 2 times each (below 330000). We can see how weak the criteria is: One face is allowed to show up to 3 times, half the number of throws!

The criteria is at 30.5% of all possible combinations (at 141 out of max 462 on the x-axis). For higher numbers of throws this percentage will increase and the test will get better. We will see the quality of the test further below when looking at Type II errors (accepting a false hypothesis).

10 throws

For throwing 10 times the green lines of the test criteria progress to 46.6% of all possible combinations (1401/3003) from 30.5% when throwing 6 times. This means it is a better test as again we shall see later.

Furthermore, for some levels of probability per combination, there is more than one summarized combination. For example combinations in 541000 and 622000, have the same probability: 10!/(5!*4!*1!)*(1/6)^10 =10!/(6!*2!*2!)*(1/6)^10=2.083810e-05. To indicate this, 622000 is noted to the right and below 541000 on the plot.

Unfair die, Quality of the test

Now we will introduce an unfair die. This is necessary to check the quality of the test. We keep the unfair die very simple. To one face we add an "unfairness", from the remaining faces we substract 1/5*the unfairness. We will add the plot for the unfair die to the fair die. However we do not sort by ascending probability but adjust the plot the probabilities of the unfair die versus the summarized combinations already obtained for the fair die. Obviously more steps will appear for the unfair die, as 820000 has no longer the same probability as 280000 if the first face has the added unfairness.

The unfair die shown has an unfairness of 0.3. The following table shows the probabilities for the fair and unfair die:

fair	0.1667	0.1667	0.1667	0.1667	0.1667	0.1667
unfair	0.4667	0.1067	0.1067	0.1067	0.1067	0.1067

Quality of test

As shown on the plot the fair die has 100%-5%=95% of its cumulated probability on the righ of acceptance limit of the green lines. This is the 5% (100%-95%) significance level. The unfair die has β=60.4% risk of acceptance (type II error). One also talks of "power of the test" μ=1-β=39.6%. β should be no more than maybe 10%, so it is much too high. The test cannot differentiate well between a die with unfairness 0.3 and a fair die. We need to increase the number of throws. We could also increase the unfairness to see from which level of unfairness a test with 10 throws is effective.

Also seeing that the acceptance limit 522100 allows one face to show 5 times, i.e. half the number of throws gives an indication of the weekness of the test.

Quality of test for different numbers of throws

We will now do exactly what we considered in the previous paragraph: Vary unfairness and numbers of throws. Then we plot β, the risk of accepting an unfair die as fair (a Type II error), versus the unfairness for different numbers of throws. The following unfairnesses are included: 0.05 0.10 0.20 0.30 0.40 0.50.

Looking at the graph on the left, we see that the higher the number of throws, the lower the risk of a Type II error. At the beginning, it is not so clear, this is because the significance level for each number is at the closest step to 5% in the density function, and varies around 5%.

β should at least be below 10%. So for the considered unfairnesses (they could further up, up to 5/6=0.833), the test with 6 throws is of no use, as the minimum beta is 31%. The test with 26 throws can be used to exclude unfairnesses of about 0.35 and above.

Further thoughts

For usual commercialised game dies, the sum of opposite faces is 7. This has not been observed for the die used in this test and shown at the top of the page (contributed by edm).

Sources / References

Comment on this page

Please log on to view and add comments. You can request a logon at the message page.