We have some knowledge but it is not complete knowledge, not even arguably near complete. So what should we do about the areas where knowledge is lacking? We should certainly continue to investigate. But what do we say in the mean time? What can we justify saying about the unknown side of partial knowledge?
There are three basic approaches to the unknown: (a) assume as little as possible about the unknown and project that onto the unknown; (b) assume the unknown is exactly like the known and project the known onto the unknown; or (c) assume the unknown is like what is common or typical with what is known and project that onto the unknown.
Approach (a) uses the principle of indifference, maximum entropy, and a modest estimate of the value of what is known to the unknown. It takes a very cautious, anything-can-happen approach as the safest way to go.
Approach (b) uses the principle of the uniformity of nature, minimum entropy, and a confident estimate of the value of what is known to the unknown. It takes an intrepid, assertive, approach as the most knowledgeable way to go.
Approach (c) uses the law of large numbers, the central limit theorem, the normal distribution, averages, and a moderate estimate of the value of what is known to the unknown. It takes a middle way between overly cautious and overly confident approaches as the best way to go.
The three approaches are not mutually exclusive. All three may use the law of large numbers, the normal distribution, and averages. They all may sometimes use the principle of indifference or the uniformity of nature. So calling these three different approaches is a generalization about the direction that each one takes, knowing that their paths may cross or converge on occasion.
It is also more accurate to say there is a spectrum of approaches, with approaches (a) and (b) at the extremes and approach (c) in the middle. This corresponds to a spectrum of distributions with extremes of low and high variability and the normal distribution in the middle.
This suggests there is a statistic of a distribution that varies from, say, -1 to +1 for extremes of low and high variability that is zero for the normal distribution. So it would be a measure of normality, too. The inverse of the variability or standard deviation might do.
Compare the three approaches with an input-output exercise:
- Given input 0 with output 10, what is output for input 1?
- Could be anything
- The same as for input 0, namely, 10
- The mean of the outputs, namely, 10
- Also given input 1 with output 12, what is output for input 2?
- Still could be anything
- The linear extrapolation of the two points (10+2n), namely, 14
- The mean of the outputs, namely, 11
- Also given input 2 with output 18, what is output for input 3?
- Still could be anything
- The quadratic extrapolation of the two points (10+2n+n^2), namely, 25
- The mean of the outputs, namely, 40/3
- Now start over but with the additional information that the outputs are integers from 1 to 100.
- The values 1 to 100 are equally likely
- The values 1 to 100 are equally likely
- The values 1 to 100 are equally likely
- Given input 0 with output 0, what is output for input 1?
- Bayesian updating
- The same as for input 0, namely, 0
- The mean of the outputs, namely, 0
- Also given input 1 with output 5, what is output for input 2?
- Bayesian updating
- The linear extrapolation of the two points (5n), namely, 10
- The mean of the outputs, namely, 2.5, so 2 or 3 are equally likely
- Also given input 2 with output 9, what is output for input 3?
- Bayesian updating
- Since there are limits, extrapolate a logistic curve ((-15+30*(2^n) / (1+2^n)), namely, 12
- The mean of the outputs, namely, 14/3, rounded to 5
2008