Michael Tobis wrote a response to Judith Curry’s posts on dealing with uncertainty. It seemed that the reaction to this was largely clouded by objections to the tone of the writing rather than the substance. As a “blogospheric experiment” if you will, I decided to take the loaded terms out of the essay, leaving only the argument itself, to see if the reaction might be any different.
This was done without consulting Dr. Tobis, so any errors introduced by my redaction are mine alone.
One brief note: I am turning moderation off in the spirit of allowing a freewheeling debate. This of necessity means that all of the posts on this personal blog will be unmoderated, including many that are of a fairly personal nature. I’d ask for your restraint in responding to any of those.
Judith Curry’s concept of how to frame uncertainty is confusing and contradictory.
You can read her essay “Doubt” for some context on this:
Curry:
Lets frame belief, disbelief, and doubt in the context of the Italian flag, that was introduced previously on the hurricane thread in which evidence for a hypothesis is represented as green, evidence against is represented as red, and the white area reflecting uncommitted belief that can be associated with uncertainty in evidence or unknowns.
Let’s look at an example in the above-linked article:
Most of the observed increase in global average temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations.
This statement is often used as a litmus test for belief regarding global warming, i.e. you believe this statement (consensus) or you don’t (skeptic). Very likely denotes a probability of anthropogenic influence between 90 and 99% (lets pick 95%) and I interpret most to mean between 51 and 90% (lets pick 70%), with the remainder (30%) associated with natural variability. Hence, the Italian flag analysis could represent this in the following way:
5% assigned to uncommitted belief (white),
67% assigned to anthropogenic forcing (green),
28% assigned to natural variability (red).
…
my personal weights for the Italian flag are:
white 40%,
green 30%,
red 30%.
My assignment allows the anthropogenic influence to be as large as 70% and as small as 30%
As I have pointed out previously, that last sentence conflates a hypothesis (a proposition that must be either true or false) with a weighting.
This leads to an unambiguous contradiction.
Suppose we eliminate the white altogether, and consider only the yes and the no. Suppose I had a measurement with a very well-characterized uncertainty of the quantity. My belief that the majority of the warming is attributable to anthropogenic influence, say, is in line with IPCC:
- p (f < 50%) : 0.02
- p (f > 50%) : 0.98
Suppose, to be more specific, I believed (consistent with the above) as follows:
- p (f < 50%) : 0.02
- p (50 % <= f < 60%) : 0.02
- p (60 % <= f < 70%) : 0.1
- p (70 % <= f < 80%) : 0.72
- p (80 % <= f < 90%) : 0.1
- p (90 % <= f <= 100%) : 0.02
Then I would be quite confident that the percentage would be in the range 60% … 90 %. But according to Curry’s construction, “My assignment allows the anthropogenic influence to be as large as 98% and as small as 98%”
In other words, a well defined uncertainty yields an inconsistent certainty.
QED
This leaves aside how to deal with the third value in a two valued logic.
In addressing that, consider that if one believes that the odds of a proposition A is P, then according to ordinary rules the odds of not-A is assigned a value of (1oo% – P).
The desire to separate out uncertainty from “uncommitted belief” is something that a lot of people have already thought about it. Read up on frequentists vs Bayesians.
Curry’s construction effectively demonstrates the problems with using frequentist logic in situations where an estimate is required, rather than a hypothesis tested. Suppose, to return to a favorite example, one has in view a creature and wishes to estimate whether it is a duck.
The evidence may be sufficient for a frequentist to state with confidence that the probability of the present observations in the case of a duck is P, and the probability of the present observations in the case of a not-duck is Q. In most cases (lacking a huge observational set), the sum of P and Q will be less than 1. Indeed, once there is enough evidence to make P and Q add to 1, it would be very pathologically strange not to be able to state with certainty the duckiness of the creature in question; either P will be one and Q zero or the other way round. That’s essentially at the core of frequentist analysis.
I think this gap is what Curry may be trying to grapple with in her white zone, given that in earth science we often lack enough data for a compelling frequentist analysis of important questions.
Neither P nor Q is an estimate of the probability that, given the observations, one sees a duck. This depends, it turns out, on the expected rarity of ducks! Assuming you are willing to quantify your prior belief in ducks, you can make a consistent argument for the propbability space being partitioned into duck and not-duck with
none of that peculiar middle ground.
In the light of all this, let’s consider her most recent contribution, which begins
Historical surface temperature observations over the 20th century show a clear signal of increasing surface temperatures. Italian flag: Green 70%, White 30%, Red 0%. (Note: nobody is claiming that the temperatures have NOT increased.)
OK, is it fair to say that “Note: nobody is claiming that the temperatures have NOT increased” means that nobody claims that “Historical surface temperature observations over the 20th century show a clear signal of increasing surface temperatures”? I mean, if nobody can claim the contrary, then the signal must be clear, right? So what does that white 30% represent?
