Jeffrey Pooling

1. Upco Ensures Jeffrey Pooling Commutes

Splitting the difference between two opinions is known as linear pooling. The formula is just the familiar arithmetic mean:

where $P(E)$ is your prior opinion about $E$, before reading any columns, and $Q(E)$ is the columnist’s probability. In our example $P(E)=0.4$ and $Q(E)=0.8$, so $P^{\prime }(E)=0.6$.

But we’ll see that commutativity instead favours upco, also known as multiplicative pooling:

If $P(E)=0.4$ and $Q(E)=0.8$, then $P^{\prime }(E)\approx 0.73$, significantly larger than the $0.6$ recommended by linear pooling.

These two formulas are examples of pooling rules, functions that take two probabilities $P(E)$ and $Q(E)$ and return a new probability $P^{\prime }(E)$. Two more examples come from the other notions of ‘mean’ included in the classical trio of Pythagorean means: the geometric and harmonic means (Genest and Zidek, 1986). And there are many more, too many to name.

Our question is how these various rules behave when coupled with Jeffrey conditionalization. Suppose we begin with $P$, fix some pooling rule $f$, and use the following two-step procedure for responding to $Q$’s opinion about $E$.

Jeffrey Pooling:

• Step 1. Apply pooling rule $f$ to $P(E)$ and $Q(E)$ to obtain $P^{\prime }(E)$:

  $P^{\prime }(E)=f(P(E),Q(E)).$

• Step 2. Revise all other credences by Jeffrey conditionalization:

  $P^{\prime }(H)=P(H\mid E)P^{\prime }(E)+P(H\mid \overline{E})(1–P^{\prime }(E)).$

We will call this Jeffrey pooling $P$ with $Q$ on $E$ using $f$. But that’s a mouthful, so we’ll often leave some of these parameters implicit when context permits. We’ll say that $f$ ensures Jeffrey pooling commutes if, for any $P$, $Q$, and $R$, Jeffrey pooling $P$ with $Q$ on $E$ and then Jeffrey pooling the result with $R$ on $F$, has the same final result as Jeffrey pooling $P$ with $R$ on $F$ and then Jeffrey pooling the result with $Q$ on $E$.

Upco ensures that Jeffrey pooling commutes, as long as the necessary operations are defined. Zeros can gum up the works in two ways. First, if $P(E)=1$ and $Q(E)=0$ or vice versa, then Step 1 fails: upco cannot be applied, because its denominator is $0$. Second, the conditional probabilities used in Step 2 need to be defined, so $P(E)$ cannot be either $0$ or $1$. For a subsequent update on $F$ to have defined conditional probabilities as well, we also need the updated probability of $F$ to be non-extreme.

To avoid these difficulties, we will temporarily make the simplifying assumption that $P$ is regular, i.e. that it assigns positive probability to $EF$, $E\overline{F}$, $\overline{E}F$, and $\overline{\mathit{\text{EF}}}$. This ensures no problematic zeros arise when Jeffrey pooling on $E$ and then $F$, or vice versa. In the Appendix we show that this assumption can be dropped; the result we are about to present holds whenever the relevant Jeffrey pooling operations are defined, even if $P$ is not regular.

If $P$ is regular, then upco is sufficient to make Jeffrey pooling commutative. We attribute this result to Field (1978) for reasons that will become clear in Section 3.

Theorem 1 (Field). Upco ensures that Jeffrey pooling commutes for any regular $P$, and any $Q$ and $R$.

In the Appendix we generalize this result to pooling over countable partitions, i.e. to cases where we don’t just hear $Q$’s opinion about $E$, but about every element in a countable partition.

An example makes clear why Theorem 1 is true. Recall the case we opened with, where $P(E)=4/10$ and $Q(E)=8/10$. Let’s further suppose that $R(F)=6/10$, and that $P$ has the following details:

According to Theorem 1, $P$’s final opinions will be the same whether they Jeffrey pool with $Q$ first and $R$ second, or vice versa, provided they use upco for the first step in Jeffrey pooling.

Begin with the case where $P$ pools with $Q$ first. Step 1 of Jeffrey pooling combines $P(E)=4/10$ with $Q(E)=8/10$ via upco, to yield $P^{\prime }(E)=8/11$. For Step 2, the key is to observe that the relative proportions of $EF$ and $E\overline{F}$ must be preserved—this is Jeffrey conditionalization’s oft-noted “rigidity”. So the $8/11$ assigned to $E$ must be divided $3$-to-$1$ between $EF$ and $E\overline{F}$. Similarly, the $3/11$ assigned to $\overline{E}$ gets divided $1$-to-$2$ between $\overline{E}F$ and $\overline{\mathit{\text{EF}}}$. The posterior $P^{\prime }$ that results is:

Now we pool $P^{\prime }$ with $R$ using similar reasoning. Applying upco to $P^{\prime }(F)=7/11$ and $R(F)=6/10$ gives $P^{\prime \prime }(F)=21/29$. Jeffrey conditionalization then divides this up proportionally to arrive at:

In the case where $P$ pools with $R$ first and $Q$ second, parallel calculations give the following sequence instead:

As Theorem 1 claimed, the ultimate posterior $P^{\prime \prime }$ is the same either way.

This convergence may seem magical, but its inevitability emerges if we look past the denominators to the relative proportions. We began with the proportions $3:1:2:4$. Then we multiplied the first two entries by $8$ and the last two by $2$, since $Q(E)=8/10$ and $Q(\overline{E})=2/10$. This gave us for $P^{\prime }$ the relative proportions $24:8:4:8$, although we divided through by the common factor $4$ to write this as $6:2:1:2$. Then, because $R(F)=6/10$ and $R(\overline{F})=4/10$, we multiplied the first and third entries by $6$ and the second and fourth by $4$, to get $36:8:6:8$—although again we divided through by a common factor to write this as $18:4:3:4$.

Updating in the the opposite order, we began again with $3:1:2:4$ but multiplied the first and third entries by $6$, the second and fourth by $4$, to get $18:4:12:16$, which reduced to $9:2:6:8$. Then we multiplied the first two entries by $8$ and the last two by $2$, to get $72:16:12:16$, or $18:4:3:4$.

In both cases the final proportions had to be the same because, ultimately, all we did was multiply the values $3$, $1$, $2$, and $4$ by the values $8\cdot 6$, $8\cdot 4$, $2\cdot 6$, and $2\cdot 4$, respectively (then divide the results by a common factor). We can think of this as multiplying by the values $8$, $8$, $2$, $2$ first, and by $6$, $4$, $6$, $4$ second, or the other way around. The commutativity of Jeffrey pooling with upco follows by the commutativity of multiplication.

2. Only Upco Ensures Jeffrey Pooling Commutes

While upco ensures that Jeffrey pooling commutes, linear pooling doesn’t; nor do geometric and harmonic pooling. Indeed, among the pooling rules that boast four plausible properties—properties the rules just named all share—upco is the only one that ensures this. As we will indicate in the course of introducing these properties, we don’t think they will be desirable in all situations. But we do claim that they are desirable in a great many important ones. And in those cases, upco is the only rule that delivers.

The first property is monotonicity: if we fix $Q(E)=1/2$, then as $P(E)$ increases, so does $P^{\prime }(E)$. This is a familiar feature of linear pooling, and upco has it too.⁶ Notice that this is also a feature of conditionalization in many cases. For any proposition $Q$, conditionalization sets $P^{\prime }(E)=P(E\mid Q)$, which Bayes’ theorem renders

If the likelihood terms $P(Q\mid E)$ and $P(Q\mid \overline{E})$ stay fixed as $P(E)$ changes, then $P^{\prime }(E)$ increases with $P(E)$.⁷

The second property our argument will rely on is uniformity preservation: if $P(E)=Q(E)=1/2$, then $P^{\prime }(E)=1/2$ too. Crudely put, two empty heads are no better than one. A bit less crudely, if neither party has an opinion about the question at hand, then combining their opinions doesn’t change this. There are conceivable cases where this feature would be undesirable. For example, the fact that both parties are so far ignorant about a question could indicate a conspiracy to keep everyone in the dark. But such cases are the exception rather than the rule.

Third is continuity: in nearly all cases, if we fix $Q(E)$ and let $P(E)$ approach a value $c$, then the pool of $c$ and $Q(E)$ should be the limit of the pools of $P(E)$ and $Q(E)$ as $P(E)$ approaches $c$. Nearly all? Yes, because we have to ensure that all of the pools just mentioned are defined. So we restrict to cases in which, as $P(E)$ approaches $c$, the pool of $P(E)$ and $Q(E)$ is always defined, and the pool of $c$ and $Q(E)$ is as well.

To illustrate continuity, fix $Q(E)=1/2$ and consider what happens in linear pooling as $P(E)$ decreases to $0$. As $P(E)$ gets smaller, the value of $P^{\prime }(E)$ gets closer and closer to $1/4$. And, indeed, that is the value $P^{\prime }(E)$ takes when $P(E)$ finally does reach $0$. There is no sudden jump in the value of $P^{\prime }(E)$ when $P(E)$ finally hits $0$.

As with uniformity preservation, there are conceivable cases where this feature would not be appropriate. These might arise if we were to think that some probabilities have a particular significance. For instance, a Lockean might think there is a probabilistic threshold beyond which you count as believing the proposition to which you assign the probability, but below which you don’t. And they might think that sudden change in doxastic status should be reflected in our pooling rule—perhaps your probability gains more weight when it suddenly becomes a belief. We’ll assume this isn’t the case.

Our fourth property is symmetry: swapping the values of $P(E)$ and $Q(E)$ makes no difference to $P^{\prime }(E)$. This is perhaps the most restrictive feature, since exceptions are commonplace. When one party is more competent or better informed than the other, it matters who holds which opinion. Frequently we will want to give more “weight” to $P(E)$ than to $Q(E)$, or vice versa, in which case exchanging their values should make a difference.

But our argument only concerns cases where this is not so: cases where the two parties are equally competent and well informed on the topic.⁸ When e.g. one party has more information, upco may not be appropriate (although in some cases it will be appropriate even then).

There are asymmetrically weighted versions of the various pooling rules we’ve mentioned, which may be appropriate to such cases. But we won’t address these cases here. If we can show that upco is specially suited when symmetry is appropriate, that will be a significant step forward. Not to mention a strong indicator that a weighted version of upco would be the way to go in some asymmetric cases.

Finally, there’s an assumption implicit in the very idea of a pooling rule, which we should pause to examine. Since a pooling rule is a function of $P(E)$ and $Q(E)$ and nothing else, we are assuming from the outset that $P(E)$ and $Q(E)$ are the only factors relevant to $P^{\prime }(E)$. But other of $P$’s opinions could be relevant, such as their opinion about what evidence $Q(E)$ is based on. Even the fact that it’s an opinion about the proposition $E$, and not some other proposition, could be relevant. Someone might be competent on the topic of $E$ but incompetent on the topic of $F$. In which case you might apply one formula when faced with their opinion about $E$, but use another should they opine about $F$.

So there is a tacit fifth assumption here, which we might call extensionality. By assuming extensionality, however, we are not assuming that there is one pooling rule appropriate to all circumstances, regardless of your background beliefs or the content of the question under discussion. On the contrary, different rules will be suited to different circumstances. But the question we are asking is: which rules are suited to circumstances where the above four conditions hold, Jeffrey conditionalization is appropriate, and the order in which sources are consulted should not matter.

In answer to this question, we offer the following result.

Theorem 2. Among the monotonic, continuous, uniformity preserving, and symmetric pooling rules, only upco ensures that Jeffrey pooling commutes for any regular $P$, and any $Q$ and $R$.

As we noted in connection with Theorem 1, upco ensures Jeffrey pooling commutes even when $P$ is not regular, provided the relevant operations are defined. But Theorem 2 tells us no other pooling rule can claim this feature, even if we restrict our attention to regular $P$.

It’s important to appreciate what this result does not say: it does not tell us that rules like linear pooling never commute. It is possible to get lucky with linear pooling and encounter two sources where the order doesn’t matter. For example, suppose $Q$ already agrees with $P$ about $E$, and $R$ agrees with $P$ about $F$. Then, linear pooling will keep $P$’s opinion fixed throughout. Whichever order they encounter $Q$ and $R$ in, their opinion at the end will be the same as when they started. But Theorem 2 tells us this can’t be counted on to hold generally; only upco is commutative regardless of the particulars of $P$, $Q$, and $R$.

It’s also important to recognize that there are cases where the order should matter. For example, imagine you’re interviewing pundits instead of reading pre-written opinion columns. And pundit $Q$ can be counted on for a serious opinion if you consult them first, but they’ll be so insulted if you talk to $R$ first that they’ll lose their cool and adopt wild views. Then it really matters what order you hear their opinions in.

But again, we do not mean to argue that upco is always the best rule. Rather, we aim to show that upco is the only rule that will serve in all cases where the assumptions we’ve laid out are reasonable. And one of those assumptions is that the order shouldn’t matter.

That completes our argument for upco. We now turn to locating Theorems 1 and 2 in the context of existing work on Jeffrey conditionalization and commutativity. In Section 3, we show a surprising and illuminating connection with an early result due to Field (1978). Then, in Section 4, we explain how Wagner’s (2002) theorems relate.

3. Testimony of the Senses

Field (1978) was the first to identify conditions that make Jeffrey conditionalization commutative. How does his discovery fit with our results, especially Theorem 1?

Field discusses cases where sensory experience, rather than another person’s opinion, prompts the shift from $P(E)$ to $P^{\prime }(E)$. He assumes that each experience has an associated proposition $E$ and number $\beta \ge 0$, where $\beta$ reflects how strongly the experience speaks in favour of $E$.⁹

Field’s proposal is that we should respond to sense experience by the following two-step procedure.

Field Updating:

• Step 1. Update from $P(E)$ to $P^{\prime }(E)$ using $\beta$ as follows:

  $P^{\prime }(E)=\frac{\beta \cdot P(E)}{\beta \cdot P(E)+P(\overline{E})}.$
  (2)

• Step 2. Update other credences by Jeffrey conditionalization:

  $P^{\prime }(H)=P(H\mid E)P^{\prime }(E)+P(H\mid \overline{E})(1–P^{\prime }(E)).$

We will call this procedure Field updating on $(E,\beta )$. Field shows that his procedure is commutative: Field updating on $(E,{\beta }_1)$ and then $(F,{\beta }_2)$ has the same result as Field updating on $(F,{\beta }_2)$ followed by $(E,{\beta }_1)$.

This may sound familiar. And if you squint, you might see that Field’s Equation (2) is actually the same as upco’s Equation (1). It’s just that $\beta$ is on the odds scale from $0$ to $\mathrm{\infty }$, rather than the probability scale from $0$ to $1$. To convert from odds to probabilities, we can divide through by $\beta +1$, in both the numerator and the denominator:

And this is the same as Equation (1), where $Q$’s probabilities are $Q(E)=\beta /(\beta +1)$ and $Q(\overline{E})=1/(\beta +1)$.

So, formally speaking, Field updating is the same thing as Jeffrey pooling with upco. And Theorem 1 is just a restatement of Field’s classic result.

This formal parallel suggests two helpful heuristics for thinking about Field’s way of responding to sensory experience.

First, we might think of Equation (3) as pooling your prior opinion with a “naive” opinion proposed by your sensory system. Notice that, when $P(E)=P(\overline{E})$, Equation (3) delivers $P^{\prime }(E)=\beta /(\beta +1)$. So if you have no prior opinion about $E$, you will defer to your sensory system’s proposal, $\beta /(\beta +1)$. We can thus think of $\beta$ as the odds your sensory system recommends based on the experience alone, absent any prior information.

However, when you do have a prior opinion about $E$, the naive recommendation has to be merged with it. Field’s proposal is to use upco to combine the naive recommendation with your prior opinion, which makes updates commutative under Jeffrey conditionalization. Indeed, Theorem 2 shows that Field’s proposal is the only way to do this using a monotonic, continuous, uniformity preserving, and symmetric pooling rule.

A second way of understanding Field’s proposal exploits a formal analogy between upco and Bayes’ theorem. Notice that Equation (3) just is Bayes’ theorem, if we think of the $\beta$ terms not as unconditional probabilities, but as likelihoods. That is, imagine we are calculating $P^{\prime }(E)=P(E\mid E^{\ast })$ for some proposition $E^{\ast }$. If the likelihoods are $P(E^{\ast }\mid E)=\beta /(\beta +1)$ and $P(E^{\ast }\mid \overline{E})=1/(\beta +1)$, then Equation (3) is just Bayes’ theorem.

What is the proposition $E^{\ast }$ here? Let $E^{\ast }$ describe all epistemically relevant features of the experience prompting the update. The original motivation for Jeffrey conditionalization was that you may not be able to represent $E^{\ast }$ at the doxastic level—or maybe you can, but you don’t have any priors involving $E^{\ast }$, because it’s too subtle or specific. So you can’t conditionalize, because $P(E\mid E^{\ast })$ is undefined.

But we can extend $P$ to a compatible distribution $P^{+}$ that does encompass $E^{\ast }$, by stipulating

Then Equation (3) becomes conditionalization via Bayes’ theorem:

So this interpretation conceives of Field’s proposal as conditionalizing on the ineffable but epistemically essential qualities of sensory experience, by relying on the sensory system to do the effing and the expecting—i.e. to represent the experience’s epistemically relevant features, and supply the likelihood values Bayes’ theorem requires.

4. Wagner’s Theorems

There’s also an important connection between our Theorem 2 and a classic result about Jeffrey conditionalization due to Wagner (2002).

Wagner analyzes Jeffrey conditionalization in terms of “Bayes factors.” When we update a probability distribution from $P$ to $P^{\prime }$, the Bayes factor of $E$ is the ratio of its new odds to its old odds:¹⁰

Crudely put, Wagner’s insight is that Jeffrey conditionalization commutes when, and pretty much only when, the Bayes factors are consistent regardless of the order. This needs some explaining.

Suppose two agents begin with the same prior distribution, $P$. Then they update as in Figure 1. That is, one does a Jeffrey conditionalization update on $E$ that yields a Bayes factor of $B_1^E$, followed by another on $F$ that yields a Bayes factor of $B_1^F$. The second agent starts with a Jeffrey conditionalization update on $F$ that yields the Bayes factor $B_2^F$, then does a second on $E$ that yields the Bayes factor $B_2^E$. At the end of this process, we label their posteriors $P_1^{\prime \prime }$ and $P_2^{\prime \prime }$, respectively.

Figure 1

The context for Wagner’s Theorems 3 and 4

Wagner’s first result is that the two agents will end up with the same ultimate posterior if the Bayes factors for their respective $E$ updates are the same, and likewise for their $F$ updates. As before we will assume regularity to ensure everything is defined.¹¹

Theorem 3 (Wagner). In the schema of Figure 1, if $P$ is regular, then $B_1^E=B_2^E$ and $B_1^F=B_2^F$ together imply $P_1^{\prime \prime }=P_2^{\prime \prime }$.

Loosely speaking, Bayes factor “consistency” is sufficient for Jeffrey conditionalization updates to commute.

Field updating produces exactly this sort of consistency. We can verify with a bit of algebra that a given input value $\beta$ always yields the same Bayes factor. In fact, solving for $\beta$ in Equation (2) we find that $\beta$ just is the Bayes factor:

So we can think of Field’s Theorem 1 as a corollary of Wagner’s Theorem 3.

But, crucially for us here, Wagner also shows that this kind of Bayes factor consistency is necessary for commutativity, in almost every case. Exceptions are possible, for example if $E$ and $F$ are the same proposition. But our regularity assumption precludes this since $E\overline{F}$ can’t have positive probability if $E=F$. In fact, regularity suffices to rule out all exceptions.¹²

Theorem 4 (Wagner). In the schema of Figure 1, if $P$ is regular then $P_1^{\prime \prime }=P_2^{\prime \prime }$ implies $B_1^E=B_2^E$ and $B_1^F=B_2^F$.

Does this theorem mean that only Field’s Equation (2) can make Jeffrey conditionalization commute? No, other rules can also consistently yield the same Bayes factor for the same value of $\beta$.

One silly example is the “stubborn” rule, which just ignores $\beta$ and always keeps $P^{\prime }(E)=P(E)$. Substituting this rule into Step 1 of Field updating makes the Bayes factor $1$ for all updates. And, trivially, updating this way is commutative: if you never change your mind, the order in which you encounter various sensory experiences won’t make any difference to your final opinion.

A less trivial example—call it “upsidedownco”—replaces Field’s Equation (2) with

Doing a bit of algebra to isolate $\beta$, we find that this implies

So the same value of $\beta$ always results in the same Bayes factor. By Theorem 3 then, this variation on Field updating is also commutative.

However, both of these alternate rules violate the conditions we laid out in Section 2. Specifically, they violate symmetry. The stubborn rule is plainly not symmetric, since it privileges $P(E)$ and neglects the $\beta$ proposed by experience entirely. And upsidedownco increases $P^{\prime }(E)$ as $P(E)$ increases, yet decreases $P^{\prime }(E)$ as $\beta$ increases.

So Wagner’s Theorem 4 is not, by itself, enough to secure Field’s proposed Equation (2). Or, returning now to the social interpretation of upco and Equation (1), Wagner’s result doesn’t secure our Theorem 2. But with the help of further conditions like symmetry, we can rule out alternatives like the stubborn rule and upsidedownco. And this is exactly how our proof of Theorem 2 proceeds. We pick up where Wagner’s result leaves off, using the four conditions of Section 2 to rule out any option but upco.

5. Conclusion

No way of combining probabilities is best for all purposes. For some purposes, there are even impossibility results showing that no pooling rule will get you everything you want.¹³ But for some purposes, we can identify a single pooling rule that is the only one that will do. If your purpose is to combine your probability with an epistemic peer’s and Jeffrey conditionalize on the result, and you want to be assured of commutativity, then upco is the only monotonic, continuous, uniformity preserving, and symmetric game in town.

6. Appendix: Theorems & Proofs

Here we generalize and prove Theorems 1, 2 and 4. We don’t prove Theorem 3, proving Theorem 1 directly instead, for simplicity. Readers interested in a proof of Theorem 3 can consult Wagner (2002, Theorem 3.1).