Suppose again we have two people, Ahmed and Beatrice. They will each draw n balls from an urn with replacement. The urn contains white and blue balls, with θ as the unknown proportion of white balls. In a draw of n balls, let r be the number of white balls and, hence, n–r the number of blue balls. Each person’s credences may be about θ, or they may be about the probability that the next ball to be drawn will be white or blue – i.e., predictions on X~, where X~=1 represents a white ball and X~=0 represents a blue ball.

Predicting the next ball and estimating θ both correspond to different practical problems. For example, in the context of Covid-19, a doctor might be interested in the probability that the next patient she sees is positive (predicting the color of the next ball) whereas a policymaker in her city may be more interested in the proportion of the population that is positive (estimating θ). As long as one uses a proper scoring rule, the Bayesian logic follows a similar structure for either task.

Accordingly, it is not enough to have a “credence” over the space of outcomes because there are many different distributions for θ which correspond to the same predictions about which ball will be drawn. Thus, we first need to specify a full distribution for θ. Once we have that, we can make point estimates, interval estimates, and predictions about X~.

In a draw of n balls from an urn that contains only white and blue balls, the data are generated according to a Bernoulli process with the following likelihood function:

Now we need to identify prior beliefs regarding θ. In the Bayesian approach, a good candidate prior for θ when data is generated according to (1) is the so-called beta distribution, because it is a very flexible distribution with two parameters, α≥0 and β≥0, accommodating a wide variety of information states regarding a Bernoulli process and it arises naturally in the context of modeling binary exchangeable data (Lindley and Phillips, 1976). An early development of this model can be found in Johnson (1924), It was later applied by Carnap (Carnap, 1950, 1952), in his construction of the continuum of inductive methods, and by DeFinetti (De Finetti, 1937), in his refinement of Laplace’s Rule of Succession.

Let π⁢(θ) be the prior probability density for θ, where

is a beta density function with B⁢(α,β)=Γ⁢(α)⁢Γ⁢(β)/Γ⁢(α+β) and Γ⁢(n)=(n–1)!. The core of this distribution, its kernel, is given by θα–1⁢(1–θ)β–1. The combinatorial term in front is a normalizing constant. The mean of a beta distribution is given by E⁢[θ]=α/(α+β). This will be an important quantity in the material to follow, as will α and β. With the likelihood in (1) and the prior in (2), the posterior density for θ is

The posterior distribution is of the same kernel form as the prior distribution. This is because a beta distribution is conjugate to the Bernoulli process. This means that if we start with a beta prior for θ, and update via Bayes’ Rule with data from a Bernoulli process, our posterior will likewise be beta but with updated parameters.

Such a model lends itself to a very intuitive interpretation.6 The parameters of the updated beta distribution (the posterior distribution) are given by the sum of α and the number of white balls, r, together with the sum of β and the number of blue balls, n–r. As a result, the parameters α and β can be interpreted as pseudo observations or pseudo counts upon which the prior beliefs are based. For instance, to say that one has a beta(2, 2) prior for the proportion θ of white balls in the urn is equivalent to assuming that prior to making the actual draws, that person observed two balls of each color.

Bayesian updating is very simple and intuitive within a beta-Bernoulli model. If we start with a beta(7, 3) prior, and we observe 4 out of 10 white balls, our posterior for θ would be beta(7+4, 3+6). A beta(1, 1) prior for θ is the uniform or flat prior. This would also be the maximum entropy prior for a proportion.

If we want to formulate a credence about X~ (the color of the next ball to be drawn), we need the predictive distribution. Assuming that the draws are conditionally independent given θ, this is given by:

Huttegger (2017a, b) refers to this expression as the Generalized Rule of Succession and shows that this form of the predictive probability follows from several modest assumptions about the structure of the data-generating process, in particular exchangeability, which will be satisfied throughout. From a decision-theoretic perspective, the posterior mean minimizes expected square error loss. The important point for us is that when the problem is fully specified, each person will have a full distribution about θ. They will then base their prediction on the mean of that distribution.

We can now put this model to its first use and capture the notion of resilience for a credence function. Suppose A starts with a beta(1,1) prior and B starts with a beta(10, 10) prior regarding θ, the proportion of white balls in the urn. They both obtain equivalent non overlapping evidence: namely, each draws 10 balls, 6 of which are white. Using (3), we can determine that their posteriors will become beta(7, 5) and beta(16, 14), respectively. Using (4), we conclude that A’s probability that the next ball to be drawn is white moves from 0.5 to 0.58 whereas B’s moves from 0.5 to 0.53. B is much more resolute in her prior, even though the actual credal value (the valence) is the same among them. The resilience of a credence function for θ, therefore, corresponds to the size of α and β.

The higher (α+β) the more resolute the person will be about her credences. Keep in mind that after we make observations, r contributes to our new α, n–r contributes to our new β, and n contributes to α+β. As a result, the preceding definition captures Joyce’s dictum that resilience corresponds to the weight of one’s data – i.e., to n.

As the above example comparing a beta(1, 1) prior against a beta(10, 10) prior illustrates, it is possible to have equal valence and unequal resilience. This is why the credence profile sufficiency assumption is problematic. When we combine valences, there are many degrees of resilience compatible with the ensuing group credence. Which level of resilience we then impute to the group will later affect how it responds to new evidence. As a result, credence is not reducible to valence. Without capturing resilience we fail to specify every individual’s full quantification of uncertainty.

One might wonder whether the only way resilience reveals itself is under learning experiences – diachronically, so to speak. This is not the case. We can illustrate the difference between resolute and irresolute agents synchronically as well.

The point estimate for θ, θ^, is the same as the predictive probability for a single draw, P⁢(X~=1). But instead of just producing the value we think is most likely – which we know to be false anyway, since θ is continuous – we can instead produce an interval estimate for θ. In Bayesian inference, a (1–γ)⁢100% credible interval (a,b) satisfies:

Consider an example: A’s credences are beta(2, 2) whereas B’s credences are beta(100, 100). Both estimate the proportion of white balls to be 0.5, and both estimate that the probability of the next ball being white is 0.5. Their probabilities (valences) are identical, as are their predictions about the next ball. However, A’s 95% credible interval is (0.1,0.9) whereas B’s 95% credible interval is (0.43,0.57). This is a dramatic difference in uncertainty around the prediction. A is very open minded, whereas B is quite dogmatic.

Indeed, diachronic and synchronic resilience are related. As α and β increase, the variance of the distribution, given here by σ2=α⁢β/((α+β)2⁢(α+β+1)), decreases. This is to be expected – intuitively, large n implies that it is hard to change the distribution with extra information (diachronic), while small σ2 implies that the current estimate is tight (synchronic). So when we increase α and β we tighten the variance. Therefore, for a given mean (point estimate) by increasing α and β we ordinarily shrink the width of the credible interval. This is easiest to illustrate if we approximate a beta prior with a normal distribution,7 where intervals are symmetric. In this case, a 95% credible interval simplifies to μ±1.96⁢σ, and in our example,

We can see that as α and β increase, then σ decreases and so for a given μ, the length of the credible interval shrinks.

Suppose we have an urn with blue and white balls in unknown proportion θ, and two people, A and B, each of whom holds a uniform beta(1, 1) prior for θ. They each draw 10 balls, independently, and observe 4 and 7 white balls, respectively, with no overlap. What should the group distribution be?

First, since both A and B approach the problem with a uniform prior, the group prior before any observations are made should be uniform as well.8 Second, and more importantly, we have to make explicit the group’s shared evidence. We have 4+7=11 distinct white balls and 6+3=9 distinct blue balls, for a total of 20 balls. We can think of the group as accomplishing a division of labor – assigning 10 draws for A to handle, and 10 draws for B to handle. They each do their job and come back to combine the evidence. The group distribution is therefore beta(12, 10). This is a full distribution for the unknown proportion θ, from which we can derive any statistic of interest. For instance, the group’s (posterior mean) point estimate for θ is 12/22=0.54. The median is 0.55. The 95% credible interval is (0.34,0.74). We now have a full representation of the group’s uncertainty.

By contrast, if we combine credences through simple ordinary averaging, for instance, we might pool the two point estimates from A’s beta(5, 7) and B’s beta(8, 4) distribution, which would be (0.41+0.66)/2=0.53. But notice that on ordinary averaging approaches we do not have the full distribution, using instead only the probabilities as described by the credence profile. As a result, it would be impossible to determine whether person A’s 0.41 estimate came from a beta(5, 7) distribution, a beta(10, 14) distribution, a beta(20, 28) distribution, or any other beta distribution which satisfies α/(α+β)=0.41. Because all of these distributions are compatible with the reported probability, we also cannot say how the group should update if it makes additional observations. For example, if its prior distribution is beta(5, 7) and it observes 2 white balls it should move to beta(7,7) and a 0.5 estimate of θ. But if its prior distribution is beta(20, 28) and it observes 2 white balls then it should move to beta(22,28) and an estimate of 0.44 for θ. Likewise, the 95% credible interval in the beta(5, 7) case is (0.17, 0.69) whereas in the beta(20, 28) case it is (0.28, 0.55).

Further, it is well known that ordinary averaging is not commutative with respect to updating: updating and combining does not always give the same result as combining then updating.9 Our approach, by comparison, does not have this problem, and we establish and discuss this for the general case in Theorem 1 below. Indeed, it is easy to see that this will be true because we are simply summing up the number of observations in each category. Since addition is commutative, so too is the evidence-first method.

The preceding examples are particularly easy to handle because each person receives independent signals – no balls are observed together. But it is rarely the case that a group of people approach a problem with mutually exclusive private information. When some balls are observed in common, the key is to capture overlapping evidence appropriately.10 This way, everyone contributes exactly their evidence, and only their evidence, to the group belief. We now generalize the above idea and mathematically formulate the evidence-first method. This model captures both dependence and resilience.

We use the case of two people and two categories for maximal simplicity. Extending to n people and k categories is straightforward; but since the number of parameters grows quickly, in both k and n, it risks burying our message in the details. In a problem with two categories (two colors of balls), for two people, we need to know six quantities/parameters: the number of white balls each observed, the number of blue balls each observed, and the number of each color of balls observed in common.

Suppose we have two people, i=1,2, who will estimate the probability p that the next ball drawn from the urn will be white (label it as a success). Each person has her own full subjective distribution over p, which is a beta distribution with parameters ri and ni–ri:

Thus, for person i, the probability that the next ball will be white corresponds to Ei⁢[p]=rini. Moreover, ni captures the notion of resilience described above – the larger ni, the more resolute person i is that the probability is close to pi.

In order to combine these two people’s opinions/credences, we now need to model their shared information structure – which must reflect the way each came to their subjective probability distributions and any overlap in their evidence. Our model is as follows: Every person starts with a beta prior with parameters α0 and β0. Typically, these parameters will be small, like 0≤α0≤1 and 0≤β0≤1. Such a prior is proper (i.e., there exists a normalizing constant) if α0>0 and β0>0. This is often, but not always, the case. We will consider some improper priors below.

Each person will observe a few draws from this urn. This is their evidence. More specifically, αc is the number of successes observed by both people, βc is the number of failures observed by both people, αi is the number of successes observed only by person i, and βi is the number of failures observed only by person i. Now we can specify ri and ni. In particular, ri=αi+αc+α0 and ni–ri=βi+βc+β0.

The group credence function, which we will denote by Π⁢(p) (i.e., small π for the individual distribution, large Π for the group distribution), then corresponds to a situation when all of this information is combined. Therefore, it is a beta distribution with parameters r∗=r1+r2–αc–α0 and n∗–r∗=(n1–r1)+(n2–r2)–βc–β0:

This implies that the group probability is p∗=r∗n∗ and the new sample size (resilience) of the group is equal to n∗. The group probability p∗ can be expressed (using ri=pi⁢ni and n∗=n1+n2–αc–α0–βc–β0) as

This final quantity, in (8), is what the group uses as its probabilistic estimate. This is the reported probability – the valence. However, unlike approaches which focus on deriving the probability alone, we derive the full group distribution, (7), and the two are related because p∗=E⁢[p]. In sum, equations (6)-(8) describe the evidence-first method. We now state a useful theorem about this method.

Thus, unlike ordinary averaging (weighted or simple) our approach is update commutative. The proof of this theorem is straightforward. Any distribution in our framework is simply characterized by the number of successes and failures, with probability given by the proportion of successes. So when we combine the distributions, we just count the total number of successes and failures. The only wrinkle to keep in mind is that we must avoid double counting trials that were observed by both people. When we “combine and then update” we first count the total number of successes and failures in the priors, and then add new successes and failures. When we “update and then combine”, we first count successes and failures for each agent, and then count the total.

There is another aspect of the model worth flagging. We suppose our individuals have a common diffuse/uniform prior, since we use α0 and β0 instead of α0i and β0i. This assumption is not essential and it can ultimately be dropped, but because some readers may find it problematic, we explain this modeling choice. To understand why we make this assumption, consider two different cases. First, suppose A and B have no prior information, and they adopt a uniform distribution on the basis of something like the Laplacean principle of indifference.11 That is, both are completely ignorant before making the relevant observations and their prior is a true flat ur-prior.12 Thus, α0=1 and β0=1. Suppose we now want to combine these priors before updating on any information. What should the group distribution be? Our model implies that it should be beta(1,1) and not beta(2,2). This is intentional. We do not want two truly ignorant individual priors to sum up to an informative group prior. Another way to put this is that combining ignorance with ignorance should not lead to wisdom or confidence, just as 0 + 0 = 0.

Second, suppose A and B do have concrete prior information. For example, A has previously drawn balls from an urn in a game of chance at the Ringling Brothers circus whereas B has done so at the Barnum & Bailey Circus. They now find themselves together at the Ramos Brothers Circus, having to make predictions about an urn neither has previously encountered. But they happen to know that all three circuses keep a similar house edge so the proportions cannot vary too widely. Suppose they start with beta(7, 3) and beta(3, 7) priors for the proportion of white balls in the urn at the Ramos Brothers Circus. They then observe 4 draws, two of which are white and two of which are blue. What should the group distribution be?

To answer this, we must make clear the sequence of updating. What happened here is that both people updated on two sets of observations/two experiments – first, independently, at the Ringling Brothers/Barnum & Bailey Circus, and second, at the Ramos Brothers Circus, together. Thus, we need to determine what their ur-prior was before both sets of observations. Suppose again it was beta(1, 1).13 This implies that they each observed 8 balls at the first circus, which is why the sum of α and β for both is 10 before the second circus.

Which approach they use to set their ur-prior does not matter as long as there is a shared understanding of what rationality calls for in the absence of information. Accordingly, our assumption is like a weak version of the common prior familiar from microeconomic theory.14 It is “weak” in the sense that we do not assume rationality writ-large requires universal agreement about ur-priors. We simply assume that the members of the group agree in this regard. Importantly, however, they can pick any starting point and our assumption that a uniform prior corresponds to an ignorant or uninformative distribution is merely illustrative.

Given this specification of the problem – beta(1, 1) ur-priors, followed by (6,2) and (2,6) white/blue observations alone at the first circus, followed by (2, 2) white/blue observations at the Ramos Brothers Circus – the group posterior distribution becomes beta(11, 11). We subtract only the initial α0=β0=1 from the ur-prior and not the (6, 2) / (2,6) observations made at the first circus, since these are ordinary independent observations. If they started with beta(0, 0) ur-priors, the group posterior would be beta(12, 12). The message is that we must be clear both about how the prior is selected before observations are made, and about what evidence is available to each person, both individually and jointly. In short, the only burden our framework imposes is that when modeling common information, we have to be careful to model it via αc and βc and not α0 and β0.15 To further illuminate our model, we will look at several cases where (8) takes a simple form.

In our approach, the simple case where there is no evidential overlap corresponds to what Dietrich and Spiekermann (2013) would describe as a set of opinions which are common cause conditionally independent. Let us examine this kind of situation. Suppose

This is the case if both people start with completely ignorant beta(0, 0) priors and observe no information in common. That is, αc=α0=βc=β0=0. Then, from (8),

In this case, we recover the ordinary weighted averaging rule, as defended in Moss (2011) and (Pettigrew, 2019), among others, where the weights are determined by n1 and n2, the total number of each person’s observations – i.e., their resilience. This is intuitive, and indeed consistent with Pettigrew (2019)’s defense of ordinary weighted averaging because the more resolute of the two people will exert a greater weight on the group credence function. As Pettigrew suggests, it appears reasonable that the weights of an aggregation function reflect expertise – so that more knowledgeable members exert more influence on the group’s belief. It is also consistent with the interpretation given to the weights in ordinary weighted averaging in Romeijn (2024). Romeijn interprets the weights in terms of the truth conduciveness that one agent assigns to the other, which can also be thought of in terms of the trust placed in them.16 Accordingly, not only do we recover the ordinary weighted averaging rule, but in doing so we also provide a principled reason for how to assign the weights in that rule: namely, by using them to encode resilience.

Consider the case where n1=n2=n. Here things become even more straightforward since if common information is small, then we will combine individual credences by simple ordinary averaging:

This is intuitive. When resilience is equal, the weights in the ordinary averaging rule ought to be equal. And it can be motivated on similar grounds as above: if we have a group of equally knowledgeable agents, it is reasonable to assign them equal weights.

But according to (8), the prior parameters (α0 and β0) and the number of successes and failures observed by both people (αc and βc) can change this formula. From (8),

Without loss of generality, suppose that p1≤p2. Then 0≤αc+α0≤p1⁢n and 0≤βc+β0≤(1–p2)⁢n. The case where αc+α0=0, βc+β0=0 corresponds to a situation where the body of common evidence is small, and as a result, p∗=p1+p22, n∗=2⁢n. But now consider two further cases, where either αc+α0 or βc+β0 is at a maximum or a minimum.

Case 1. αc+α0=p1⁢n, βc+β0=0. Then, by (13), p∗=p2⁢n2⁢n–p1⁢n=p22–p1; n∗=(2–p1)⁢n. So not only is the combined resilience now less than 2⁢n, but p∗ is different from the average of individual probabilities. Even if p1=p2=p, the combined probability is still p∗=p2–p<p. This is because in this case successes are observed by both people together, while failures are observed by each person separately.

Case 2. αc+α0=0, βc+β0=(1–p2)⁢n. Then, by (13), p∗=(p1+p2)⁢n2⁢n–(1–p2)⁢n=p1+p21+p2; n∗=(1+p2)⁢n. As in the previous case, the combined probability is again different from what simple averaging would suggest.

We now examine the impact of the prior distributions. Assume αc=βc=0 and α0=β0=d. Let pa=p1+p22, which would be the combined probability under ordinary simple averaging. Then,

This highlights an important feature of our model, which we call extremization (Lichtendahl et al., 2021). By extremization we refer to a phenomenon that Easwaran et al. (2016) call synergy. It suggests that the group belief can lie outside the interval formed by the lower and upper bounds of individual beliefs. Examining the last line in (14), we can see that p∗ extremizes away from 12 whenever the quantity on the right side of the sum is not 0. That is, the group credence extremizes unless d=0, or pa=12, or ri=0 or ni–ri=0. Meanwhile, adopting a uniform prior in the above case would correspond to a situation where d=1, and for small n the extremization can be quite substantial. Note also that extremization will occur even if p1=p2=pa.

Extremization is not possible under ordinary averaging rules, where the group belief must lie in the convex hull of the set of individual beliefs. But we agree with Easwaran et al. (2016) that extremizing can be rational, especially in cases where, as here, the common evidence is small. Consider a more realistic scenario. A company’s executive committee is predicting whether the company will break even next year. It consists of the heads of marketing, finance, and operations. All three independently report that the company has a 97% probability of breaking even. Given that each of these executives is coming from a different area of the company, and is likely basing their forecast on largely independent evidence, it is particularly plausible that the group credence should be above 0.97. Indeed, if the credence remains at 0.97, as ordinary averaging requires, we are likely throwing away information (see also Christensen, 2011).

Finally, note that under a uniform prior, i.e., where α0=β0=d=1, the posterior distribution is proportional to the likelihood. Therefore, if we combine two distributions, and we assume that each person started with a uniform prior and received independent signals, i.e., αc=βc=0, then the combined posterior will be proportional to the product of their individual distributions. In such a case, the individual distribution of person i is beta with ri and ni–ri, and the combined distribution is beta with r1+r2–1 and n1–r1+n2–r2–1, which is what we would get if we multiply their individual distributions:

Therefore, with independent signals under uniform priors we recover the so-called Upco rule from Easwaran et al. (2016) for updating on the credences of others. Upco is derived as the product of odds ratios – for one person, the odds ratio is p/(1–p), for another it is q/(1–q), so the product is q⁢p/[(1–p)⁢(1–q)], and after normalization we obtain Upco.

But, our method produces a combined distribution for p, and the expected value of that distribution would be the probability for the next ball drawn to be white. In Upco on the other hand (as defined on Easwaran et al., 2016, pg. 3), the rule applies directly to the probabilities of the next ball, which are not sensitive to considerations of resilience. Thus our approach coincides with Upco only under uniform priors and independent signals. For example, if r1=r2=100 and n1=n2=1000, then our combined probability will still be around 10%; with Upco, if we combine p=q=10% we would get a group probability of about 1%.