# Polarisation of beliefs due to mistrusted information

edit: June 16, 2012 – Fixed up some of my poor grammar.

For my first post I would like to present and discuss what I consider to be one of the most interesting yet straightforward results of Bayesian inference. The following scenario is based on one given by E.T. Jaynes in his tour de force book “Probability Theory: The Logic of Science”, in chapter 5 section 3 “Converging and diverging views”. I highly recommend this book to anyone interested in probability theory and/or logical inference by the way; it is extremely enlightening on many matters.

Anyway, consider the following scenario. You are sitting at home watching TV, when Mr $N$ appears with a sensational claim that some commonly used drug is unsafe. We will call this information $D$, and call the hypothesis that the drug is safe $S$. Prior information held by each individual is denoted $I$. Along with you (Mr $A$), Mr $B$, and Mr $C$ also see this claim. Their (and your) prior beliefs that the drug is safe ($P(S|I)$ are 0.9, 0.1 and 0.9 respectively, i.e. initially Mr $A$ and Mr $C$ fairly strongly believe the drug to be safe, while Mr $B$ fairly strongly thinks it to be unsafe.

Now, one might expect that if Mr $A$, $B$ and $C$ are all rational people who reason perfectly logically in accordance with Bayesian rules, that upon receiving some information $D$ (that the drug is unsafe) that their belief in the safety of the drug would be lowered, in all cases. However, Jaynes shows that in fact it depends very much on what each rational person thinks about Mr $N$, since in fact the information they have obtained is not direct evidence that the drug is unsafe. Rather, they learn only that Mr $N$ claims that the drug is unsafe. And this turns out to be of great importance.

Let the beliefs of Mr $A$, $B$ and $C$ about the character of Mr $N$ to be as follows: They all agree that if the drug really was proved unsafe, then Mr $N$ would indeed be appearing on TV to tell them about it (so $P(D|\bar{S},I) = (1, 1, 1)$). However, they each have strongly different views about what he would do if the drug IS safe. Say A trusts his honesty, but C does not, and B is somewhere on the fence (so $P(D|S,I) = (0.01, 0.3, 0.99)$). To explain a little further, $P(D|S,I)$ is the probability that Mr $N$ would appear on TV claiming that the drug is unsafe ($D$), given that the drug is actually safe ($S$), and given the prior beliefs about the character of Mr $N$ ($I$).

Now it is straightforward from Bayes’ theorem to compute the posterior beliefs that A, B, and C will have that the drug is safe, $P(S|D,I)$:
$P(S|D,I)=\frac{P(D|S,I)}{P(D|I)}P(S|I)$
with
$P(D|I)=P(D|S,I)P(S|I) + P(D|\bar{S},I)P(\bar{S}|I)$.
The answers for each individual are 0.083, 0.032 and 0.899 for Mr $A$, $B$ and $C$ respectively. To make it more obvious what has happened allow me to summarise the transition from prior to posterior for each individual in the form $A: prior \rightarrow posterior$:

$A: 0.9 \rightarrow 0.083$ (initially thinks drug is safe $\rightarrow$ trusts Mr $N$ $\rightarrow$ no longer thinks drug is safe)
$B: 0.1 \rightarrow 0.032$ (initially thinks drug is unsafe $\rightarrow$ is wary of Mr $N$ $\rightarrow$ still doesn’t think drug is safe)
$C: 0.9 \rightarrow 0.899$ (initially thinks drug is safe $\rightarrow$ thinks Mr $N$ is a big biased liar $\rightarrow$ still thinks drug is safe)

It is important to make explicit that the variety of initial beliefs about Mr $N$ lead to varying conclusions about the safety of the drug, even though everyone involved received exactly the same information from him. It is a straightforward extension of this scenario to imagine Mr $M$ appearing on TV claiming the drug to be safe after all, but with some people thinking Mr $N$ is trustworthy, and others thinking that Mr $M$ is trustworthy. The result would then be that the population becomes polarised between the two camps of opinion about the safety of the drug, even if they had quite moderate prior beliefs. In many ways it is entirely obvious that this should happen, but it is nice to see our intuitions faithfully reproduced by the probability theory.

In this scenario the important thing is not the prior beliefs about the drug, but rather the prior beliefs each person has about the trustworthiness of Mr $N$ or Mr $M$. Jaynes goes on in the chapter to show that regardless of prior belief about the drug, the direction in which the belief in the safety of the drug moves is determined by the amount of trust each person has in the messenger. It is in fact possible for the posterior belief in the safety of the drug to move in entirely the opposite direction to that indicated by the message of Mr $N$ or Mr $M$, if a person considers it likely enough that they are lying. Unfortunately I am out of time to spend on this right now, but I might make some plots about this and talk about them in a future post.