In my article on trusting expert consensus, I talked about the value of having hard data on the opinions of experts in a given field. The unspoken subtext was that you should be careful of claims of expert consensus that don't have hard data to back them up. I've joked that when a philosopher says there's a philosophical consensus, what he really means is "I talked to a few of my friends about this and they agreed with me."

What's often really happening, though (at least in philosophy) is that the "consensus" really reflects the opinions of a particular academic clique. A sub-group of experts in the field spend a disproportionate amount of time talking to each other, and end up convincing themselves they represent the consensus of the entire profession. A rather conspicuous example of this is what I've called the Plantinga clique on my own blog—theistic philosophers who've convinced themselves that the opinions of Alvin Plantinga represent the consensus of philosophy.

But it isn't just theistic philosophers who do this. When I was in school, it was still possible to hear fans of Quine claim that everyone knew Quine had refuted the analytic synthetic distinction. Post PhilPapers survey, hopefully people have stopped claiming this. And one time, I heard a philosophy blogger berating scientists for being ignorant of the findings in philosophy that all philosophers agree on. I asked him for examples of claims that all philosophers agree on, I responded with examples of philosophers who rejected some of those claims, "Ah," he said, "but they don't count. Let me tell you who's opinions matter..." (I'm paraphrasing, but that was what it amounted to.)

I strongly suspect this happens in other disciplines: supposed "consensuses of experts" are really just the opinions of one clique within a discipline. Thus, I tend to approach claims of consensus in any discipline with skepticism when they're not backed up by hard data. But I don't actually know of verifiable examples of this problem outside of philosophy. Has other people with backgrounds in other disciplines noticed things like this?