Someone deserves a large hattip for this, but I'm having trouble remembering who; my records don't seem to show any email or OB comment which told me of this 12-page essay, "Epistemic Viciousness in the Martial Arts" by Gillian Russell. Maybe Anna Salamon?

      We all lined up in our ties and sensible shoes (this was England) and copied him—left, right, left, right—and afterwards he told us that if we practised in the air with sufficient devotion for three years, then we would be able to use our punches to kill a bull with one blow.
      I worshipped Mr Howard (though I would sooner have died than told him that) and so, as a skinny, eleven-year-old girl, I came to believe that if I practised, I would be able to kill a bull with one blow by the time I was fourteen.
      This essay is about epistemic viciousness in the martial arts, and this story illustrates just that. Though the word ‘viciousness’ normally suggests deliberate cruelty and violence, I will be using it here with the more old-fashioned meaning, possessing of vices.

It all generalizes amazingly. To summarize some of the key observations for how epistemic viciousness arises:

The art, the dojo, and the sensei are seen as sacred. "Having red toe-nails in the dojo is like going to church in a mini-skirt and halter-top... The students of other martial arts are talked about like they are practicing the wrong religion."
If your teacher takes you aside and teaches you a special move and you practice it for 20 years, you have a large emotional investment in it, and you'll want to discard any incoming evidence against the move.
Incoming students don't have much choice: a martial art can't be learned from a book, so they have to trust the teacher.
Deference to famous historical masters. "Runners think that the contemporary staff of Runner's World know more about running than than all the ancient Greeks put together. And it's not just running, or other physical activities, where history is kept in its place; the same is true in any well-developed area of study. It is not considered disrespectful for a physicist to say that Isaac Newton's theories are false..." (Sound familiar?)
"We martial artists struggle with a kind of poverty—data-poverty—which makes our beliefs hard to test... Unless you're unfortunate enough to be fighting a hand-to-hand war you cannot check to see how much force and exactly which angle a neck-break requires..."
"If you can't test the effectiveness of a technique, then it is hard to test methods for improving the technique. Should you practice your nukite in the air, or will that just encourage you to overextend? ... Our inability to test our fighting methods restricts our ability to test our training methods."
"But the real problem isn’t just that we live in data poverty—I think that’s true for some perfectly respectable disciplines, including theoretical physics—the problem is that we live in poverty but continue to act as though we live in luxury, as though we can safely afford to believe whatever we’re told..." (+10!)

One thing that I remembered being in this essay, but, on a second reading, wasn't actually there, was the degeneration of martial arts after the decline of real fights—by which I mean, fights where people were really trying to hurt each other and someone occasionally got killed.

In those days, you had some idea of who the real masters were, and which school could defeat others.

And then things got all civilized. And so things went downhill to the point that we have videos on Youtube of supposed Nth-dan black belts being pounded into the ground by someone with real fighting experience.

I had one case of this bookmarked somewhere (but now I can't find the bookmark) that was really sad; it was a master of a school who was convinced he could use ki techniques. His students would actually fall over when he used ki attacks, a strange and remarkable and frightening case of self-hypnosis or something... and the master goes up against a skeptic and of course gets pounded completely into the floor. Feel free to comment this link if you know where it is.

Truly is it said that "how to not lose" is more broadly applicable information than "how to win". Every single one of these risk factors transfers straight over to any attempt to start a "rationality dojo". I put to you the question: What can be done about it?