We do not censor other people more conventional-minded than ourselves. We only censor other people more-independent-minded than ourselves. Conventional-minded people censor independent-minded people. Independent-minded people do not censor conventional-minded people. The most independent-minded people do not censor anyone at all.
Conventional-minded people cannot imagine ideas that are both heretical and true. Because of this, conventional-minded people cannot distinguish between heresy and falsity. To a conventional-minded person, "heretical" and "false" are the same thing. When conventional-minded people try to ban false statements, they actually ban heresy instead.

Any attempt to censor harmful ideas actually suppresses the invention of new ideas (and correction of incorrect ideas) instead. Freedom of speech protects a society from its conventional-minded people accidentally suppressing truth in the name of safety.

This pattern doesn't just apply to freedom of speech. It applies to freedom^[1] of everything. Literally every regulation that restricts informed consenting adults from participating in mutually-beneficial activity ends up suppressing $X$ (usually in the name of safety).

You might ask yourself "Why would anyone ever want to do $X$ ?" and answer "Nobody." But that is because you're conventional-minded. If nobody wanted to do $X$ then there would be no need to restrict it. The more independent-minded you are, the more society's restrictions^[2] prevent you from creating value. The more conventional-minded you are, the more blind you are to the value that society's restrictions destroy.

When conventional-minded people advocate for restrictions on freedom, they under-estimate the damage these restrictions cause. Most people are conventional-minded. Human societies systemically err on the side of too little freedom. Society is over-regulated^[3] in every domain of human experience, except those so new or so weird that the conventional-minded haven't noticed them (yet).

By this, I mean negative rights like freedom of speech, freedom of assembly, and freedom from unreasonable search and seizure. I do not include positive rights like "freedom from want" and healthcare. Negative rights protect weirdos from normies' biases. Positive rights contort the word "freedom". Positive rights demand government proaction, which (except for a handful of self-funded governments like Saudi Arabia's) is an implicit restriction of freedom. ↩︎
I originally wrote "laws" instead of "society's restrictions". There are many ways for for a society to restrict freedom other than via laws. ↩︎
Even domains with externalities tend to over-regulated because instead of imposing a tax (which allows some freedom while pricing in the externality), people tend to support prohibition instead. ↩︎