AI ALIGNMENT FORUM
AF

Wikitags

LessWrong Jargon

Written by David_Gerard, fubarobfusco, taryneast, pedanterrific, John Maxwell, Grognor, brook, TheAncientGeek, et al. last updated 21st Jan 2023

This is a short list of common terms and phrases, i.e., jargon used on LessWrong.


A

ADBOC: Agree Denotationally, But Object Connotatively. Discussion in When Truth Isn't Enough

AFAICT: As Far As I Can Tell

Affect: Mood or emotion as demonstrated in external physical signs.

: When positive attributions combine with the in a positive feedback loop.

AGI:

: Bad rules for thinking itself, capable of protecting false beliefs.

B

:

  • A theory of probability based on updating subjective estimates in the light of new evidence. Contrasted with the frequentist approach, which views probability as being the mean of an infinite series of the same experiment.
  • Probablistic reasoning in general.
  • Good, well done reasoning in general.

: A fictional secret society of Bayesians.

: What you do to your beliefs, opinions and cognitive structure when new evidence comes along.

Black swan: In the usage of , a black swan is a rare event whose magnitude is so high as to impact the average of a series. These are characteristic of 'fat-tailed' distributions, as opposed to thin-tailed distributions such as the normal distribution, in which rare events are too unlikely to have a large impact.

Blues, Greens: Roman Empire chariot-racing teams that became part of politics. Used in place of real party names. See Mind-killer.

C

: Coherent Extrapolated Volition, "In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted."

Clever arguer: Someone skilled at writing convincing-sounding arguments for an existing belief. Inventing clever arguments for a belief does not change the truth value of the belief. Discussed in "The Bottom Line"

: A moral theory that places value on the consequences of actions. Covered in more depth here.

: What to have when you may have been quite wrong for a long time.

D

: Rhetorical techniques crafted to exploit human cognitive biases. Considered bad behaviour even if the belief you want to communicate is good.

Deontology/ deontological ethics: An approach to ethics that judges the morality of an action based on the action's adherence to a rule or rules. See Wikipedia article on deontological ethics for more. Contrast .

E

EEA: Environment of Evolutionary Adaptedness. An term synonymous with the more commonly-used "ancestral environment". For humans, refers to the state of tribal bands of hunter-gatherers.

Egan's law: "It all adds up to normality." Surprising truths do not make the sky orange and grey; it stays blue.

ETA: Edited To Add (though some would rather you say "Edit:" instead)

EY: Eliezer Yudkowsky

F

FAI: Friendly AI

FOOM: Onomatopoetic vernacular for an .

: An argument which can be used to discount any conclusion the arguer does not like.

: The desired but less useful counterpart to utils. They make you feel you're altruistic and socially contributing.

H

: A unit philosophers use to quantify pleasure. (Note: no actual quantifying is done.)

: What Spock does, not what actual rationalists do.

I

IA: Intelligence augmentation

IAWYC: I Agree With Your Conclusion. Generally used when nitpicking, to make it clear that the nitpicks are not meant to represent actual disagreement. Discussed in Support That Sounds Like Dissent.

I don't know: Something that can't be entirely true if you can even formulate a question.

IMO/IMHO: In my (humble) opinion

: The number of inferences, or intermediate steps, it takes someone to get from their existing knowledge to an understanding of the point you're making. See also .

ISTM: It Seems To Me

K

Kolmogorov complexity: Given a string, the length of the shortest possible program that prints it. See also: Solomonoff induction

L

LCPW: . A technique used to prevent oneself from evading the point of a question by nitpicking details.

: A response to criticism which insulates the responder from having to address the criticism directly, without appearing to be conventional rudeness.

LW: Less Wrong

M

Meetup: Groups of Less Wrong members sometimes arrange to meet each other in meat space (in person). Some geographic areas have groups that do this regularly.

: A topic that reliably produces biased discussions, e.g. politics or Pick-Up Artists.

MoR: Also HPMoR,

: Reasoning used to reach desired conclusions rather than true conclusions.

MWI: Many-Worlds Interpretation, an interpretation of quantum mechanics advocated in Eliezer Yudkowsky's quantum mechanics sequence

N

NPC: Non-Player Character (think of an MMORPG like World of Warcraft that has characters controlled by humans and characters controlled by a computer; the characters controlled by humans would be PCs (player characters) and the characters controlled by a computer would be NPCs (non-player characters)).

Noncentral fallacy: A rhetorical move often used in political, philosophical, and cultural arguments. "X is in a category whose archetypal member gives us a certain emotional reaction. Therefore, we should apply that emotional reaction to X, even though it is not a central category member."

O

OB:

One-box: One of the choices for .

: A hypothetical superintelligent being, canonically found in .

Ontology/ontological: The philosophical study of the nature of being, existence, or reality, deals with questions concerning what entities exist or can be said to exist, and how such entities can be grouped, related within a hierarchy, and subdivided according to similarities and differences. See also ontological argument at Wikipedia for an example of (ab)using ontology to try and prove the existence of God.

P

: An AI that has been created to maximize the number of paperclips in the universe. A hypothetical .

: A group estimation game in which one player, unknown to the others, tries to subvert the group estimate.

Password: The answer you guess instead of actually understanding the problem. See Guessing the teacher's password

PC: Player Character (think of an MMORPG like World of Warcraft that has characters controlled by humans and characters controlled by a computer; the characters controlled by humans would be PCs (player characters) and the characters controlled by a computer would be NPCs (non-player characters))

PCT:

PD:

or P-Zombie: A creature which looks and behaves indistinguishably from a human down to the atomic level, but is not conscious. See

: What you update from in Bayesian calculations. In practical terms, everything you think you know now.

: The fallacy of singling out a specific hypothesis for investigation when there isn't enough evidence at hand to select this hypothesis over others. e.g."We have no idea who committed the murder, so let's consider the possibility that Mortimer Q. Snodgrass did it, and investigate him."

Or "The origin of the universe sure is mysterious! Have you considered that it could have been done by the God of the Bible?"

Q

QALY: Quality-adjusted life year; a concept from the economics of health care

R

Rationalism: The mode of thinking within the

: A technique for unpacking words into concepts: taboo the use of a given word or its synonyms. Particularly useful in arguments over definitions.

: "The world's greatest fool may say the Sun is shining, but that doesn't make it dark out."

: See also: Egan's Law
: A term that looks like an explanation but, on closer examination, doesn't actually explain anything. Also called curiosity stopper.

S

: How to do a utility calculation without scope insensitivity.

: Conveying information by performing an action which would be costly to perform, if the information were not true.

: A formalized version of Occam's razor based on Kolmogorov complexity.

: A term for the opposite of a Straw Man: the strongest possible form of an opponent's argument, even if they didn't make it themself.

Strong man: Another term for the opposite of a Straw Man: the strongest actual form of an opponents argument.

T

Taboo the word ... See .

TAI:

: Discussing an event as though it were caused by its future consequences.

tl;dr : Too long; didn't read. Polite use: one-line summary at top of your long article. Impolite use: dismissive response to another's long piece of writing or unparagraphed slab of text.

: When LessWrong was started, Eliezer put a temporary moratorium on discussion of the Singularity or AI. You will see this used in old discussions to allude to these topics.

Two-box: One of the choices for .

: Japanese: "I want to become stronger."

U

uFAI: Unfriendly AI

: A subject that is thought about less over time due to behavioral conditioning.

: A utility function assigns numerical values ("utilities") to outcomes, in such a way that outcomes with higher utilities are always preferred to outcomes with lower utilities.

Utils: Units of utility; sometimes called "utilons". Contrast .

Update: See

W

Weak Man: The opposite of a Strong Man, and relative to a Straw Man: the weakest version of your opponents actual arguments.

WBE:Whole Brain Emulation

Y

YMMV: Your Mileage May Vary. See

1
1
halo effect
Affective death spiral
Artificial general intelligence
Discussion0
Discussion0
Fuzzies
fuzzies
Consequentialism
consequentialism
Philosophical zombie
illusion of transparency
Utility function
Paranoid debating
Least convenient possible world
Rationalist taboo
rationalist taboo
Beisutsukai
Nassim Nicholas Taleb
Harry Potter and the Methods of Rationality
Hedon
Reversed stupidity is not intelligence
Topic that must not be named
Reality is normal
Paperclip maximizer
intelligence explosion
Tsuyoku naritai
Overcoming Bias
Logical rudeness
Other-optimizing
unfriendly artificial intelligence
Omega
Ugh field
Teleology
rationalist movement
Bayesian
Inferential distance
Evolutionary psychology
Signaling
Semantic stopsign
Mind-killer
Perceptual control theory
Priors
Transformative Artificial Intelligence
CEV
Solomonoff induction
Shut up and multiply
Crisis of faith
Zombies (sequence)
Dark arts
Fully general counterargument
Motivated cognition
Privileging the hypothesis
Hollywood rationality
Belief update
Belief update
Anti-epistemology
Steel man
Newcomb's problem
Newcomb's problem
Newcomb's problem
Prisoner's dilemma