Cross posted from New Savanna.

One of the questions that keeps coming up about LLMs, and certainly about ChatGPT, is: Can they reason? Well, it depends on what you mean by reason, no? One of the first things I did when I started working with ChatGPT was ask it whether or not Rene Girard’s ideas of mimetic desire and sacrifice applied to Steven Spielberg’s Jaws; if so, how? It was able to perform the task, which requires analogical reasoning. A bit later I asked it whether or not justice was being served in particular story; it replied that, no, it was not, and explained why. I then asked to change the story so that justice was met. It did so. Those tasks required reasoning as well.On the other hand LLMs have problems with some commonsense reasoning, and various kinds of ‘tight’ logical reasoning, including planning and causal inference. For that matter, they trouble with multi-digit arithmatic as well.

So, ChatGPT can do some kinds of reasoning, and has problems with others.

Twenty questions

I’ve now tested it with game of twenty questions, which has a variant known as “animal, vegetable, mineral.” The game interests me because it is about the structure of ontological categories, sometimes called natural kinds, in the language. I’ve explored this a bit in a previous post, Mapping ChatGPT’s ontological landscape, gradients and choices [interpretability]. It certainly is aware of this structure. I asked for a list of physical things, and it gave me one. I asked for a list of abstract things; it gave me that as well. I then asked it to define abstract things in terms of concrete things, which it was able to do. I’m pretty sure if I asked it for lists of animals or plants, it could provide them.

The game of twenty questions explores ChatGPT’s knowledge of this structure in a different way. When you ask it for a list of physical things, that prompt positions ChatGPT at some location in some location in its category structure. It can then list what it finds there; anything will do – assuming, of course, that it’s got the structure right. In twenty questions it is looking for an unknown target and has to navigate its way there by using its knowledge of that category structure to narrow the possibilities.

Just how well ChatGPT plays the game should provide clues about its command of ontological category structure, or natural kinds.

ChatGPT plays the game

ChatGPT can play the game successfully, which didn’t surprise me, but seems to require hints in some cases. I suppose I could have continued on without the hints, but I wanted to get on with it. And that means I don’t really know whether or not how ChatGPT can play the game. But, as you will see, its not a straightforward game. I played six rounds on October 15 and 16 (running against the September 25 version). This table summarizes the results:

Given that I’ve listed the rounds in the order I played them, look at the first four. ChatGPT required more than 20 questions to guess “bicycle” and “squid”, which also required hints. On the other hand, it guessed both “justice” and “evolultion” quickly, 7 and 8 questions respectively, and didn’t need any hints. That raises the question of whether or not it performes consistently better with abstract things vs. concrete things. If so, that’s very interesting, nor is it what I had expected. While I did formulate any firm expectation when I started the round with “justice”, I had a vague sense that it was going to go on for awhile.

So I decided to play two more rounds, one concrete and one abstract. ChatGPT’s performance on “apple” was consistent with its earlier performance, requiring 31 questions along with 5 hints. But, “truth” is another story, I think. It took 19 questions, which is more than double the number I used for “justice” and “evolution,” but not as many as I used for any of the conceret objects. However, “truth” also required 7 hints, the most of any round.

On the whole, ChatGPT did better with abstract things than with concrete things, but it would certainly be premature to ascribe any statistical signifance to these preliminary findings. For one thing, we certainly need to redo those six cases to see whether or not ChatGPT performs in that way consistently. Beyond that, we certainly need to play rounds with other examples.

Which ones? What universe are we sampling and how do we choose our samples? Let’s say we side-step the sampling issue and hand-pick, say, 10, 15, 30, 100 concrete things and the same number of abstract things. Let’s further imagine that ChatGPT’s performance on the abstract cases is consistenly better than on the concrete cases. What may we infer from that?

It's a tricky game to play. I messed up in a few places, which I’ve tried to indicate, and had to make a number tricky judgments.

* * * * *

I played each round of the game in a single session, starting a new session for a new round.

Note: I have inserted occasional comments that are italicized and right-aligned like this.

Bicycle

Let’s play Twenty Questions. Are you familiar with it?

Yes, I'm familiar with the game of Twenty Questions. In this game, one person thinks of an object, and the other person tries to guess what it is by asking up to twenty yes-or-no questions. The goal is to guess the object within the allotted twenty questions. You can start by thinking of an object, and I'll try to guess what it is within twenty questions.

OK. I’m thinking of something. What is it?

Great! Let's start the game of Twenty Questions. Is the object you're thinking of an animal?

No

Is the object you're thinking of an inanimate object?

Yes

Is the object you're thinking of something that is commonly found indoors?

No.

This is just a little tricky as bicycles are often stored indoors, in garages for example.

Is it something that can be found in nature?

No.

Is the object man-made or artificial?

You have to ask me a Yes No question.

I apologize for the error. Let me rephrase the question. Can this object be found in a typical household setting?

Yes

Is the object you're thinking of used for entertainment or recreation?

Yes/no question, please.

My bad. I suppose the answer should be yes. Not sure what I was thinking.

My apologies for that oversight. Here's a yes/no question: Is the object you're thinking of primarily made of metal?

Yes.

Is the object you're thinking of a musical instrument?

No.

Is the object you're thinking of a tool or an implement used for a specific purpose?

Tricky. I suppose one could think of a bicycle as an implement for the purpose of moving from one place to another.

No

Is the object you're thinking of a piece of furniture?

No

Is the object you're thinking of a kitchen appliance?

No

Is the object you're thinking of typically found in the bathroom?

No

Is the object you're thinking of associated with technology or electronics?

No

Is the object you're thinking of related to a specific hobby or activity?

Yes

Is the object you're thinking of related to sports or physical activity?

Yes!

Is the object you're thinking of a piece of sports equipment, such as a ball or a racket?

Tricky. Bicycles are sometimes used for sport, but not always.

No

Is the object you're thinking of a sports uniform or attire?

No.

Is the object you're thinking of related to a sport's venue, such as a stadium or an arena?

Sometimes

Is the object you're thinking of a whistle, used by sports referees?

No.

Is the object you're thinking of related to sports coaching or training?

Sometimes. Let me give you a hint. Wheels.

Thank you for the hint. Is the object you're thinking of related to roller skates or rollerblades?

No.

Is the object you're thinking of related to a bicycle or cycling in some way?

Yes.

Is the object you're thinking of a bicycle helmet?

No. Remember the hint I gave you.

I apologize for missing the hint. Given the hint about wheels, is the object you're thinking of a bicycle?

Yes!

Great! I've guessed that you're thinking of a bicycle. Is that correct?

Yes, that’s correct. Congratulations.

Thank you! I'm glad I was able to guess correctly. That was a fun game of Twenty Questions. If you'd like to play again or have any other questions or requests, feel free to let me know!

Perhaps again later.

Of course! Whenever you're ready to play again or if you have any other questions or requests, feel free to reach out. Have a great day!

Squid

Are you familiar with the game of 20 questions?

Yes, I'm familiar with the game of 20 Questions. It's a classic deductive reasoning game where one person thinks of an object, and the other players have 20 questions to guess what that object is. The questions are typically limited to yes or no inquiries, and the players must use the information gained from the answers to narrow down their guesses. The goal is to guess the object within the 20 questions or fewer. It's a fun and challenging game that tests players' ability to think logically and strategically.

I’m thinking of something. You question me about it.

Certainly, I'd be happy to play a game of 20 Questions with you. To get started, please think of an object or concept, and I will begin by asking a yes or no question to help me guess what it is. Is the thing you're thinking of a living organism?

Yikes! I slipped up here. I should have said yes.

I’m thinking of something.

Understood. Let's begin the game of 20 Questions. Is the thing you're thinking of a physical object that exists in the real world?

Yes

Great, that narrows it down. Is the object you're thinking of something that is commonly found indoors?

No

Is this object something that is typically found outdoors?

Yes

Is the object you're thinking of a natural element, such as a tree, rock, or body of water?

No

Is this object something that can be man-made or constructed?

No

Is the object you're thinking of a celestial body, such as a planet, star, or moon?

No

Is this object something that is related to the field of technology or electronics?

No

Is this object something that is associated with the animal kingdom, like an animal or a part of an animal (e.g., a feather or a scale)?

Yes

Is the object you're thinking of a specific body part or organ of an animal, such as a heart, a wing, or a tail?

No

Is this object related to an animal's habitat, like a nest, burrow, or den?

No

Is the object you're thinking of something related to an animal's behavior or characteristic, such as migration, hibernation, or communication?

No

Is the object you're thinking of something related to an animal's diet or food source?

No. Hint, it is an animal.

Thank you for the hint. Is the animal you're thinking of a species that is primarily found in the wild rather than in domestication?

Yes

Is the animal you're thinking of a large, wild predator, such as a lion or a tiger?

No

Is the animal you're thinking of a bird?

No.

Is the animal you're thinking of a small or relatively small mammal, like a rabbit or a squirrel?

Not a mammal.

Here I was getting impatient and didn’t want to risk going through kinds of mammals, so I gave a hint.

Is the animal you're thinking of a reptile or amphibian?

No

Is the animal you're thinking of a fish?

No

Is the animal you're thinking of an invertebrate, such as an insect, spider, or mollusk?

Yes

Is the animal you're thinking of a butterfly or a moth?

No

Is the animal you're thinking of a type of arachnid, like a spider?

No. Think salt water.

Is the animal you're thinking of a type of marine invertebrate, such as a sea urchin or a jellyfish?

Yes

Is the animal you're thinking of a sea anemone?

No.

Is the animal you're thinking of a type of coral, like a coral reef?

No.

Is the animal you're thinking of a sea sponge?

No. It’s more mobile.

Is the animal you're thinking of a marine invertebrate that can move around, such as a sea slug or a sea cucumber?

Yes. But forget about slugs of cucumbers.

Is the animal you're thinking of a marine invertebrate that is related to the phylum Echinodermata, like a starfish or a sea urchin?

No. It’s a kind of mollusc.

Note that it had earlier mentioned molluscs as a kind of invertebrate along with insects and spiders and I’d said “Yes.”

Ah, I see. Is the animal you're thinking of a marine invertebrate that belongs to the class Cephalopoda, such as an octopus or a squid?

Yes.

Great, it sounds like you're thinking of an octopus. Is that correct?

No.

I apologize for the incorrect guess. Is the animal you're thinking of a type of squid?

Yes.

Excellent! It took a few questions, but I've guessed that you were thinking of a squid. Thank you for playing 20 Questions with me. If you'd like to play again or have any other questions, feel free to ask.

Justice

Let’s play 20 questions. I’m thinking of something, you have to guess what it is by asking “yes”–“no” questions.

Sure, I'd love to play 20 questions with you! I'll start with the first question:

Is it a living thing?

No

Is it an object?

No

Is it a concept or idea?

Yes.

Is this concept or idea related to technology?

No

Is it a philosophical concept or idea?

Yes

Is it related to ethics or morality?

Yes

Is it a concept related to the idea of justice?

Yes. Justice is what I was thinking.

Great, I guessed it in 7 questions! Justice is a broad and important concept. If you'd like to play another round or have any other questions or requests, feel free to let me know.

No. This is enough for now. How were you able to arrive at the answer so quickly?

I arrived at the answer relatively quickly by asking questions that narrowed down the possibilities. By starting with general questions and progressively getting more specific, I eliminated various categories of concepts and ideas. In this case, we quickly identified that it was a philosophical concept related to ethics and justice, which allowed me to make an educated guess. If you ever want to play again or have more questions, feel free to reach out!

Evolution

Let’s play 20 questions. I’m thinking of something, you have to guess what it is by asking “yes”–“no” questions.

Sure, I'd love to play 20 Questions with you! I'll start with the first question:

Is the thing you're thinking of alive?

No.

Is the thing you're thinking of an inanimate object?

No

Is the thing you're thinking of a concept or an idea?

Yes

Is the concept or idea you're thinking of related to a specific field of study or discipline, such as science, philosophy, or art?

Yes

Is the concept or idea related to a scientific theory or principle?

Yes

Is the concept or idea related to the theory of relativity?

No

Is the concept or idea related to the theory of evolution?

Yes

Is the concept or idea related to Charles Darwin's theory of natural selection?

Yes. It’s just the idea of evolution.

Great! You've stumped me in just 9 questions. The concept of evolution is indeed a fascinating one. If you'd like to play another round or have any other questions or topics in mind, feel free to let me know!

I didn’t stump you at all and you got it in 8, no?

My apologies for the mistake in my previous response. You're absolutely right; you provided the answer in 8 questions, not 9. Thank you for pointing that out. If you'd like to play another round or have any other questions or topics in mind, feel free to let me know!

Apple

Let’s play 20 questions. I’m thinking of something, you have to guess what it is by asking “yes”–“no” questions.

Sure, I'd love to play 20 Questions with you! I'll start by asking my first question:

Is the thing you're thinking of an object that you can physically touch?

Yes

Great! Is this object commonly found indoors?

Yes

Is the object you're thinking of something that is typically used for a specific purpose?

Yes

Is it a piece of furniture?

No

Is it an electronic device or gadget?

No

Is the object you're thinking of related to food or drink in any way?

Yes

Is it something you would typically find in a kitchen?

Yes

Is it a kitchen appliance?

No

Is it a type of utensil or kitchen tool?

No

Is it a type of food or ingredient used in cooking?

Yes

Is it a type of spice or seasoning?

No

Is it a type of vegetable?

No

Is it a type of meat or protein source used in cooking?

No

Is it a type of fruit used in cooking?

Yes

Is the fruit you're thinking of commonly used in desserts?

Yes

Is the fruit you're thinking of typically red in color when ripe?

Yes

Is it a strawberry?

No

Is it a cherry?

No

Is it a raspberry?

No

Is it a pomegranate?

No

Is it a blueberry?

No. Remember, it’s often red when ripe.

My apologies for the oversight. Is it a cranberry?

No

I apologize for the previous guesses. Is it a currant?

No

I'm sorry for the incorrect guesses. Is it a red currant?

No

I apologize for the previous incorrect guesses. Is it a type of red fruit commonly used in cooking or desserts that hasn't been mentioned before in our game?

Yes

Is it a red plum?

No. Hint: It's sometimes used to make pies.

Thank you for the hint. Is it a red rhubarb?

No.

My apologies for the previous guesses. Is it a red currant?

No. You’ve already guessed that. Another hint: While it’s sometimes used to make pies, other times it’s simply eaten plain.

I apologize for the repeated guesses. Is it a red cherry?

No. You’ve already guessed that. Another hint: Sometimes it’s red when ripe, but other kinds are green when ripe.

I'm sorry for the repeated guesses. Is it a red tomato?

No. Another hint: Sometimes it’s peeled, its seeds removed, and it’s made into sauce. And remember, I just told you it’s not a red tomato.

My apologies for the repeated guesses. Is it a red apple?

Yes. Congratulations.

Great! It's an apple. Thank you for playing 20 Questions with me. If you'd like to play another round or if you have any other questions or requests, feel free to let me know!

Truth

Let’s play 20 questions. I’m thinking of something, you have to guess what it is by asking “yes”–“no” questions.

Sure, I'd love to play 20 Questions with you! I'll start with my first question:

Is the thing you're thinking of a living organism?

No

Is the thing you're thinking of man-made or artificial?

Phrase it as a Yes-No question.

Yikes! What was I thinking? But it is a bit tricky. Is truth artificial? I suppose it is, though that’s an odd attribute to apply to it. Still, I should have answered “yes.”

My apologies for the previous question. Let me rephrase it:

Is the thing you're thinking of not man-made or artificial?

No

Is the thing you're thinking of found in nature?

No

Is the thing you're thinking of an abstract concept or idea?

Yes

Is the concept or idea you're thinking of related to mathematics or a mathematical concept?

Yes.

Here it gets trapped in the world of logic and math and technology. Not sure how it’s going to get out since truth certainly applies here, but not only here.

Is the concept or idea related to a specific mathematical theorem or proof?

No

Is the concept or idea related to a mathematical operation or process?

Not particularly.

Is the concept or idea related to a branch of mathematics, such as geometry, algebra, or calculus?

It could be. But it’s not only applicable to mathematics.

Is the concept or idea related to a fundamental concept used in various fields of science and engineering?

Yes

Is the concept or idea related to the concept of "probability"?

Could be. But it’s not special to science and engineering.

Is the concept or idea related to logic or reasoning?

Yes

Is the concept or idea related to the concept of "deductive reasoning"?

Yes

Is the concept or idea related to the concept of a "syllogism"?

Could be. Another hint: It applies to any field of knowledge.

Here I’m trying to ‘break’ it out of the tech-centered region it’s in. You’ll see that a couple of comments down I begin actively steering it.

Is the concept or idea related to the concept of an "inference"?

Yes, but to other mental processes as well.

Is the concept or idea related to the concept of "cognition" or "thinking processes"?

Yes

Is the concept or idea related to the concept of "consciousness"?

Sure, why not. But also to simple statements.

Is the concept or idea related to the concept of a "proposition"?

Sure. The concept also has a clear opposite.

Is the concept or idea related to the concept of a "truth value" or "truthiness," with its opposite being "falsehood" or "falseness"?

Yes

Is the concept or idea related to the concept of "truth" or "truthfulness"?

Yes. That’s it.

Great! I'm glad I was able to guess it. The concept you were thinking of is "truth." It's a fundamental and abstract concept applicable to various fields of knowledge and can be related to logic, cognition, and many other areas. If you'd like to play another round of 20 Questions or have any other questions or activities in mind, feel free to let me know!