Hello everyone, I'm a long-time lurker here, but this is my first time commenting. I'm a researcher at the National Research Council of Canada, and a big part of my research has been about taking ACT-R and figuring out how it could be implemented in the brain: http://terrystewart.ca/
I very much agree with the summary in the main post here. ACT-R is the best current model if you are trying to match human experimental data, including things like accuracy and reaction times. And it's been applied to a much wider variety of tasks than any other competing theory. It's definitely missing lots and lots of things, but it also seems to be getting a lot of things right. In particular, I'm very convinced by the constrained communication between brain modules (the "buffers" in the diagram posted by JenniferRM, each of which is restricted to contain around 7 key-value pairs at any point in time -- the evidence for this tends to come from tasks where in some situations it is very easy to do two things at once and in other situations it is very hard), and the declarative memory system (a beautiful mathematical model where the activation of a memory decays as ∑t−0.5i where ti is how long it has been since you've used that memory).
For those looking for an introduction to it, I'd recommend "ACT-R: A cognitive architecture for modeling cognition" https://wires.onlinelibrary.wiley.com/doi/abs/10.1002/wcs.1488 (which was also linked in the post).
I'll also agree with JenniferRM's idea that ACT-R is essentially a programming language, but I'll also add that one of the main ideas is that it's a programming language with a lot of weird constraints, but the hope is that the constraints are such that if you build a program to do a task within those constraints, then there's a reasonable chance that the resulting program will be a good model of how humans do that task, including things like making similar mistakes as people do, and taking similar amounts of time. It doesn't always live up to that, but that's the goal.
There are many theories of cognition. But if you want to work within a framework with the following properties:
Well, then, I'm not aware of any theories which fit the bill as well as ACT-R theory.
You might also be interested in the common model of cognition (initially named standard model of the mind), which is consistent with the ACT-R picture, but also consistent with several competing theories. Think of it as the high-certainty subset of ACT-R.
References for ACT-R
I am no expert on ACT-R, so unfortunatery I can't tell you the best place to get started! However, here are some references.
Books by John R Anderson
John R Anderson is the primary researcher behind ACT-R theory. I have not read all of the following.
John R Anderson also has several other books, which I haven't looked at very much; so maybe a different one is actually a better starting place.
Other References
Here are some other potentially useful starting places.
References for the Common Model of Cognition
If you search "common model of cognition" in Google Scholar, you will find a number of other papers discussing it.
How should we evaluate all this?
I don't personally think about ACT-R very much, but that's because my thinking on AI alignment has little to do with neuroanatomy-inspired AI. Some other people around here think a lot more about those things. ACT-R theory might be useful to those people? Also, if you care about the nitty gritty of human modeling, EG for the sake of inverse reinforcement learning or other value-learning purposes, ACT-R might be useful. It is, after all, a really sophisticated model of a human.
Personally, I am hoping that learning more about ACT-R theory could help me think about human (ir)rationality in more detail.