Abstraction Research

Researching how biological learners create their own abstractions, and how artificial learners can also.

This work exists to investigate the following observation:

There is a very specific sense in which current LLMs are not AGI. If you define “general” as “capable of representing a general assortment of inputs”, then they are general. But if you define “general” as “capable of learning a general assortment of data-limited domains”, then they are not general, because they cannot endogenously explore and learn complex domains.

Supervised learning learns robustly from external data, but can't mint its own knowledge. RL learns everything for itself, but learns with specific regard to an external verifier, and thus can't navigate great complexity. The process of creating new abstractions requires a synthesis of both.

Other forms this problem statement may take:

• Sample efficiency ("how do you learn more from limited data?")
• Continual learning ("how can the things you know today influence the way in which you learn new things tomorrow?")
• Interpretability ("how do you judge the quality of your own thoughts before you have supervision on them?")
• Taste/style/originality ("how do you create abstractions which are both endogenous and coherent?")

Ideas which are related but subtly different:

• Meta-learning (learning over the class of abstractions rather than the class of data is the right idea, but traditional meta-learning still creates abstractions passively with respect to the data it sees)

The research happens in the open. The lab notebook, with the experiments and the idea docs used to reason about their results, is public at github.com/jagilley/abstraction and updated weekly.

FAQs

Why is abstraction creation an important problem?

Consider several known open questions in the field of AI. My view is that each of these phenomena is an instance of what it looks like to form model abstractions passively, based on the data and verifiers the model sees during the training process.

• Q: why do the models seem to generalize much worse than humans?
  • A: under the current model training paradigm, generalization, for example, is a property of the particular distribution of data a model sees during training. If the model only forms abstractions with respect to the data it sees during training, it must rely upon transfer learning over the geometry of the data to generalize to inputs it has never seen.
• Q: why can the models easily achieve superhuman capabilities on tasks like competitive programming but not those like open-ended research?
  • A: because competitive programming requires skillful deployment of existing abstractions rather than the invention of new ones
• Q: why does finetuning the models on domain-specific data not produce gains outside that domain, when for humans it is possible to learn things that generalize from seemingly specific domains?
  • A: because models adapt during fine-tuning by treating the fine-tuning domain as the new ground truth against which abstractions must earn their worth
• Q: why are superhuman Go engines trained by self-play vulnerable to adversarial policies?
  • A: because self-play only creates abstractions passively, in regions it has visited
• Q: why do models collapse when recursively trained on their own outputs?
  • A: because they have no need to form new abstractions passively to represent data already generated by their policy

Why do you think frontier research can be done on non-LLMs?

I think there are two questions bundled together here:

1. Can non-LLM research currently be frontier-level useful?
2. Do LLMs work the same way as other types of AI models, in a learning-theory sense?

To the first question, I would agree with the scaling-maximalist consensus that the answer is "no", and I find the fact that the answer is "no" to be thought-provoking. To my mind, this fact would seem to imply something about the data on which we train LLMs: what is it about human-generated textual data that makes it uniquely useful? I suspect the answer has something to do with the directness with which our language encodes the abstraction byproducts of our cognitive processes.

To the second question, I would say "yes", and I don't think many would disagree. Whatever it is that makes training a transformer on exabytes of pure language more useful than training a transformer on exabytes of pure images seems like it ought to be downstream of the language data itself, and the model representations required to be a good next token predictor over it.

Biological learners seem to be capable of building abstractions over non-linguistic data, and the language data which makes LLMs so powerful comes from biological learners. So it seems like the default position ought to be that non-language data can be fertile ground for frontier research, until proven otherwise.

What's the research strategy?

This work is generally guided by investigating how the human/mammalian brain works, and building models that functionally approximate its biology and how it handles various realistic tasks.

What's the relationship between abstractions and superintelligence?

I define "superintelligence" as capabilities which are nontrivially in excess of those of current AI models. My belief is that abstraction creation is an acute deficit in current model capabilities. Therefore, "superintelligence" is one word you could use to refer to this research program.

That being said, it's pretty robustly established that biological learners - even some which are surprisingly phylogenetically distant to humans - do not have this same deficit. So I would argue that intelligence (super- or otherwise) is likely better thought of as a vector than a scalar.

Papers