Project summary

I will build on Vanessa Kosoy’s recent work on frugal compositional languages and, more broadly, her learning-theoretic agenda. The central question of the project is whether we can find a more informative measure of hypothesis simplicity in inductive reasoning than Kolmogorov complexity, and model the process of inductive reasoning around it. While Kolmogorov complexity is guaranteed to be asymptotically close to many reasonable simplicity measures, it is not computable, does not provide a proper local ordering of hypotheses, and has other problems. Finding a better simplicity measure would be a step towards modelling how AI agents learn more precisely, thereby enabling more precise inferences about their goals and behaviour.

What are this project's goals? How will you achieve them?

The main goal is to advance our understanding of how intelligent agents behave, and how they perform inductive reasoning about their environment in particular. I will achieve this through theoretical and mathematical analysis.

How will this funding be used?

Personal expenses to allow me to work on the project full-time.

Who is on your team? What's your track record on similar projects?

I am the only person on my team right now, but other people may join if I see ways to parallelize the work. I have studied AI safety and alignment, Martin-Löf Type Theory, foundations of mathematics, proof checkers, deep learning and philosophy. I spent a lot of time thinking about formal epistemology, how data and computation can be organized, programmed for many years, always with a focus on "how do I make this work in a conceptually best way" which rhymes a lot with both conceptual pre-paradigmatic research in general, and the question of "how do I (in general, as an AI) systematically find the best abstraction for a certain task". I have always been a big fan of generalizing things and finding the rules behind processes.

What are the most likely causes and outcomes if this project fails?

The most likely cause of failure I envision is that I simply won't find the right abstractions for complexity and inductive reasoning soon enough, or fail to generalize them sufficiently. This will still likely produce important intuitions about the unpredicted complexities of the task, which will enable next steps in the research.

How much money have you raised in the last 12 months, and from where?

None