grantmaking.ai Launch Round
Condensation proves that there's a canonical mapping between 2 sets of latents with good condensation scores. With logical uncertainty, we can prove that for "old" and "new" concepts, addressing ontology identification problem
How do you map your old concepts to new concepts when your model of the world gets updated? This is the ontology identification problem. Even if we make the first AGI "aligned", if it gets smarter and improves it's world model, there's arbitrarity in how the concepts it cares about now could move to new concepts it would care about, and if done incorrectly, this could lead the future into weird and undesirable states. Could there be a systematic way to do this correctly?
I plan to do this by extending "Condensation - a Theory of Concepts" by Sam Eisenstat. It takes observables, takes latents (each latent contains information needed for subset of observables), and scores how well those latents "condense" information for observables. Redundancy is punished. This forces latents to represent "natural concepts". It uses Shannon entropy in its score. It also proves that if the condensation score is good, those latents are, in some sense, objective: there is a canonical mapping between two sets of latents with good condensation score (called "objectivity theorem").
But it only cares about epistemic uncertainty, and not about logical uncertainty (which is necessary for bounded cognition). I plan to introduce Deterministic Entropy (which is K-complexity with time constraint) into the equation, which would allow for constrained deliberation. This way, we introduce time dimension, and Condensation could think for a little or it could think for longer, arriving at concepts of varying quality. If I prove a version of objectivity theorem, but between "old" and "new" latents, this would be progress towards solving ontology identification problem.
I am accepted to the Affine Superintelligence Alignment residency in September, and I need to cover some travel expenses (~2000$).
In general, I am a very social person, and if I'm working with someone, this almost doesn't feel like work, and if I work alone, I often can't even do it. The way I do research now is by hiring an assistant, and even though it's slightly out of my budget (his current rate is 100$/week), it's worth it because this way I have some company. Having more budget would make paying assistants easier.
Because of my need in social circle, being closer to researchers is very important for me, and money would allow me to stay closer to the alignment research community. Also, money would cover living costs for a few month so I could spend my time on research (~8000$ for 5 month).
I have known Roman for almost six years, since our first undergraduate year. While AI safety is not my focus area, my conversations with Roman convinced me that everyone, including myself, should pay attention to this problem.
Roman has consistently demonstrated his devotion to the AI alignment field. For a long time, his dream was to discover a new type of fundamental particle, and he was actively moving towards his goal by beginning research in this area from his early undergraduate years. However, as rapid advances in LLMs indicated risks associated with uncontrolled AGI development, he became deeply committed to the idea of creating safe AI. Influenced by Roman's arguments, I personally was alarmed by the fact that we cannot even notice the moment the AI singularity happens until it's too late.
Another example of Roman's dedication is his resilience and initiative under uncertainty. It's apparent that being in Russia now, researchers are almost isolated from the international research community. However, Roman's ambitions enabled him to attend the international bootcamp last year, which further accelerated his motivation to conduct research towards creating safe AI.
I can confirm that for Roman, research in AI safety is more than just curiosity-driven. For him, it's a mission, shaping his life and values. All this, combined with Roman's outstanding capabilities in STEM developed during his school, undergraduate, and graduate years, makes me confident that Roman must be supported. This support will help to kick-start his career in alignment research. Having collaborated with Roman on various projects during our studies and knowing him well in daily life, I'm convinced, both as a friend and a peer, that Roman is someone you can rely on.
If you have any questions, please feel free to reach out to me.
E-mail: i.d.solomakhin@gmail.com
LinkedIn: https://www.linkedin.com/in/ivan-solomakhin/