Rss preview of Blog of LessWrong

My Theory of Change

2025-12-31 10:46:44

Published on December 31, 2025 2:11 AM GMT

2026 presents unfathomable opportunities for taking positive action to help the future go well. I see very few people oriented to what I see as many of the foundational basics.

1) With GPT-5.2-Codex-xhigh, Claude Code Opus 4.5, GPT-5.2-Pro, and the increasing DX tooling being created every hour, building wildly ambitious software in absurd time is exceptionally feasible. I am talking 1-6 devs building software platforms in weeks for agent-economies factorizing cognitive work at scale, with ultra-parsimous, easy-to-build-upon Rust codebases. A 2024-cracked-SWE-year's worth of work per day is basically possible already for thousands of people, if they can manage to be healthy enough and find flow.

2) Causing great outcomes basically has to be an additive, personal, bits-on-the-ground thing. You have to find yourself giving brilliant people influence over huge agent swarms, that can just deeply research, understand, and counter certain problem domains, like prion terrorism risk or whatever (because a significant fraction of people crash out sometimes). Real world collective superintelligence, this year, and you are a key part of it.

Trying to get real human builders to be all that oriented to downside risks, or jockeying for unprecedented government lockdown of compute--does not seem to be nearly as existentially practical as just building with good primitives and conscientiously stewarding emergence.

3) The real game is actually more somatics than anything. That doesn't mean full hippie kumbaya all the time, but you probably really do need to move a lot throughout the day, get Bryan Johnson-level sleep, distance from people you really contract around, get some hugs or self-hugs at least, and be gentle with your parts, like your socially-penalized instincts.

Backpropagating from things like millions of healthily-trained agents naturally coordinating cognitive work at scale with simple primitives--pairwise ratio comparisons eliciting cardinal latent scores for arbitrary attribute_prompts over arbitrary entities cached in agent-explorable SQL; multi-objective rerankers; market-pricing; frameworks that agents love for prior elicitation, reconciliation, and propagation--

it's about sitting in front of a laptop and prompting well, a lot. And to do that well, you've got to build up uncommonly good taste for what body signals are safe to ignore and what should really be respected. For me, estrogen (and some exogenous T) and a period of shrooms was absolutely necessary to downregulate, unclench, feel enough, and unlearn unhelpful reactivity patterns, to end up regulated enough to have a minimum viable amount of agency--I was simply too cripplingly heady and stuck before).

The world is a trippy, wildly complicated place as a self-modifying embodied organism, and I think that's to be embraced. The first person-perspective and unflattening of experience is a very real part of the game.

4) Overall, there's a resource flywheel game. Gotta find yourself with more and more dakka. I need more felt-safety, more nutrients, more stewarding of psychic gradients so good prompting and creativity keeps happening more, and I manage to bootstrap my limited $600/month AI compute + Hetzner server budget into $20,000/month into possibly $1M+ a day, by the time agents and harnesses get that good in 2026, to keep doing research and spawning side businesses that solve real problems in the ecology.

Discuss

Progress update: synthetic models of natural data

2025-12-31 09:31:00

Published on December 31, 2025 1:31 AM GMT

This post presents a brief progress update on the research I am doing as a part of the renormalization research group at Principles of Intelligence (PIBBSS). The code to generate synthetic datasets based on the percolation data model is available in this repository. It employs a newly developed algorithm to construct a dataset in a way that explicitly and iteratively reveals its innate hierarchical structure. Increasing the number of data points corresponds to representing the same dataset at a more fine-grained level of abstraction.

Introduction

Ambitious mechanistic interpretability requires understanding the structure that neural networks uncover from data. A quantitative theoretical model of natural data's organizing structure would be of great value for AI safety. In particular, it would allow researchers to build interpretability tools that decompose neural networks along their natural scales of abstraction, and to create principled synthetic datasets to validate and improve those tools.

A useful data structure model should reproduce natural data's empirical properties:

Sparse: relevant latent variables occur, and co-occur, rarely.
Hierarchical: these variables interact compositionally at many levels.
Low-dimensional: representations can be compressed because the space of valid samples is highly constrained.
Power-law-distributed: meaningful categories exist over many scales, with a long tail.

To this end, I'm investigating a data model based on high-dimensional percolation theory that describes statistically self-similar, sparse, and power-law-distributed data distributional structure. I originally developed this model to better understand neural scaling laws. In my current project, I'm creating concrete synthetic datasets based on the percolation model. Because these datasets have associated ground-truth latent features, I will explore the extent to which they can provide a testbed for developing improved interpretability tools. By applying the percolation model to interpretability, I also hope to test its predictive power, for example, by investigating whether similar failure modes (e.g. feature splitting) occur across synthetic and natural data distributions.

The motivation behind this research is to develop a simple, analytically tractable model of multiscale data structure that to the extent possible usefully predicts the structure of concepts learned by optimal AI systems. From the viewpoint of theoretical AI alignment, this research direction complements approaches that aim to develop a theory of concepts.

Percolation Theory

The branch of physics concerned with analyzing the properties of clusters of randomly occupied units on a lattice is called percolation theory (Stauffer & Aharony, 1994). In this framework, sites (or bonds) are occupied independently at random with probability $p$ , and connected sites form clusters. While direct numerical simulation of percolation on a high-dimensional lattice is intractable due to the curse of dimensionality, the high-dimensional problem is exactly solvable analytically. Clusters are vanishingly unlikely to have loops (in high dimensions, a random path doesn't self-intersect), and the problem can be approximated by modeling the lattice as an infinite tree^[1]. In particular, percolation clusters on a high-dimensional lattice (at or above the upper critical dimension $d \geq 6$ ) that are at or near criticality can be accurately modeled using the Bethe lattice, an infinite treelike graph in which each node has identical degree $z$ . For site or bond percolation on the Bethe lattice, the percolation threshold is $p_{c} = 1 / (z - 1)$ . Using the Bethe lattice as an approximate model of a hypercubic lattice of dimension $d$ gives $z = 2 d$ and $p_{c} = 1 / (2 d - 1)$ . A brief self-contained review based on standard references can be found in Brill (2025, App. A).

Algorithm

The repository implements an algorithm to simulate a data distribution modeled as a critical percolation cluster distribution on a large high-dimensional lattice, using an explicitly hierarchical approach. The algorithm consists of two stages. First, in the generation stage, a set of percolation clusters is generated iteratively. Each iteration represents a single "fine-graining" step in which a single data point (site) is decomposed into two related data points. The generation stage produces a set of undirected, treelike graphs representing the clusters, and a forest of binary latent features that denote each point's membership in a cluster or subcluster. Each point has an associated value that is a function of its latent subcluster membership features. Second, in the embedding stage, the graphs are embedded into a vector space following a branching random walk.

In the generation stage, each iteration follows one of two alternatives. With probability create_prob, a new cluster with one point is created. Otherwise, an existing point is selected at random and removed, becoming a latent feature. This parent is replaced by two new child points connected to each other by a new edge. Call these points a and b. The child points a and b are assigned values as a stochastic function of the parent's value. Each former neighbor of the parent is then connected to either a with probability split_prob, or to b with probability 1 - split_prob. The parameter values that yield the correct cluster structure can be shown to be create_prob = 1/3 and split_prob = 0.2096414. The derivations of these values and full details on the algorithm will be presented in a forthcoming publication.

Caveats

Because the data generation and embedding procedures are stochastic, any studies should be repeated using multiple datasets generated using different random seeds.
The embedding procedure relies on the statistical tendency of random vectors to be approximately orthogonal in high dimensions. An embedding dimension of O(100) or greater is recommended to avoid rare discrepancies between the nearest neighbors in the percolation graph structure and embedded data points.
A generated dataset represents a data distribution, i.e. the set of all possible data points that could theoretically be observed. To obtain a realistic analog of a machine learning dataset, only a tiny subset of a generated dataset should be used for training.

Next Steps

In the coming months, I hope to share more details on this work as I scale up the synthetic datasets, train neural networks on the data, and interpret those networks. The data model intrinsically defines scales of reconstruction quality corresponding to learning more clusters and interpolating them at higher resolution. Because of this, I'm particularly excited about the potential to develop interpretability metrics for these datasets that trade off the breadth and depth of recovered concepts in a principled way.

^{^}
Percolation on a tree can be thought of as the mean-field approximation for percolation on a lattice, neglecting the possibility of closed loops.

Discuss

Personalization Requires Data

2025-12-31 08:45:09

Published on December 31, 2025 12:45 AM GMT

In 2025, AI models learned to effectively search and process vast amounts of information to take actions. This has shown its colors the most in coding, eg through harnesses like claude code that have had a sizable impact on programmers’ workflows.

But this year of progress doesn’t seem to have had that much of an effect on the personalization of our interactions with models, ie whether models understand the user’s context, what they care about, and their intentions, in a way that allows them to answer better. Most chatbot users’ personalization is still limited to a system prompt. Memory features don’t seem that effective at actually learning things about the user.

But the reason machine learning has not succeeded at personalization is lack of data (as it often is!). We do not have any validated ways of grading ML methods (training or outer-loop) for personalization. Getting good eval signals for personalization is hard, because grading a model’s personalization is intrinsically subjective, and requires feedback at the level of the life and interactions that the model is being personalized to. There is no verified signal, and building a generic rubric seems hard. These facts do not mesh well with the current direction of machine learning, which is just now starting to go beyond verifiable rewards into rubrics, and is fueled by narratives of human replacement that make personalization not key (If I am building the recursively improving autonomous agi, why do I need to make it personalized).1

Start by giving models more information

But how do we obtain data for personalization? I think the first step to answering this question is having consumers of AI curate their personal data and then share it to enrich their interactions with AI systems.

Instead of just a system prompt, giving models a searchable artifact of our writing, notes, and reading history. Something agents can explore when your questions might benefit from context—and write to, to remember things for later.

whorl - My first guess

Over the break, I built a very simple software tool to do this. It’s called whorl, and you can install it here.

whorl is a local server that holds any text you give it—journal entries, website posts, documents, reading notes, etc…—and exposes an MCP that lets models search and query it. Point it at a folder or upload files.

I gave it my journals, website, and miscellaneous docs, and started using Claude Code with the whorl MCP. Its responses were much more personalized to my actual context and experiences.

Examples

First I asked it:

do a deep investigation of this personal knowledge base, and make a text representation of the user. this is a text that another model could be prompted with, and would lead them to interacting with the user in a way the user would enjoy more

It ran a bunch of bash and search calls, thought for a bit, and then made a detailed profile of me, and my guess is that its quality beats many low effort system prompts, linked here.

I’m an ML researcher, so I then asked it to recommend papers and explain the motivation for various recs. Many of these I’ve already read, but it has some interesting suggestions, quite above my usual experience with these kinds of prompts. See here.

These prompts are those where the effect of personalization is most clear, but this is also useful in general chat convos, allowing the model to query and search for details that might be relevant.

It can also use the MCP to modify and correct the artifact provided it, to optimize for later interactions – especially if you host a “user guide” there like the one I linked. Intentionally sharing personal data artifacts is the first step to having agents that understand you.

Conclusion

Personalization requires data. People need to invest into seeing what models can do with their current data, and figuring out what flows and kinds of interactions this data is useful for, towards building technology that can empower humans towards their goals. whorl is a simple tool that makes that first step easy. People who have already created a bunch of content should use that content to enhance their interactions with AIs.

Discuss