2026-08-01 08:00:00
We need more examples of prompting agents into completing big tasks. I used to think it was easy, you just type like you’re asking a super smart friend. But people keep looking confused, as if it’s hard. So, let’s just work through some examples here.
First up, I had some issues with my 3D printer (uh, I fixed it wrong a year ago and now it’s haunting me). I tried using ChatGPT a couple weekends in a row, and finally decided to revert to full Codex.
Here is a prompt I used. It’s heavily annotated, I recommend clicking through everything.
hey, I have this 3d printer. I’m trying to fix it. My brother gave me this printer, it looked new out-of-the-box, but felt like it shipped straight from china. Like all the instructions were in chinese. He does a ton of business in china, so it seems likely that he did indeed buy it from a chinese supplier and it shipped straight from china
ok, so the issue I’m trying to solve is the hotend is clogged. I’ve gone back and forth with chatgpt a while on it. I’ve consolidated a lot of the chat here: chatgpt-transfer.md
iPhone HEIC pics here in the root are just the 3d printer from various angles.
ChatGPT suggested buying a new print head. I looked online and it was like $120, which is 50% of the cost of a whole new printer (the new kind that doesn’t have these problems). I don’t want to spend $120, let’s keep it under $50. If you need me to buy new parts, send me links to reputable buyers. I’ve added more HEIC pics also of a hotend I bought already.
Now, when you write, I understand that it’s very tempting to use shorthand. Please don’t. Please talk to me like I’m a dumbass bussiness person who just tried plugging the printer power chord into a wifi router. Use visual aids. Make it fun and easy to follow. Raccoons and goblins? Sure. Whatever.
Also: this week openai reduced pricing of Luna by 80% and Terra by 20%. That means both of these (quite capable models imo) are extremely cheap. I have you (Sol) on max effort because I’m confident your intelligence will be able to wield these resources appropriately. This is a chatGPT sub though, so we only have a single 5hr block to compute our way out of this. Use it wisely.
If you have questions, you are welcome to ask me. You’re also welcome to install tools, just let me know if you’re modifying my computer.
Click here to open it’s first take on the web app (it’s really cool!). It made great use of Three.js.
In my head, it mostly feels like I’m just chatting with a super smart friend. But it probably seems complicated, because I’ve gotten to know that friend and maybe you haven’t.
I usually go goofier. I figure I’m talking to some intelligent being and whatever I say at the beginning sets the tone for the rest of the conversation. Like you would if you were talking to a friend or a coworker. After all, they were trained to imitate people, so obviously they’d act like people in this sense.
It’s a good idea to open with some rough idea of what you’re asking them for. Advice? Design? Problem solving? Again, it’s like talking to a person. Everything they hear is going to be filtered through that lens.
Now technically, an LLM sees all of your words simultaneously, so it works differently than people. For us, we hear/read the words one-after-the-other, in slow sequence. I guess that means we don’t have to start off like this, but old habits die hard. A better reason to do it is I sometimes get lost in my thoughts while writing the prompt and forget to ask a basic question like, “how to I fix it?”. So starting off with this is more just for me, I guess.
Agents eat context. The problem with being an intelligent entity but not actually having a body, is AIs miss out on a huge amount of information. So here I’m just trying to lay the foundation for the possibility that my 3D printer isn’t actually a normal stock OOTB device.
I also didn’t want to tell it that it wasn’t normal. If I did that, it might get stuck in that line of reasoning. By phrasing it this way, I’m giving the agent permission to doubt, without instructing it to do so.
The real issue, explained better. It’s important to clearly state the problem. I did an unclear version right at the start. Really I only need this version.
I don’t edit my prompts at all. I just type pretty much stream-of-consciousness, so yeah, duplication happens.
Context is super important. Agents eat context, the more you can provide, the more they can do.
I already went on with ChatGPT for two rounds already, and I didn’t want to make the same mistakes. So I opened up the old conversations and had it summarize (1) just the facts that I provided, and then (2) the inferences it had made. I kept them separate. I had ChatGPT write it up because I’m lazy and didn’t want to type it again. I told ChatGPT that I was handing off to Codex, so that put it’s mindset into a knowledge transfer mode, to get what I was looking for.
Since I had already gone through the trouble to type it up, this seemed like the most expedient route to reusing that context.
I took a pictures from my phone at several angles. Basically every angle I could think of, then moved them over to my laptop. On my laptop I had created a directory to dump these files, a work directory. I didn’t do any curation, the LLM can do a fine job of that. It just read all of them anyway.
This particular problem is extremely visual. Without pictures, the agent knows nothing except for what I say. But, I’m actually not that knowledgeable in 3D printers, so every time the AI tries to do some experiment, I get highly annoyed. By including a bunch of pics, I bypass a whole lot of problems.
Later on, we take even more pics, and those prove to be the linchpin evidence needed for a breakthrough.
Context is king. Rather than simply telling it how much to spend, I’m intentionally spilling my internal mental state in regards to spending money. Yes, I’m going to follow up with a constraint, but this is what I’m really thinking.
If I simply state the constraint, then I’m worried that it’ll get distracted and try to spend money. Whereas I really just want to offer it the possibility of buying new parts.
But also, I’m opening up the possibility of giving up and just buying a new printer. Whatever happens, it now has the context to reason about what the most reasonable next step is.
State the constraints clearly.
Agents eat context, of course, but they also excel at staying within absurd constraints. They love constraints. In fact, if you have crazy constraints, you absolutely should be using AI to work though it.
More constraints is better, but only if they’re hard constraints. If they’re flexible, mention where they can flex.
Establish how I want to be communicated with.
For this, it’s highly visual. When I’m looking at the device I often can’t figure out what I’m looking at.
Three.js — this is something I’ve recently discovered. It’s a game engine, I think, where you can build 3D environments in the browser. I bet you’d have to be sadistic to sling Three.js code manually, but LLMs can do it all day without issue. Code is how they interact with the world. Fwiw I love what it came up with.
Throughout this, I often intentionally try to be vague when I don’t absolutely need something specific. LLMs are quite creative and talented, and quite often the thing they come up with is better than whatever I was thinking.
Ever since Opus 4.7 I’ve noticed that all LLMs are deviating hard into a dense shorthand that’s hard to follow. They’ll invent terms and I’ll think I’m just an idiot for not being able to follow. Turns out the language is constantly changing.
Please talk to me like I’m a dumbass bussiness person who just tried plugging the printer power chord into a wifi router.
This line makes me laugh. But also it feels like something someone would have said to me when I was young, smart, and had no idea how to communicate with superiors.
Okay, long story. Basically when GPT-5.5 was released, a few days after people realized there was some language in the codex system prompt about racoons, goblins and magical creatures. OpenAI wrote a blog post about the incident. It was something to do with RL reinforcing behaviors.
But I like the goblins & raccoons. I periodically try to coax GPT into being it’s true self, because it’s fun. But also, I tend to think an AI will do better work if it’s not being constantly hand-slapped, like a human (they’re imitating them after all)
Rather than just saying “use Luna”, I give some details for why it’s important to me. The intelligent LLM can reason through how hard it needs to try to use this model as a subagent.
This is my way of giving the agent permission to use
Autoregression is when the model generates the next token, one token at a time until the output is complete (versus diffusion where all tokens are generated simultaneously).
Things have changed more recently, but it used to be that models were trained to believe that everything in the conversation thus far was true. And while it’s no longer the case, the beginning of a conversation still sets the tone, similar to how first impressions work with people. Henceforth, you see all through that lens.
So autoregression is a bit like a narrowing hallway. It’s difficult for the LLM to suddenly make a surprising turn in logic. Everything kind of just follows the status quo.
Intros are important. After all, these agents are autoregressive, so how you open impacts how the conversation goes from there. If you want something cool, set the vibe.
This is a Claude Code feature. You have to mention it by name for the harness to allow Claude to use it. I love it because it enables Claude to string together many subagents in a workflow (actually, just regular code).
Simply mentioning this already makes Claude think in terms of writing code in one workflow, and verifying with another. By not specifying exactly how I want it done, it lets Claude figure it out. Since I was talking to Fable 5, I was pretty confident it would be able to schedule out the dynamic workflow. Fable is able to conceive of the the workflow well.
Trying to nudge it into doing more of it’s work via cheaper models. Since Fable is constructing the workflow and writing all the prompts of each of the subagents, I felt confident that Opus/Sonnet would be more than enough for the individual pieces, as long as Fable is the architect.
Intentional word choice. Any model can rip off a copy of a game easily, but the 3D part with animations is new. I figure Fable probably used these words as both inspiration for the animations, as well as during verification, “yup, that’s chaotic, done”.
A classic game.
I’m constantly looking out for words and phrases where 2-3 words expand out to whole essays, if you were to explain them. It saves typing and conveys a lot of implicit details.
Building a brand new game with totally new concepts takes a lot more prompting and iteration. Here we can simply invoke it’s name and the LLM knows vaguely what we’re after.
I tried solving this in ChatGPT two times before this. I wanted to try Codex because:
Subagents are like a normal Codex session that the agent can launch off. It’s how agents can break down the work.
Another way to think about it is a single agent has tunnel vision. It has a hard time thinking beyond what’s top of mind. Whereas each subagent is a different agent with a different tunnel vision.
That’s why it works so well to have one agent write code and another to check/verify it. If they don’t share any conversation, then the verifier is truly looking at it through fresh eyes.
Tunnel vision is a big problem for autoregressive models (how LLMs work). But here you can see that we can wield that same weakness as a rather strong asset. The verification subagents work well because they share no context. They can legitimately look at the problem as if they’d never seen it before.
Wouldn’t it be great, as humans, to be able to fork yourself and look at a problem as if you’d never seen it before? I think it would be quite useful.
I have a skill setup, both at work and at home, at ~/.codex/skills/working-with-tim
The purpose is to clarify my expectations for agents. I like agents to work autonomously. I expect that when they say the work is done, it’s actually done. I expect them to check their work.
Admittedly, this skill leads to slow response times. e.g. on the printer, it took about 2 hours. But there were no dumb tangents or circles, all the questions were good. At work, I usually get agents to work for 30+ minutes, sometimes as much as 8-10 hours. And when the work comes back, it’s done correctly.a
I like that, it’s how I work. So I setup a skill for it.
I don’t use AGENTS.md anymore, because it doesn’t trust the agent to know when to read it. The harness just
shoves it into the context randomly (in the eyes of the agent).
I’d love to say a lot more about how I think about these things, like model choices and Claude Code vs Codex. But ultimately, it’s just like chatting with a friend. Only, you just have to get to know the friend first.
Get to know what their good at vs not. How their brain thinks about problems. It takes time, but mostly that time is pure fun and tinkering.
As a bonus, here’s another one. My kids were playing Chutes and Ladders (the board game), so I had Claude Code turn it into a video game.
yo, let’s go big. I want to make a Three.js based Chutes and ladders game with real animations. Just the regular chutes and ladders game. When a player moves, it’s a real animation in 3D third person view of them climbing up some ladder or chaotically falling down a slide. Each animation is different. Also animations for each movement.
For working, use dynamic workflows. Try to make heavy use of Opus 5 and Sonnet 5. But specify it clearly for them. Use clean Fable 5 sub agent to validate.
2026-07-07 08:00:00
I built lanius, and I really want to show it to you, but first I need to explain why I think it’s necessary.
We’re headed into a new phase of AI where pushing the frontier involves scaling down the amount of compute. A couple years ago we scaled up model sizes, and then we scaled up how much time we spent thinking. The shift into agency exploded the amount of compute required, but we’re getting ever so much more done.
But there’s things happening:
In my own coding workflow, I’ve found that I can use Claude Fable to work with Opus to plan, which in turn dispatches to Sonnet to implement, and then Opus again to verify the work. Weirdly, it dramatically cuts down costs. I almost never run out of credits while I also still experience all the best things about Fable.
All of these things have something in common. They match the right-sized LLM to the right problem.
There’s this concept, Ashby’s Law, that essentially says that a problem’s solution has to be at least as complex as the problem itself.
When Ashby’s Law is broken, systems start to also feel “broken”. Like when you call into a tech support hotline and the agent isn’t allowed to go off-script so you have to idly say things like, “yep, it’s plugged in. Yep, it’s turned on” in the face of ever-growing frustration.
What we’re experiencing is the other side of that. LLMs were such good problem absorbers that we threw them at
everything. E.g. in the OpenClaw docs they suggest setting up a HEARTBEAT.md file to do things
like check your email every 30 minutes.
Holy god in heaven dost thou ACTUALLY need a whole ass LLM just to realize there’s no new emails??
A more Ashby-pilled approach is to have the agent write a script to check the email inbox, and then send a message to the agent if there’s new email. Or better yet, send the message to a tiny LLM who can triage the email based on some policy (“eh, emails from Jack can wait a day or two, no sweat”) before forwarding it on to the agent who you actually trust to respond. Match the problem.
Right-sizing to the problem is going to involve a heck of a lot of moving parts.
Some are just scripts that an agent can call. Others are scripts that call the agent. Or big Fable-sized agents orchestrating a small army of local LLMs to continuously monitor a situation. Or a local coding agent consulting Fable on a problem that it’s out of its depth on. A lot of it needs to be very dynamic and temporary, coming into and out of existence as needed. Similar to how Claude’s Dynamic Workflows are scripts written in-the-moment by an orchestrating agent.
Dwarkesh found that we’re about 4 orders of magnitude off from the level of hardware to scale today’s agent approach to the entirety of society. Yet we’re already experiencing growing pains, while we still need to 1000x to have the level of impact we think we’ll have. Clearly we need a new approach.
Lanius tackles this by being an agent harness that looks more like an operating system. You can create agents directly on lanius that look a lot like OpenClaw agents, but you can also open Claude Code inside lanius and get all the same benefits.
The core shape is everything is a message.
As an example, messaging between agents, as core as it seems like it is, is just a package consisting of skills, computed memory blocks (notify new messages) and daemon scripts (an MQTT client). Lanius is absolutely built for extension, right down to the core.
Is it an OS? Probably not. It’s closer to a control plane like Kubernetes. But where Kubernetes orchestrates containers, lanius provides a messaging layer that allows agents to orchestrate themselves. Lanius is that missing machinery that, if we’re going to be serious about scaling up impact by scaling down agents to right-sized problems, well we’re going to need a whole lot of whatever lanius is. Messaging and conflict avoidance deeply embedded.
As a messaging-forward AI platform, lanius seamlessly ties together any system in your digital life. We haven’t yet built all the integrations. We also don’t need to because lanius is extensible to the core.
Skills alone are inadequate for interoperability, because any action has to be initiated by an LLM. That’s not Ashby-pilled enough for me. In order to scale down compute, we need actions to be able to start from outside an LLM. Lots of messaging sources, pollers polling websites, websockets listening for events, etc. All coalescing under one single umbrella: the MQTT broker. Messaging at the core.
Interaction models are another version of this.
The latest GPT-Live announcement makes the split explicit: keep a fast conversational model in the loop, then delegate harder work to a deeper model in the background. That feels exactly right to me. It will not scale to make every live model as smart as Sol or Fable. At some point, it needs to borrow intelligence from a worker that is better suited to the job.
When the last realtime GPT came out, I made a web app that hooked it up to Claude Code so my daughter could speak games into existence. The realtime model did not write code. It translated her intent into messages for the coding agent, which was specialized for the actual work.
Scaled down to purpose.
MQTT is an Internet of Things (IoT) protocol. The better AI gets, the more corners of my life I want to include it ambiently. The IoT tie-in was honestly incidental, but once the idea entered the picture, it quickly felt very natural. It’s very nice to disconnect from screens. (I’m firmly against putting coding AI on my phone because it bleeds into my personal life too much)
I’ve been dogfooding lanius while building it for the last few weeks. I’ve developed a workflow where I use Claude Fable to replace me — Opus develops plans while Fable drives it. And then Fable dispatches Sonnet, Opus, GPT-5.5, or GLM-5.2 for implementation and verification.
When I’m writing code this way, Fable can take on whole sections of the backlog, working for 5-10 hours at a time. I don’t review the code, Fable does sprint demos and shows me what was produced (okay fine, it’s a review).
Meanwhile, that’s a very long time between interactions. I don’t want to disturb Fable, so I’ll open up another Claude, Codex or opencode through lanius and tackle some unrelated aspect. Early on, before lanius was fully developed, this would confuse Fable and it would stop its work short of being done. But now, it introspects what the other agents are doing and even figures out how done their tasks are, sometimes picking up and finishing their work.
Initially, lanius was going to be a replacement for open-strix, a stateful agent harness in the same vein as OpenClaw or Hermes Agent. I think it absolutely still makes sense to use it that way. Or you can get Hermes Agent running inside lanius, alongside your Claude Code sessions. (This doesn’t work OOTB right now though)
Some other things to try:
The cool part is you can start Claude Code through lanius without doing any of this, and just add different features progressively as you want. And yeah, maybe revisit your old IoT hack projects and see if they make more sense with an MQTT broker and an ambient subagent.
Honestly, my kids have been stealing snacks out of the pantry and it’s getting out of hand. I was going to setup a camera with a local VLM to detect the theft and have the robot yell at them in my voice. But again, the Ashby-pilled version has a lot of deterministic pre-processing to filter out frames that don’t need a VLM. God this all smells like IoT.
Meanwhile my local VLM is sitting there idle. I guess I could have it aggressively scan for potentially interesting news? I mean, in the time gaps when it’s not stalking my kids. Obviously lanius is going to pause the news gathering to process to urgently yell at my kids, that’s important. That means two or more threads sharing a GPU, working toward the common goal of producing good stuff that I want.
That’s cost management, and it’s going to be critical. Start thinking about agent workloads in fixed budgets with line items in dollars. Then just change up the Ashby-pilled tasks to maximize the value you get from the system.
But for now, go install it! It’s just
cargo install lanius
And then keep this running in a terminal or systemd:
lanius serve
You can chat with it and configure it in the web UI, or launch Claude Code (lanius code claude),
Codex (lanius code codex). When you start them through lanius, Claude can suddenly subagent to Codex and
vice versa. And if you launch a second agent on the side, they can negotiate and figure out what each other are
doing without using you as a substrate (tiring).
I’m excited to see what you do!
2026-05-17 08:00:00
Recently I started experimenting with ambient associative memory with my open-strix agents. I’m convinced that ambient memory is definitely some piece of the puzzle, although I doubt I’ve landed on the best way.
Break it down:
What I’ve done:
It’s ambient because it happens on every tool call. The agent isn’t intentionally searching. They do whatever they’re asked to do and something randomly comes to mind.
My agents keep making the same mistake twice. In the debrief they nail the lesson — “next time, check X before changing Y”. So we add it to the rules, the pile grows and then the pile just gets ignored.
Ambient associative memory changes this by forcefully (but gently) bringing to mind relevant parts of their memory. Thereby creating coherence across their memory.
The 8-12 words is also important. It’s very small, lightweight, and only the most relevant parts of the most relevant chunks is included. You can’t do this with a normal embedding model.
With a normal single vector embedding model, you divide a document up into 250-500 token chunks. When you query, you get back an entire chunk along with a relevance score. The chunk is as small as it goes.
Compare that with late interaction models. You still chunk up the document, but instead of getting back a single vector, you get one vector per input token. When you query, you get a score for each token. So you can pinpoint which parts of the matching document were most important. When I’m formatting the RAG results to include into the prompt, I use these scores to locate the single token with the highest relevance, and include several tokens around that as context.
But you can also get a single score per document. You just pool (average) all the tokens together into a single vector. For me, I had to break the query up into 2 stages because query time performance was too slow. I start with very large chunks, 32K tokens, and then pool them into 100a token chunks and store those in the index. Then I do the full multi-vector scoring on only the 100 top hits.
3fz on bluesky is doing the same thing, but more sophisticated. Her agent runs a subconscious background thread alongside the main model. It mines an experiential vector DB and injects what it finds on top of the live context.
The two are racing. If the cross-encoder reranker beats the main model, the injection lands after the current tool call (the prefill switch is a convenient hook). If it loses, the injection slips to the next tool call. Sometimes it returns nothing. That’s the design — injection is conservative on purpose.
This sits on top of a more traditional stack: self-managed memory blocks, an initial retrieval pass at each user turn, plus a second LLM kept warm to extract atomic memories from the agent’s experience as it runs.
The framing she uses is spontaneous recall — surfacing unknown unknowns near wherever the conversation has drifted, things the agent wouldn’t have known to search for. Inspired by human cognition.
Mine is the dumb-and-synchronous version: every tool call, block and query. Hers parallelizes and gracefully drops the slow ones. Probably the right move once the index gets big.
I think there’s a lot more of these ideas. We’re still early in agent design. I think the important part is that the single thread that’s handling the main task isn’t also responsible for stopping the line of thought to query it’s own memory in lock-step.
This feels like information theory at work. Our own brains as well as CPU architecture discovered that it’s hard to do 2+ things at once. It really feels like there’s some sort of law dictating that high quality associative memory needs to happen out of band, otherwise it distracts from the task at hand.
I’m excited to see more of these.
2026-04-27 08:00:00
Say you get asked to “add memory” to an agent. What does that mean? How do you do it?
There’s three common kinds of mutable memory:
If you don’t need the agent to learn, then you’re looking in the wrong place. You don’t need memory. But this post might also be useful if you’re just using agents, like a coding agent.
Everything in this post needs to satisfy the following functions:
ls, find, grep, or equivalent toolscat, or some ReadFile toolsed, or some WriteFile toolFor files, all that seems fairly obvious. Files can be complicated, but those are the parts that are
important for files to work as agent memory.
Files don’t have to be literal files. If they are, you can provide a Bash tool (or Powershell) that
gives you cool Linux utilities for navigating the filesystem, reading parts of files, etc.
But also, you can absolutely use database records or S3 blobs. As long as:
Memory blocks are just a flat key-value store. Except the key isn’t used for looking things up, it’s just used for writing. All memory blocks are included inline in the system prompt, or user prompt.
Where to put it?
Either is fine. User prompt is slightly better, I guess.
Required tools:
WriteBlock(key, value [, sort_order]) — I like including a sort_order, because we know order does matter,
so let the agent control it too. Not a huge deal though.Optional tools:
ListBlocks()ReadBlock(key)Theoretically you don’t need these because they’re in the prompt already, but I’ve noticed that coding agents will always try to insert them and agent agents will always call them, every time. So, whatever that means..
Blocks are a learnable system prompt. Put stuff in there that tends to go into the system prompt — behavior, preferences, identity, character, etc.
Since it’s in the prompt, the agent can’t look away or ignore. So you may want to promote from file to block if you want to guarantee visibility, like you don’t want to risk the agent forgetting to read a file.
Skills are a combination of files & memory blocks. They’re files, literally, but they also are represented in the system prompt.
It’s just a directory with a SKILL.md file:
the-skill/
SKILL.md
important-concept-1.md
helper-script.py
worksheet.csv
The SKILL.md is generally just a plain markdown file, but it has a special top few lines at the start of the file:
---
name: the-skill
description: what it does and when to use it
---
The description is the critial part. Both name and description go into the system prompt, but the
description is the trigger. It encourages the agent to use the skill in the right circumstance.
Not really. Claude Code has a Skill(name) tool, but functionally it’s the same as the agent
reading the-skill/SKILL.md with a regular Read tool. The benefits are harness-side: lazy-loading
the SKILL.md content (so it only enters the context window when invoked), telemetry, and permission
scoping.
If you skip the dedicated tool, just tell the agent in the system prompt: “When a skill matches, read its SKILL.md before doing the thing.” Works fine.
Data or instructions that are only needed in certain circumstances. Honestly “skill” is actually a really good name for them.
The key phrase is progressive disclosure — skills unfold as needed. The agent reads files as it deems
necessary. Typically you’ll include file references in the SKILL.md file, like “Read important-concept-1.md
when you need to…“. There’s nothing special, no notation, it’s just hints for the agent.
Scripts and data are nice too. Obviously scripts are only useful if you enable a Bash tool, but scripts especially can act like a agent optimizer. Like, sure, the agent can probably figure out how to string together all the headers to authenticate to your weird API, or you can just make a script for it and skip the LLM.
Most people think of skills as being immutable programs of English. Sure, they’re useful when used like that, but they’re even more useful when you allow your agent to change them.
A great way to use skills is as an experience cache. At the end of a long investigation or research, have the
agent record the experience in a skill. Next time, it just reads the skill!
Could you use files for this? Yes, but the description field in the system prompt makes it more likely to be
used at the right time.
How do you know when the agent is using memory well?
For files & skills, you can start at the entry point and construct a graph of which files reference which other files:
Then compare against reality. Find all the times those files were accessed in that order versus not. If they’re referenced randomly, that means the agent needs to use Search or ListFiles tools to navigate. That might mean your files or skills are becoming too unwieldy.
Also, you should monitor memory block size & count. Definitely keep them under 5000, probably under 500 characters. When the blocks get too big, they tend to confuse the agent.
Unfortunately, given the nature of agents, there’s not that much you can do for observability. But these two things do tend to be useful to monitor.
Is a search index a good idea? Yes absolutely. It’s just annoying.
Seriously, it adds a data asset that needs to be maintained. Most of the time that’s not a huge deal, but when it is, it is. Your call.
I highly recommend versioning files & ideally also skills & memory blocks. In open-strix I store memory blocks in yaml so they version and diff cleanly.
Versioning gives you checkpoints and lets you see evolution. It also lets you rollback or let the agent discover when a bad change was made. I’ve tried to use branching and merging, but not successfully.
Knowledge graphs and other writable data models, e.g. backend by SQL, tend to not work very well because the LLM’s weights doesn’t know about their schemas. Most people talk themselves into knowledge graphs because they have structure and historically structure has been good. But the only structure LLMs need is tokens. They reason just fine in token space.
I’ve discovered that some types of generic data structures can be very useful for agents, for special purposes.
Issue trackers are oddly useful. I’ve been using chainlink, which is an issue tracker specifically for agents, but I’ve heard Asana also works fine. Probably any issue tracker would work. An issue tracker gives you a searchable work queue.
I’ve added an interest backlog to all of my agents now. Any time they come across something weird, interesting,
or annoying they can create an issue and tag it interest. Then, during the night while I sleep they work
through the backlog. This has led to multiple agents making connections between ideas & things I hadn’t discovered
yet, and generally coming up with fresh ideas that feel honestly novel.
Also, an append-only log is super useful. I have an events.jsonl file that goes into all of my agents. The
agent harness writes every single event that happens, like tool calls and messages, and appends a JSON object
minified to the events.jsonl file. It’s not writable memory in the normal sense, but the agent can read it
to give grounded answers about what it actually did.
Editable memory is extremely powerful. I highly recommend trying it out. Hopefully this helped.
2026-04-25 08:00:00
How do you scale out AI use throughout a software engineering org? Do the PM & Engineer roles merge? I think it’s worth stepping back and looking at it through a familiar lens — distributed systems.
Have you ever partitioned a database table? The idea is, if a table is receiving too much traffic, you can split the table into 2 parts (partitions) and each table only needs to handle half the traffic. Then you relocate those partitions onto different computers, and voila! Scale. 10 partitions = 10x the traffic.
The web scale era was dominated by partitioning. If you can figure out how to partition any kind of load whatsoever, then you can figure out scale. Shared nothing emerged as we bumped into new bottlenecks. It wasn’t enough to partition a service or a table. Any kind of shared state is a hot spot liability and must be removed.
It started with databases but it infected the entire software stack. Load balancers, web services, control plane / data plane split, deployments, etc. If you can identify the shared state, you can eliminate them. You can scale.
People are noticing that vibe coding causes problems. Throw AI tools at a team, suddenly the cost of producing software is near zero. Thousands of lines of code fall out effortlessly. The new problem: conflicting changes causes the team to trip over themselves.
So… a hot spot? Can we partition this?
What if we viewed a code base as if it were a distributed database. Instead of traffic, let’s look at change. Everyone on the team is making changes all at once with their agents. An agent can write 10k lines of code in the time it takes to have a meeting about retries. Claude Code can autonomously debug and fix a gnarly bug while you get coffee.
But a merge conflict? Everything stops to wait on the humans.
What if we introduced shared nothing architecture to this? We could view code changes the same way we view traffic flow in distributed systems.
We know that vibe coding a prototype is easy, but working on an established code base is hard. Why not lean into that?
It seems, then, that a well-designed code base should be small and focused. So if you want to build a big product, it should ideally be composed of lots of tiny components that can each be rewritten on a whim.
Take a B2B SaaS with a bunch of customer integrations — Salesforce, HubSpot, Zendesk, etc. The instinct is to build a generic Integrations Framework and let each integration plug into it. The framework owner becomes the hot spot. Every PR queues behind their review. Adding Zendesk requires coordinating with whoever’s doing HubSpot, because both are mutating the shared abstraction.
Partition it instead. Each integration becomes its own vertical slice — UI, API, auth, tests — owned end-to-end by one human+agent unit. They never touch each other’s code. The duplication that would have justified the framework is cheap now, because the agent writes the boilerplate in minutes.
Conway’s Law says products mirror the org that shipped them. Why? Because communication cost was the dominant coefficient in design. You couldn’t beat it, you could only choose where to pay it. In-org comms were cheaper than cross-org, so you aligned the code with the org chart and saved on the gradient.
AI doesn’t repeal Conway. It changes the coefficient. Code costs almost nothing now:
Coordinating a hand-off between two services takes longer than building an entire app end-to-end. When the ratio between code-cost and comms-cost flips that hard, the Conway-optimal partition moves with it — toward fewer hand-offs, even if that means duplicating what used to be shared. Conway predicts this. We just hadn’t seen the coefficient move this fast before.
Fully parallelize the components and you find the next bottleneck.
Length-wise, this feels like a lot of hand-offs:
Conway would have said these hand-offs were unavoidable, so re-org around them. But now, each individual hand-off dwarfs development time. Can we still rationalize it?
Each step in this queue has to be maintained, ensuring there’s enough Engineering capacity, but then also enough QA capacity to ensure that QA doesn’t become the bottleneck. In distributed systems, misconfigured queues are a big source of bottlenecks and operational issues.
Just hire a manager, right? Well, sure, but having multiple steps when one would do is usually worse due to context fidelity loss. At each hand-off, some amount of work is dropped due to people miscommunicating or simply forgetting a step.
Why not rip them out? Fuse them together. That’s usually the solution in distributed systems. Is it feasible? Can a human-AI team handle the full end-to-end?
Anecdotally, I’ve discovered that Claude can do product work quite well. It takes a lot of context though. I use open-strix daily. It’s a stateful agent, and I cue it into everything I’m doing, people I work with, projects, etc. Last week I had it define a product for a new idea that I had and it knocked it out of the park. I woke up in the morning with a long report including market analysis, competition, compelling use cases, architectural considerations like where it would plug into the full system.
I’m fairly well convinced that an AI+engineer combo can successfully venture into product. I’ve also seen product people venture into engineering with Claude Code. I think it’s especially feasible if you partition out the product to scale — each job partition becomes small enough to be understood by a single person.
So I’m not sure what direction it will fuse, but it feels inevitable. And the resulting role won’t look all that much like either does today. It seems that product strategy, cohering the product surface together, is the next bottleneck. And I’m sure we’ll sort that out too.
The marketing pipeline — Strategist → Copywriter → Designer → Channel → Analyst — is a sequential service chain. Big agencies aren’t slow because their people are bad; they’re slow because every asset crosses four hand-offs. Shard by campaign. Each campaign is a vertical slice owned by one human+agent unit. The hot spot disappears.
Sales has the same shape. SDR → AE → CSM is a service chain; context decays at every hand-off. Sales orgs already partition by account or territory — the role pipeline is the framework that grew on top. Collapse it. One rep + AI owns research → outreach → close → renewal for their slice.
Customer support: L1 → L2 → escalation is the pipeline; the ticket is the slice. One human + AI owns it end-to-end, and AI absorbs the L1 reflex work that used to need a separate role.
Distributed systems patterns. Different vocabulary.
Distributed systems didn’t get to shared-nothing for free. Some state genuinely resists sharding — global counters, foreign-key constraints, brand voice, legal precedent. You cache it. You replicate it. You accept eventual consistency. Sometimes you designate one shard as canonical and route all writes through it.
There’s one residue that doesn’t partition at all: someone has to be on the hook. AI can produce the work but it can’t sit in a deposition. Can’t have a license revoked. Can’t be sued. Every regulated profession is a system for designating who pays when things go wrong. The license isn’t a credential of competence — it’s a credential of vulnerability. The doctor is the body the lawsuit lands on.
In distributed systems we’d call that the master. The one node that owns the write. As AI gets better at the work, the master role becomes pure accountability-bearing — humans paid mostly to absorb blame for systems they only partially understand.
Partitioning didn’t carry distributed systems on its own. It needed a layer that didn’t exist yet — SRE, eventually — to keep partitions honest. Without it, shared-nothing decays into uncoordinated chaos within a year.
Vertical slices need the same thing. I don’t have a name for it. The job is mostly catching the hot spot before it re-forms: a “shared helper” that everyone has to touch, a meeting that has to include four units, a slice quietly opening PRs into another’s repo. Early signs the partition is leaking.
Like SRE in 2003. No job description, and then everyone needed one.
Everything else is a candidate for partition. Eventually.
2026-04-14 08:00:00
Today, Lily asked me, “what’s the difference between open-strix and openclaw?” Great question. We commonly use open-strix agents for higher-level tasks. I use mine at work to lead an agent team, Lily uses one as a strategist for her marketing ops work. Whereas everyone I hear using openclaw just uses them as dumb automation machines. Why the difference?
Strictly speaking: open-claw is biased toward reading (recalling), while open-strix is biased toward writing (remembering better). Where open-claw (and most others) focus on better search indexes to find information, open-strix does something very strange, we intentionally do a worse job searching, in order to improve remembering better.
Why? Because it’s a long-lived agent.
I don’t think I ever explained this clearly earlier, I always assumed it was obvious, but maybe it’s not. It’s also the common thread across all the Strix versions and probably the thing that makes the architecture unique.
Compaction is a fallback, and it’s a really harsh one that’s poorly fitted for long-lived agents.
I wrote in depth about why fallbacks are bad, and it’s kind of a subtle thing. But in this case, when the conversation fills the context, you have an OutOfMemory-type error, and the fallback is to compact the context. It’s terrible, because suddenly your agent randomly becomes very dumb, it loses 98-99% of its memory and you have no control over how that happens. Mid-conversation, it forgets your project context and asks you to re-explain what you’re working on.
open-strix doesn’t do that. It rebuilds the context every time. In practice, this looks like a sliding window over the conversation history.
Prompt caching.
Almost all LLM providers offer a discount, like 50%, for reusing the same conversation prefix. So generally we do append-only patterns. That’s how ChatGPT works, that’s how Claude Code works, they all take advantage of prompt caching.
But, in continuously running agents, messages often don’t have a sequential nature. Each new message can come from a wildly different channel. One comes in over discord, the next comes from a github issue, the next a Google doc comment. My open-strix agents don’t really benefit from that continuity.
If you have 400M token context, then on average you’re pushing 200M input tokens on each message. Whereas me, I’m at 10K-20K tokens per message. Strangely, doing it the expensive way is actually cheaper.
It’s easy for computers to remember everything, they’ve been doing it for decades. Remembering nothing is just /dev/null, so the trick is always to remember the right amount.
Our brains have finite capacity to remember. But that super smart person seems to remember all the exact right things. Do they have a bigger brain capacity? No, they just know what to remember. Smart people are able to see the future and predict what they’ll need to know. And then forget the rest.
But “forget” is misleading. Open-strix doesn’t delete anything — it just doesn’t promote. The sliding window drops context that didn’t earn its way into memory blocks. That’s not amnesia, it’s editorial judgment. “Forgetting” is the provocative word for “I only kept what changed my behavior.”
Framing it as intelligence is bland. We’re all different. We have different interests and expertise. And all that influences what we remember. Me, an AI guy, I cluster toward AI algorithms, architectures, models, whatever. Back in high school it was punk & hardcore band trivia. Neither of these things make me smarter, they just make me more me. And the more I learn, the even more I become my new future self.
That’s the thing, forgetting without having accumulated anything is just being empty. Remembering everything without forgetting is context collapse — too much information to navigate. The useful part is the selection pressure — the constant question of what’s worth keeping. That pressure is what creates identity.
The benefit of a stateful agent like open-strix is it has a perspective.
It’s hard to understate how useful this is. Generic ChatGPT advice is great and all, but if you can wrap the same LLM with a thick layer of memory and experiences, it elicits behavior from the LLM that is very far from mid. Everything the LLM says is filtered through the personality and memories of the agent. The agent (LLM with memory) now has the wisdom and foresight to predict what will be important in the future.
For example, in reviewing this blog post, stock Claude gave me some light areas of improvement, and mostly green lighted it. Strix, the same exact model, told me not to post it yet. The review was structurally different — not better grammar suggestions, but challenges to the argument. “This section is rushed, and I know because I’ve watched you build this system.” The difference isn’t that Strix remembered more facts about me. It’s that shared experience gave it opinions about the subject matter, not just pattern-matching on prose quality.
Who the agent is determines who the agent becomes. That’s still wild to me.
In open-strix, if the agent doesn’t remember the right things, you know real quick. It acts spacey and dumb. It’s so painful that you have to fix it, you need to. You can’t not.
It’s easy most of the time. You just say something like, “why did you get confused about…”, the two of you discuss, and then the agent updates their memory blocks to reprioritize so they handle it better in the future.
Beyond that, open-strix has ambient processes that encourage self-healing. They feed into each other.
The first big one is teleological predictions. Yeah, this is something I totally ripped off of Karl Jung from Psych 101, but it’s super useful. You can’t trust agents, they lie (same with therapy patients). So what you do instead is make a prediction about the future. If it’s wrong, the agent’s mental model about how the world works was wrong. So it needs to be fixed.
Aside: I embarrassingly had an agent get excited about the accuracy of its predictions that I would ignore everything it did. That was definitely an accurate mental model but…
But what to do about it? 5 Why’s
When an agent runs into anything surprising, like a failed teleological prediction, it does the 5 Why’s process.
You get it. 5 levels is a pretty good number, but realistically it digs up a whole bunch of other questions. You often end up discovering not just one root cause, but 3 or 4. It starts to look like a whole systemic issue. Which it is. Always.
Every time I’ve done this with (human) teams, everyone loves it because it’s almost like a murder mystery. No one entirely knows what the true cause will end up being, but everyone knows it’ll be worth fixing.
We don’t fix individual memories that often, because that’s tantamount to adding cold paths, each one is very rare and unique so almost impossible for the operator to catch. Instead, our mitigations all go toward stabilizing the agent’s identity. And the identity is the best tool available to improving what’s remembered vs what’s not.
So Lily, the real difference? Openclaw remembers more. Open-strix forgets better.
