Luke WroblewskiModify

2025-09-30 08:00:00

I recently had the pleasure of speaking with Ben Lorica on O'Reilly's Generative AI in the Real World podcast about how software applications are changing in the age of AI. We discussed a number of topics including:

• The shift from "running code + database" to "URL + model" as the new definition of an application
• How this transition mirrors earlier platform shifts like the web and mobile, where initial applications looked less robust but evolved significantly over time
• How a database system designed for AI agents instead of humans operates
• The "flipped" software development process where AI coding agents allow teams to build working prototypes rapidly first, then design and integrate them into products
• How this impacts design and engineering roles, requiring new skill sets but creating more opportunities for creation
• The importance of taste and human oversight in AI systems
• And more...

You can listen to the podcast Generative AI in the Real World: Luke Wroblewski on When Databases Talk Agent-Speak (29min) on O-Reilly's site. Thanks to all the folks there for the invitation.

2025-09-26 08:00:00

In his How to solve the right problem with AI presentation at Future Product Days, Dave Crawford shared insights on how to effectively integrate AI into established products without falling into common traps. Here are my notes from his talk:

• Many teams have been given the directive to "go add some AI" to their products. With AI as a technology, it's very easy to fall into the trap of having an AI hammer where every problem looks like an AI nail.
• We need to focus on using AI where it's going to give the most value to users. It's not what we can do with AI, it's what makes sense to do with AI.

AI Interaction Patterns

• People typically encounter AI through four main interaction types
• Discovery AI: Helps people find, connect, and learn information, often taking the place of search
• Analytical AI: Analyzes data to provide insights, such as detecting cancer from medical scans
• Generative AI: Creates content like images, text, video, and more
• Functional AI: Actually gets stuff done by performing actions directly or interacting with other services
• AI interaction patterns exist on a context spectrum from high user burden to low user burden
• Open Text-Box Chat: Users must provide all context (ChatGPT, Copilot) - high overhead for users
• Sidecar Experience: Has some context about what's happening in the rest of the app, but still requires context switching
• Embedded: Highly contextual AI that appears directly in the flow of user's work
• Background: Agents that perform tasks autonomously without direct user interaction

Principles for AI Product Development

• Think Simply: Make something that makes sense and provides clear value. Users need to know what to expect from your AI experience
• Think Contextually: Can you make the experience more relevant for people using available context? Customize experiences within the user's workflow
• Think Big: AI can do a lot, so start big and work backwards.
• Mine, Reason, Infer: Make use of the information people give you.
• Think Proactively: What kinds of things can you do for people before they ask?
• Think Responsibly: Consider environmental and cost impacts of using AI.
• We should focus on delivering value first over delightful experiences

Problems for AI to Solve

• Boring tasks that users find tedious
• Complex activities users currently offload to other services
• Long-winded processes that take too much time
• Frustrating experiences that cause user pain
• Repetitive tasks that could be automated
• Don't solve problems that are already well-solved with simpler solutions
• Not all AI needs to be a chat interface. Sometimes traditional UI is better than AI
• Users' tolerance and forgiveness of AI is really low. It takes around 8 months for a user to want to try an AI product again after a bad experience
• We're now trying to find the right problems to solve rather than finding the right solutions to problems. Build things that solve real problems, not just showcase AI capabilities

2025-09-26 08:00:00

In her talk Reveal the Hidden Forces Driving User Behavior at Future Product Days, Sarah Thompson shared insights on how to leverage behavioral science to create more effective user experiences. Here's my notes from her talk:

• While AI technology evolves exponentially, the human brain has not had a meaningful update in approximately 40,000 years so we're still designing for the "caveman brain"
• This unchanging human element provides a stable foundation for design that doesn't change with every wave of technology
• Behavioral science matters more than ever because we now have tools that allow us to scale faster than ever
• All decisions are emotional because there is an system one (emotional) part of the brain that makes decisions first. This part of the brain lights up 10 seconds before a person is even aware they made a decision
• System 1 thinking is fast, automatic, and helped us survive through gut reactions. It still runs the show today but uses shortcuts and over 180 known cognitive biases to navigate complexity
• Every time someone makes a decision, the emotional brain instantly predicts whether there are more costs or gains to taking action. More costs? Don't do it. More gains? Move forward
• The emotional brain only cares about six intuitive categories of costs and gains: internal (mental, emotional, physical) and external (social, material, temporal)
• Mental: "Thinking is hard" We evolved to conserve mental effort - people drop off with too many choices, stick with defaults. Can the user understand what they need to do immediately?
• Social: "We are wired to belong" We evolved to treat social costs as life or death situations. Does this make users feel safe, seen, or part of a group? Or does it raise embarrassment or exclusion?
• Emotional: "Automatic triggers" Imagery and visuals are the fastest way to set emotional tone. What automatic trigger (positive or negative) might this design bring up for someone?
• Physical: "We're wired to conserve physical effort" Physical gains include tap-to-pay, facial recognition, wearable data collection. Can I remove real or perceived physical effort?
• Material: "Our brains evolved in scarcity" Scarcity tactics like "Bob booked this three minutes ago" drive immediate action. Are we asking people to give something up or are we giving them something in return?
• Temporal: "We crave immediate rewards" Any time people have to wait, we see drop off. Can we give immediate reward or make people feel like they're saving time?
• You can't escape the caveman brain, but you can design for it.

2025-09-25 23:00:00

In her The AI Adoption Gap: Why Great Features Go Unused talk at Future Product Days in Copenhagen, Kate Moran shared insights on why users don't utilize AI features in digital products. Here's my notes from her talk:

• The best way to understand the people we're creating digital products for is to talk to them and watch them use our products.
• Most people are not looking for AI features nor are they expecting them. People are task-focused, they're just trying to get something done and move on.
• Top three reasons people don't use AI features: they have no reason to use it, they don't see it, they don't know how to use it.
• There are other issues like in enterprise use cases, trust. But these are the main ones.
• People don't care about the technology, they care about the outcome. AI-powered is not a value-add. Solving someone's problem is a value-add.
• Amazon introduced a shopping assistant that when tested, people really liked because the assistant has a lot of context: what you bought before, what you are looking at now, and more
• However, people could not find this feature and did not know how to use it. The button is labeled "Rufus" people don't associate this with something that helps them get answers about their shopping.
• Findability is how well you can locate something you are looking for. Discoverability is finding something you weren't looking for.
• In interfaces that people use a lot (are familiar with), they often miss new features especially when they are introduced with a single action among many others
• Designers are making basic mistakes that don't have anything to do with AI (naming, icons, presentation)
• People say conversational interfaces are the easiest to use but it's not true. Open text fields feel like search, so people treat them like smarter search instead of using the full capability of AI systems
• People have gotten used to using succinct keywords in text fields instead of providing lots of context to AI models that produce better outcomes
• Smaller-scope AI features like automatic summaries that require no user interaction perform well because they integrate seamlessly into existing workflows
• These adoption challenges are not exclusive to AI but apply to any new feature, As a result, all your existing design skills remain highly valuable for AI features.

2025-09-25 17:00:00

In his talk The Future of Product Creators at Future Product Days in Copenhagen, Tobias Ahlin argued that divergent opinions and debate, not just raw capability, are the missing factors for achieving useful outcomes from AI systems. Here are my notes from his presentation:

• Many people are exposing a future vision where parallel agents creating products and features on demand.
• 2025 marked the year when agentic workflows became part of daily product development. AI agents quantifiably outperform humans on standardized tests: reading, writing, math, coding, and even specialized fields.
• Yet we face the 100 interns problem: managing agents that are individually smarter but "have no idea where they're going"

Limitations of Current Systems

• Fundamental reasoning gaps: AI models have fundamental reasoning gaps. For example, AI can calculate rock-paper-scissors odds while failing to understand it has a built-in disadvantage by going second.
• Fatal mistakes in real-world applications: suggesting toxic glue for pizza, recommending eating rocks for minerals.
• Performance plateau problem: Unlike humans who improve with sustained effort, AI agents plateau after initial success and cannot meaningfully progress even with more time
• Real-world vs. benchmark performance: Research from Monitor shows 63% of AI-generated code fails tests, with 0% working without human intervention

Social Nature of Reasoning

• True reasoning is fundamentally a social function, "optimized for debate and communication, not thinking in isolation"
• Court systems exemplify this: adversarial arguments sharpen and improve each other through conflict
• Individual biases can complement each other when structured through critical scrutiny systems
• Teams naturally create conflicting interests: designers want to do more, developers prefer efficiency, PMs balance scope.This tension drives better outcomes
• AI significantly outperforms humans in creativity tests. In a Cornell study, GPT-4 performed better than 90.6% of humans in idea generation, with AI ideas being seven times more likely to rank in the top 10%
• So the cost of generating ideas is moving towards zero but human capability remains capped by our ability to evaluate and synthesize those ideas

Future of AI Agents

• Current agents primarily help with production but future productivity requires and equal amount of effort in evaluation and synthesis.
• Institutionalized disconfirmation: creating systems where disagreement drives clarity, similar to scientific peer review
• Agents designed to disagree in loops: one agent produces code, another evaluates it, creating feedback systems that can overcome performance plateaus
• True reasoning will come from agents that are designed to disagree in loops rather than simple chain-of-thought approaches

2025-09-16 08:00:00

With each new technology platform shift, what defines a software application changes dramatically. Even though we're still in the midst of the AI shift, there's emergent properties that seem to be shaping what at least a subset of AI applications, let's call them chat apps, might look like going forward.

At a high level, applications are defined by the systems they're discovered and operated in. This frames what capabilities they can utilize, their primary inputs, outputs, and more. That sounds abstract so let's make it concrete. Applications during the PC era were compiled binaries sold as shrink-wrapped software that used local compute and storage, monitors as output, and the mouse and keyboard as input.

These capabilities defined not only their interfaces (GUI) but their abilities as well. The same is true for applications born of the AI era. How they're discovered and where they operate will also define them. And that's particularly true of "chat apps".

So what's a chat app? Most importantly a chat app's compute engine is an AI model which means all the capabilities of the model also become capabilities of the app. If the model can translate from one language to another, the app can. If a model can generate PDF files, the app can. It's worth noting that "model" could be a combination of AI models (with routing), prompts and tools. But to an end user, it would appear as a singular entity like ChatGPT or Claude.

When an application runs in Claude or ChatGPT, it's like running in an OS (windows during the PC era, iOS during the mobile era). So how do you run a chat app in an AI model like Claude and what happens when you do? Today an application can be added to Claude as a "connector" probably running as a remote Model Context Protocol (MCP) server. The process involves some clicking through forms and dialog boxes but once setup, Claude has the ability to use the application on behalf of the person that added it.

As mentioned above, the capabilities of Claude are now the capabilities of the app. Claude can accept natural language as input, so can the app. When people upload an image to Claude, it understands its content, so does the app. Claude can search and browse the Web, so can the app. The same is true for output. If Claude can turn information into a PDF, so can the app. If Claude can add information to Salesforce, so can the app. You get the idea.

So what's left for the application to do? If the AI model provides input, output, and processing capabilities, what does a chat app do? In it's simplest form a chat app can be a database that stores the information the app uses for input, output, and processing and a set of dynamic instructions for the AI model on how to use it.

As always, a tangible example makes this clear. Let's say I want to make a chat app for tracking the concerts I'm attending. Using a service like AgentDB, I can start with an existing file of concerts I'm tracking or ask a model to create one. I then have a remote MCP server backed by a database of concert information and a dynamic template that continually instructs an AI model on to use it.

When I add that remote MCP link to Claude, I can: upload screenshots of upcoming concerts to track using Claude's image parsing ability); generate a calendar view of all my concerts (using Claude's coding ability); find additional information about an upcoming show (using Claude's Web search tools); and so on. All of these capabilities of the Claude "model" work with my database plus template, aka my chat app.

You can make your own chat apps instantly by using AgentDB to create a remote MCP link and adding it to Claude as a connector. It's not a very intuitive process today but will very likely feel as simple as using a mobile app store in the not too distant future. At which point, chat apps will probably proliferate.