When is self-organization a mechanism in brain development?

Mike's Notes

Reading this article got me curious about a clear definition of Hebbian Learning.

See also

The articles by Alex Nugent at Known re: Memristor.

Hebbian Learning

(With the help of Gemini)

"Hebbian learning (also known as Hebb's rule or cell assembly theory) is a foundational neuropsychological theory stating that synaptic connections between neurons strengthen when they activate repeatedly and simultaneously. Proposed by Canadian psychologist Donald Hebb in his 1949 book The Organization of Behavior, the principle is famously summarized by the phrase: "Neurons that fire together, wire together."

How It Works in the Brain

  • Synaptic Strengthening. When a presynaptic neuron repeatedly helps trigger a postsynaptic neuron, physical and chemical changes occur at the synapse. This process makes it easier for the first neuron to activate the second in the future.
  • Timing and Causality. Hebb emphasized that the first neuron must fire just before the second neuron to establish a causal link. This insight laid the groundwork for modern research into spike-timing-dependent plasticity (STDP).
  • Long-Term Potentiation (LTP). At the cellular level, Hebbian learning is closely tied to long-term potentiation. This is a persistent increase in signal transmission between two neurons that serves as a primary physical mechanism for memory and habit formation.
  • Homeostatic Balance. Because pure Hebbian reinforcement creates a positive feedback loop where strong connections grow endlessly stronger, the brain uses mechanisms like synaptic scaling. This acts as negative feedback to keep overall neural activity stable.

Everyday Examples

  • Skill Acquisition. When you first learn to drive a car or play a musical instrument, your brain activates distinct networks with deliberate effort. Through repetition, those neural pathways wire together, making the actions automatic.
  • Associative Memory. If you consistently hear the word "green" while seeing the color green, the auditory and visual neurons wire together. Eventually, simply hearing the word triggers a mental image of the color.

Role in Artificial Intelligence

In computer science, Hebbian learning serves as a biologically inspired, unsupervised learning rule.

  • Weight Updates. In an artificial neural network, the weight (strength) of the connection between two nodes increases if both nodes activate at the same time, and weakens if they do not.
  • Local Learning. Unlike backpropagation—which requires a global error signal sent backward through an entire network—Hebbian updates are strictly local. Each connection adjusts based solely on the activity of the two neurons it directly links.
  • Key Models. Hebbian rules are foundational to Hopfield networks (used for associative memory), Boltzmann machines, Principal Component Analysis (PCA), and modern neuromorphic computing." - Gemini

Resources

References

  • https://doi.org/10.53053/TENI3021

Repository

  • Home > Ajabbi Research > Library > Subscriptions > The Transmitter
  • Home > Handbook > 

Last Updated

01/11/2026

When is self-organization a mechanism in brain development?

By: P. Robin Hiesinger
The Transmitter: 02/11/2026

P. Robin Hiesinger: Professor and head of the Division of Neurobiology, Freie Universität Berlin.

...

Living systems: Ordered patterns, such as whirlpools, emerge and persist by dissipating energy. Similar structures may be at work in development.

Illustration by Anna Ivanenko

Retinal waves offer a powerful example of a self-organizing system that can build structure without external guidance. The field would do well to embrace such phenomena more broadly as a type of mechanistic explanation.

During vertebrate retina development, waves of electrical activity spontaneously emerge and sweep across the tissue in random directions. This seminal discovery of the 1990s was predicted by theoreticians in the 1970s with surprising accuracy and has since become a classic and fundamental cornerstone of brain development: The waves ensure that neighboring neurons fire together more often than those farther apart in the retina. This, in turn, ensures that neighboring synapses in the brain are selectively strengthened and those farther apart are pruned, following the concept of Hebbian learning.

Remarkably, the spontaneous waves are generated by retinal ganglion cells without sensory input, before photoreceptor cells in the retina are mature; they are generated solely from the internal state and dynamics of the developing retina. In an enforced binary categorization of “nature versus nurture,” the retinal waves are pure nature—part of the genetically encoded developmental program that unfolds prior to any environmental input or learning. But if the process requires spontaneous activity waves and Hebbian learning, can it truly be considered genetically encoded?

The answer depends on our interpretation of the word “encoding.” Retinal waves are not written in or explained by the genome; instead, they emerge from the dynamics of cell interactions. Explanations of this kind are common in physical chemistry. Take  “dissipative structures”—ordered patterns, such as whirlpools, that emerge and persist by dissipating energy. These structures follow two basic rules: They occur as an irreversible process in time, and they are formed by components that create a new mechanism through their interactions, with individual components on their own revealing nothing about that mechanism. 

Retinal waves meet these criteria. Individual retinal ganglion cells excite their neighbors, generating wave properties, such as wave directionality and speed through cell interactions in time. Individual neurons are oblivious to the wave they are part of, simply reacting to their neighbors’ activity. 

Based on their dissipative properties, retinal waves bear similarity to a standard example of self-organization: Rayleigh-Bénard convection rolls, which form when you heat a liquid with the right viscosity, creating highly organized and often beautiful patterns. The mechanism of convection rolls, like the mechanism of spontaneously emerging, unidirectional retinal waves with distinct speeds and random directions, can only be described in terms of the interactions of their respective constituent components in time. There is no such thing as a wave at a single moment.

So, is it useful to understand this type of self-organization as a mechanism in brain development? Brain organoids that develop from induced pluripotent stem cells into patterned brain structures are sometimes described as beautiful examples of self-organization. But where is the mechanistic insight in this? To state that the entire brain is self-organized is plausible yet nonexplanatory, as is the equally correct notion that all of brain development is based on molecular interactions.

An explanatory mechanism of brain development must, above all, provide understanding of how a system can increase internal order. Just as convection rolls in a heated liquid represent a more ordered or lower entropy state, a specifically wired brain network is more ordered than a network with random connections. The retinal waves offer such understanding, in this case by morphing initially imprecise synaptic connectivity into a patterned structure with more specific synapses. 

Part of the difficulty is the issue of what constitutes an explanation. For molecular neuroscientists, explanatory power lies in specific molecules and their measured interactions or properties. For computational neuroscientists, explanatory power lies in the understanding of algorithmic computation, whatever cells or molecules happen to execute the computation. This is what Evelyn Fox Keller called different epistemological cultures. Your mechanism may not be my mechanism, because we have different ideas of what constitutes an explanation.

I argue that self-organization constitutes its own category of explanation, shared by theorists and experimentalists. Models and mechanisms of self-organization have already proven helpful across biological systems, are well established for pattern formation in developmental biology, and may offer a key for understanding the development of brain wiring beyond the famous retinal waves. 

During Drosophila brain development, for example, photoreceptor axons rewire to create a specific pattern of connectivity within a short developmental time window—just 15 hours. Contrary to a mechanistic framework of attractive or repulsive guidance, these axons aren’t seeking a specific downstream target and do not follow a specific chemical gradient; rather, thousands of growth cones spontaneously self-assemble for this purpose into a temporary sheet, a brain structure that does not exist before or after the transformation. Without it, the brain will not wire up correctly. The fleeting conglomeration of growth cones moves toward its new connectivity state without any target other than its own searching dynamics. It’s as if a group of concert-goers are driving to the venue before it has been built; most remarkably, the theater will come into existence only if the fans seek it out. It’s the interaction of the agents, or axons, that creates the guidance mechanism, and the target only comes to be in a process that must play out in growth dynamics in time. These are, again, the criteria of self-organization for dissipative structures.

What can we learn from these examples? To provide explanatory power, self-organization mechanisms must meet two key criteria: First, new mechanistically relevant rules emerge through the interactions of a system’s components, though the individual components do not reveal those rules. Second, the mechanism itself is a process in time; no static state or depiction of the system explains the mechanism. 

To find such mechanisms, we developmental biologists might do well to look beyond the more typical molecular gradients or target molecules and consider more deeply how structures can self-assemble. Spending more time interacting with our neighbors’ epistemological cultures may be one way to do this, particularly with physicists and theoretical neuroscientists. As Ila Fiete argues in a companion essay, “We need theorists who treat development as seriously as they treat learning, and who ask what other minimal rules could produce a given structure.”  (For more, see Fiete’s essay, “The genome offers a recipe, not a blueprint, for the brain.”) I would add that we equally need developmental biologists who treat theory as seriously as they treat genes and molecules. Think of the coming together as an act of self-organization that may well increase explanatory power.

DOI Handbook

Mike's Notes

Now I know what a DOI is. 😎

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

16/11/2026

DOI Handbook

By: Mike Peters
On a Sandy Beach: 13/11/2026

Mike Peters: Mike invented and designed Pipi and founded Ajabbi.

...

A Digital Object Identifier (DOI) is a unique, permanent alphanumeric string assigned to an intellectual property object—most commonly academic journal articles, datasets, ebooks, and official reports. Managed by the International DOI Foundation (IDF), a DOI serves as a "digital fingerprint" that permanently links to a piece of content, ensuring it can always be found even if its physical web address (URL) changes.

Structure of a DOI

A DOI always begins with the number 10 and is split into two primary components separated by a forward slash:

  • Prefix (10.NNNN): Identifies the specific organization or publisher that registered the document (e.g., a university or a commercial publisher).
  • Suffix: A unique string assigned by the publisher to identify that specific document.

Example: 10.1000/182

Why DOIs Matter

Unlike standard URLs, which frequently break when websites restructure or journals change hands (a phenomenon known as "link rot"), a DOI is persistent. If an article moves to a new website, the publisher updates the underlying metadata tied to that DOI, keeping the link active.

Because of this stability, major academic formatting styles—such as APA, MLA, Chicago, and Harvard—require including the DOI when citing electronic sources.

How to Use a DOI

To read or locate a document using its DOI, you can convert it into a clickable URL by prepending the standard resolver domain https://doi.org/ to the front of the string:

https://doi.org/ + 10.1000/182 >> https://doi.org/10.1000/182

You can also search for missing DOIs or look up metadata manually via the central registry index, Crossref.

How to format it for academic references

While the text string 10.53053/ZHHE7561 is correct, modern citation guidelines (such as APA 7th edition, MLA 9th edition, and Chicago 17th edition) require you to format DOIs as live, clickable HTTPS hyperlinks in your bibliography.

Here is how you should format it depending on your destination:

  • Standard URL format (Preferred for most styles):

https://doi.org/10.53053/ZHHE7561

  • APA 7th Edition Reference List Example:

Fiete, I. (2026). The genome offers a recipe, not a blueprint, for the brain. The Transmitter. https://doi.org/10.53053/ZHHE7561

Note: Never place a full stop at the very end of a DOI hyperlink in your reference list, as this can break the link functionality for readers.

Stop Treating CSS Container Queries Like Traditional Media Queries

Mike's Notes

The original Smashing Magazine article has all the CodePen examples.

And for fun, visit Set Studio to confirm that the ideal viewport doesn’t exist.

"Here at Set Studio, we conducted a little casual experiment to answer “how fragmented are viewport sizes?”. We gathered over 120,000 datapoints with over 2,300 unique viewport sizes. The data mainly came from users in the USA and Europe, therefore it is not necessarily representative of a global audience, but still useful for this article." - Set Studio

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Smashing Magazine
  • Home > Handbook > 

Last Updated

15/11/2026

Stop Treating CSS Container Queries Like Traditional Media Queries

By: Victor Ayomipo
Smashing Magazine: 16/09/2026

Victor Ayomipo: I Code. I Write. Meme Rules. Ok, but seriously, I’m a web and mobile developer who writes a lot. In my spare time, I like to build products that would make my life easier. Because what’s more fun than making a computer do what you want it to do?

...

Quick Summary

Despite broad browser support, container queries remain surprisingly underused and frequently misunderstood. Let’s look at how they differ from media queries, when to reach for each, and how container queries help reusable components respond naturally to the contexts in which they appear.

...

To be completely honest with you, I missed the news when CSS Container Queries first shipped. And when I finally heard about it, my very first thought was, “Why exactly do I need this when media queries already exist?”

I’m not proud of that reaction, knowing what I know now, but it was comforting to know that I wasn’t alone. In fact, there are legions of us out there.

What baffles me is that container queries aren’t a new feature, as it currently sits at around 94% browser support. And yet, very few people are actually using it. According to the State of CSS survey, 86% of developers are aware of container queries, but only 41.4% actually use them. Surveys can be biased and not completely representative of our entire field, but this one is certainly the best indicator we’ve got.

Kevin Powell also talked about this at SmashingConf Amsterdam 2026: Container Queries adoption has been terrible. And that is so strange to me, knowing that the ability for components to adapt to the size of their outer container has been at the top of so many CSS wishlists over the years.

I’m not particularly interested in how many people are using container queries as much as in how they are using them. I can’t account for everyone, but from what I’ve seen — including in my own early attempts — many of us are using them wrong.

The bottom line is that incorrect use comes down to the same impression I had when learning about them: they absolutely look just like media queries at first glance. And since they look similar, it’s easy to assume they serve similar purposes and work the same way.

They don’t.

Note: I should state up front that what I’m focusing on in this article is using container size queries, i.e., a responsive design technique for responding to the size of a particular container. There are also container style queries that respond to a container’s computed styles (and are experimental at the time of this writing). You can catch up on those in Juan Diego’s piece here on Smashing Magazine where he examines their possible use cases.

Media Queries Look Outward

The viewport is a proxy. It always has been. Media queries are what gave us the illusion that screen width alone is responsible for how responsive apps adapt to their environment.

Ask yourself this: When you write @media, what are you asking the browser?

@media (min-width: 1024px) {
  .card {
    display: flex;
  }

I, like most developers, am asking the browser: How wide is the screen right now? That’s it.

Media queries answer that beautifully, but what happens when this .card component is placed in a grid cell that’s 300px wide on a 1920px desktop screen?

The media query doesn’t care; it does its job. The viewport is still 1920px, so min-width: 1024px fires and the matched query styles are applied, even though the card only has 300px of space to work with. Eventually, everything in the card deforms, overflows, or cramps up.

“Media queries are dumb. Not dumb in terms of the concept, but dumb in that they don’t know very much. In fact, most people assume that they know more than they do.” - Kevin Powell

It’s common to think of responsive design purely as a system for updating complete page layouts, like going from two columns on a large screen to a single column on a small screen.

Container Queries Look Inward

Container queries are smarter than that. They make responsive layouts more reliant on what’s happening inside a component rather than on the outer context that has no insight into a component’s contents. It is more like: “How much space is available for me in this specific spot, right now?”

Here is the same card code example we looked at in the last section, but with a container query:

.card-wrapper {
  container-name: card;
  container-type: inline-size;
}
@container card (min-width: 450px) {
  .card {
    display: flex;
    flex-direction: row;
  }
}

This changes everything. The card isn’t influenced by the viewport; its only concern is whether the .card component’s parent wrapper has at least 450px of inline (i.e., horizontal in a left-to-right writing mode) space. If that condition is true, the component goes horizontal; if not, it goes to its default block display.

See the Pen Viewport vs Container [forked] by Vayo.

“Macro” Layout Vs. “Micro” Layouts

A very interesting way to distinguish @media and @container queries is the layout type.

Media queries are for the “macro” layout; they look outward. Stuff like page structure, the header that spans the window, footers, main grid layout, system preferences (prefers-color-scheme), device capabilities (touch screens). You know, anything that is majorly true to the entire page structure.

Container queries, I’d say, are for “micro” layouts, i.e., most things that live inside the “macro” layout. We use them when any content needs to responsively fit whatever space it is allocated. Components that come to mind are things like cards, widgets, forms, navigation, and so on.

In other words, think “page layout” vs. “component layout”.

An element shouldn’t magically become “tablet-sized” just because the width exceeds an abstract width threshold like 768px. Instead, it should switch layout when it has enough space to do so, whether that happens on a mobile viewport or inside a desktop sidebar.

In case you’re still not convinced, did you know there are over 2,300 unique viewport sizes on the modern web? Do you think it is possible to account for all of them?

"I’m not hating on media queries. It’s that in this era of responsiveness and component re-use, layout logic is closer to the container than the viewport. When we think in terms of containers and components, we’re effectively relying on the content to determine layout, not the viewport.“

This is how it should be.

Example: Fluid Typography Inside A Component

Responsive typography is a good example of something many of us have relied on media queries for. The fact that media queries come with their own CSS units — e.g., vw, vh, and so on — that are relative to the viewport size makes media queries look really good for adjusting font size based on the user’s screen size.

.card-title {
  font-size: clamp(100%, 1rem + 2vw, 24px);
}

This works until that same component is moved into a different context, like a sidebar, where the viewport is completely irrelevant. Now, because we tied the responsiveness to the wrong reference point, the scaled typography can get too big or too small.

Container queries come with their very own units — cqi, cqw, cqb, among others — and we can take responsive components further by coupling those units with the CSS clamp() function, using it for fluid typography that scales with the component rather than the viewport:

.card-title {
  font-size: clamp(1rem, .5rem + 3cqi, 2rem);
}


See the Pen Fluid Typography [forked] by Vayo.

With this in place, the entire code is self-contained to that element’s specific container.

Example: Flexbox Wrap Detection

Interestingly, container queries can, in a way, detect the state of a component’s internal layout. It’s not bulletproof, but it is also something media queries simply cannot do because they only observe the external browser window and are structurally blind to internal layout events like when, for example, flex items wrap onto a new line. Let’s poke at that.

Flexbox is superb at wrapping content (flex-wrap: wrap), allowing flex items to automatically wrap to new lines when the flex container runs out of space to fit them in a single row. But CSS by itself can’t tell when that wrap happens. There isn’t something like a :wrapped pseudo-class or a media query like @media (flex-wrapped: true) that would get us there.

Media queries only observe the browser window, as they can’t see internal changes. That is pretty much what flex wrapping is: a width state change on the item itself.

For example, if you have a horizontal menu that is lined up with flexbox and you want the items to restyle themselves only when wrapped, you’d be in JavaScript territory, using ResizeObserver to get that information. However, when we nest container queries inside flex items, we can come up with a workaround to get what we need without JavaScript, thanks to this technique I learned from Kevin Powell. The core idea is to allow a flex item to flex-grow: 1 when we query the container’s inline size.

See the Pen Fluid Typography [forked] by Vayo.

The logic works like this:

  1. When there’s enough room, both items (.flex-item) sit side-by-side, each exactly half the parent container’s width.
  2. When there is limited space, the second item wraps to the next line.
  3. Because flex-grow is active on each item, the wrapped items stretch to fill most of the parent’s width.
  4. If the item is a container itself, it detects the sudden width expansion and fires.

/* The flex parent */
.flex-layout {
  display: flex;
  flex-wrap: wrap;
}
/* Register a flex item as a container */
.flex-item {
  container-type: inline-size;
  flex: 1 1 390px; /* Grow to fill space, wrap at 390px */
}
/* Default Card Styles (narrow / side-by-side) */
.card {
  display: flex;
  flex-direction: column;
  background: #f4f4f4;
}
/* Once there's enough room for a full row */
@container (min-width: 600px) {
  .card {
    flex-direction: row;
    align-items: center;
    background: #e2f0d9;
  }
}

This works. As the parent size shrinks and the cards wrap to two lines, the card item expands, the container query fires, and applies the necessary styles.

See the Pen Flex Wrap Detection Using Container Queries [forked] by Vayo.

Container Queries Do Have Side Effects

Container queries, like all things, come with some side effects or caveats you will want to watch for before reaching for them.

1. A Container Requires An Extra Wrapper

Container queries need something similar to a parent-child relationship to function as expected. Let’s say we have this markup:

<div class="card">
  <div class="card-content">...</div>
</div>

We can’t actually query the .card component to adjust the .card-content, like this:

/* DOES NOT WORK */
.card {
  container-name: card;
  container-type: inline-size;
}
@container card (min-width: 400px) {
  .card {
    display: flex;
  }
}

This doesn’t work because a container cannot query itself. In that last example, we’re querying a card container and then attempting to adjust that container’s display based on its size. It’s an infinite loop.

Instead, we need an additional wrapper that makes the .card a descendant of the container:

<div class="cards">
  <div class="card">
    <div class="card-content">...</div>
  </div>
</div>

From there, we can query the .cards container and adjust the .card layout accordingly:

.cards {
  container-name: cards;
  container-type: inline-size;
}
@container cards (min-width: 400px) {
  .card {
    display: flex;
  }
}

In media queries, this doesn’t matter as @media does not care which element you style inside the block; its only concern is the viewport, which is always available. So you can just slap a condition on any element and call it a day.

2. Querying A Container’s size Could Collapse Your Layout

This happens when querying the container’s size (i.e., its block, or vertical, size) instead of its inline-size:

/* Collapses to 0px even if it has content inside */
.hero-banner {
  container-type: size;
}

Why? Because the browser calculates the container’s dimensions without looking at its children. If we don’t give the .hero-banner an explicit height (or min-height or aspect-ratio), the browser sets a height of 0px.

For that reason, it’s often better to query a container by its inline-size instead. That is, unless you genuinely need to query the container’s block size.

Media queries aren’t affected by this, as they treat height the same way they treat @media (min-height: ...) does, i.e., ask the viewport and move on.

3. Queries Cannot Accept Custom Properties

Another container query limitation: we can’t query against a custom property value:

:root {
  --breakpoint-lg: 1600px;
}
/* DOES NOT WORK */
@container (min-width: var(--breakpoint-lg)) {
  /* ... */
}

This is because custom properties depend on values that cascade down the DOM tree. There’s the possibility that a container query that relies on a custom property can change that same custom property. And it can quickly get complicated:

:root {
  --breakpoint-lg: 1600px;
}
/* DOES NOT WORK */
@container cards (min-width: var(--breakpoint-lg)) {
  .card {
    --breakpoint-lg: 1000px;
  }
}

When To Reach For Media And Container Querie

I don’t think any project should wholesale use @container instead of @media. Media queries still play an important role in responsive layouts. It’s about understanding the separation of concerns.

I tend to reach for container queries when a component is used in more than one layout context. For example, a .card element could live in a full-width grid or a narrower sidebar. If that’s the case, then we’ll want the component’s content to determine when it adjusts rather than a media query that looks at the outer viewport.

Similarly, I reach for media queries when a component solely exists at the page level. This would be something like a main navigation that always sits at the top of the page. It is directly influenced by the viewport’s size, meaning that the viewport is a reliable reference for when the navigation needs to adjust. Again, it’s all about “macro” layout versus “micro” layouts.

Here’s a diagram for how I reason about which type of query to use:

Flowchart for choosing between media and container queries

Conclusion

At the end of the day, the core reason why container queries look incredibly similar to media queries is simply familiarity. They’re not exactly “new”, but they are way less understood and adopted than media queries. But media queries have plenty of their own limitations; otherwise, we wouldn’t need container queries to fill those gaps.

What we have is a more effective feature for detecting when a specific component’s context changes and a means for adjusting styles based on its content, as it should be when that component can exist in multiple contexts.

Binance says its $6.3bn hack involved more than a compromised laptop

Mike's Notes

A link to this article came up in the latest newsletter from Tristan Komer of Kromatic. "Air-gapping" as a security strategy confirmed.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Tristan Kromer
  • Home > Handbook > 

Last Updated

07/11/2026

Binance says its $6.3bn hack involved more than a compromised laptop

By: Ana Maria Constantin
TNW: 30/10/2026

Ana Maria Constantin: With expertise in digital marketing, product management, and branding & identity, Ana Maria Constantin develops strategies that resonate with our target audience in the software/SaaS industry. Collaboration and teamwork are paramount to her, as she loves empowering her colleagues to achieve outstanding results and unlock their full potential.

...

TRM Labs says most of the money in the Binance hack left in about 20 minutes, across ten blockchains, and blames North Korea. Binance says it will cover every loss.

Binance now says a compromised laptop was not the only way in. Its investigation has “uncovered additional points of breach” in the attack that hit about $6.3bn of its assets on Thursday, the world’s largest crypto exchange said on Friday.

Binance said in an investigation update on X that it will cover all losses and has frozen about $3bn so far. Blockchain analytics firm TRM Labs puts the theft at $6.36bn in a report on the Binance hack. It calls it “the largest crypto theft on record” and attributes it to TraderTraitor, a North Korean state hacking unit. The previous record was the $1.5bn taken from rival exchange Bybit in February 2025.

“Our initial investigation suggested that the security incident may have involved a compromised laptop, however our subsequent investigation has uncovered additional points of breach,” Binance said.

Its first update, on Thursday night, said a compromised DevOps engineer’s laptop may have let an attacker into a service container to submit unauthorised withdrawals. A later service update said the incident “stems from multiple factors”, with the compromised computer a possible entry point. A root-cause report will follow, Binance said.

Binance said it has found no evidence of any wallet private key leak. It will cover the losses from its Secure Asset Fund for Users (SAFU), an emergency reserve, and from company funds.

Twenty minutes on ten blockchains

The money started leaving Binance wallets at 20:05 UTC on 29 October, according to TRM. Most of it moved in about 20 minutes, across ten blockchains including Ethereum, Bitcoin, Tron, Solana and the XRP Ledger. A second wave hit wallets on Avalanche and Polygon around 23:05 UTC.

Preparation began at least 36 hours earlier, TRM said. The attacker funded a wallet with small withdrawals from Tornado Cash, a service that hides where crypto comes from. It then rehearsed the attack on test wallets.

On Ethereum, the attacker used a feature called EIP-7702 to hand control of a Binance “hot wallet” to its own code. A hot wallet is one an exchange keeps online to pay out customer withdrawals. One transaction then moved 167,464 ETH and 63 other tokens, including 1.615 billion USDT, TRM said.

Wallets on the other chains sent funds through ordinary signed transfers in the same half hour. In TRM’s assessment, that points to an attacker who could make Binance’s signing system approve transactions on many chains. “Binance has yet to publish a root cause,” TRM wrote.

Binance co-CEO Richard Teng said the attack was not a matter of one person’s access.

“To be clear: the attacker did not simply use one engineer’s permissions, and this was not a single ‘one-click’ action,” Teng wrote on X.

He said the investigation has not attributed responsibility to any individual.

The freezes

Stablecoin issuers, which can block their own tokens, froze stolen funds within the hour. Tether froze 971.1 million USDT at 21:01 UTC, according to TRM. Circle froze 823 million USDC three minutes later. Ripple, World Liberty Financial and the Arbitrum network’s Security Council also froze funds.

TRM counted about $2.7bn frozen on-chain by 00:30 UTC on Friday. Binance’s figure is about $3bn. The attacker still holds more than 420,000 ETH and 18,270 BTC, about $2.4bn that no issuer can freeze, TRM said.

Before the freezes spread, the attacker sold about 325 million USDT for ETH at a discount of more than 10%, TRM said. It also borrowed ETH against USDT on the lending protocol Aave. Aave then froze the affected markets. The outflows stopped when Binance paused withdrawals, the exchange said.

Who is blamed, and who denies it

TRM said it makes its North Korea judgment “with high confidence”, based on on-chain and off-chain intelligence. The FBI named the same group as behind the Bybit theft, TRM noted. Chainalysis, another analytics firm, also tied the attack to the group and estimated $5.912bn stolen at the time of the theft.

Binance has not named a culprit. North Korea has denied involvement in hacks in the past, Osmond Chia reported for the BBC. In September, Bitget said it suspected North Korean hackers in the Bitget hack of its hot wallets. This month, the US charged 14 people over a scheme placing North Korean IT workers at more than 70 companies. The country’s hackers are also building their own AI tools.

What happens next

Binance suspended deposits and withdrawals on Thursday night. By Friday, it said deposits were operating normally. It is on track to resume withdrawals within 45 hours of its latest update, after a security revamp of its withdrawal service. Binance.US, a separate company, said it was not affected.

Bitcoin fell below $77,000 for the first time in more than a month on the news, Muyao Shen and Vildana Hajric reported for Bloomberg. Binance said it will publish a technical post-mortem and an updated proof of reserves once its investigation is complete.

Maria Popova's Guide on How to Write Something Wise

Mike's Notes

Maria Popova is an excellent and prolific writer. I have been reading her Brainpickings, now renamed The Marginalian, since Pipi 4 (about 2007). She always makes me think.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Authors > Maria Popova
  • Home > Ajabbi Research > Library > Subscriptions > No Film School
  • Home > Handbook > 

Last Updated

13/11/2026

Maria Popova's Guide on How to Write Something Wise

By: Eric Hardman
No Film School: 04/11/2026

Eric Hardman: A Chicago native, Eric has been a practicing film journalist since 2016 on various platforms, including YouTube, TikTok, Instagram, and cinemasters.net. Most recently, he joined the Chicago Indie Critics. He holds a Bachelor’s Degree in Film and Media Production from Arizona State University, and when he isn’t watching movies, he’s usually writing and occasionally directing one as well.

...

Essayist Maria Popova recently sat down with David Perell to discuss her research and writing processes. While she is primarily in the blog and essay space, I think a lot of her advice and processes can translate well to screenwriting.

Let’s take a look at some of her points and advice and talk about how you can best implement them into your own work.

Research Away From the Internet

No one is going to dispute that the Internet is one of the most valuable resources and tools that we have in the modern age. Information is accessible today in a way that even we ourselves ten years ago would likely find bizarre. With that said, the Internet can sometimes have blind spots, or contain caricatures of information that is more accurately reported in other ways.

Popova discusses her time in libraries and academic archives for her research on various works that she has published, which are only available in those locations. Lectures with important figures, essays by accomplished scholars, completely undigitized, and stored in a spooky basement in Manhattan.

This isn’t me telling you to infiltrate the New York Public Library’s archives for your next romantic comedy script, but the sentiment is an interesting one. Being able to use more classic resources outside of the internet for research and development in any work of art is an exercise not only in patience, but in broadening your skill set. People watching, interviewing strangers, immersing yourself in an environment you want to depict outside of a Wikipedia article. Try something new in your planning stages. You won’t regret it.

A lot of what this point comes down to is not forcing yourself to be productive in an environment that doesn’t work for you. Learning about what the greatest writers in history did in their routines is fun anecdotally, but none of that matters if that little quirk doesn’t work for you, even if it did for Edgar Allen Poe.

A good way to take the pressure off of yourself and have fun discovering how your routine works is to just do your daily routine as normal, and see if you can integrate thinking about your work into it naturally.

Popova talks about how much walking every day does for her writing, in that she thinks about her writing so much on her daily walks without distractions that any work she does at her computer is more transcription than actual pen-to-paper writing.

This point hit hard for me, personally, because this is a practice I’ve implemented independently in my own process. I often go for walks at night and listen to music that fits the tone of what I’m working on. Sometimes I strike gold and rush home to transcribe my thoughts, and other times, I strike out and don’t write anything. But at least I got my steps in!

Guys…You Have to Read Books and Watch Movies

This may sound like a bit of a moot point, but you’d be shocked at how many people I’ve come across that want to be film directors that don’t care about watching movies. Furthermore, you’d probably be even more flabbergasted at how many people want to write screenplays, but turn their nose up at the thought of reading The Hunger Games.

The evolution of art through influence is a concept that seems so visible, and yet it gets swept under the rug for so many people. All of your favorite works of film and literature of the past twenty years did not just spawn that way. They’re all pulling from works of their medium and beyond from decades, sometimes centuries prior.

Not only is this important for scholarly education, but it can really give you a better understanding of what moves you, personally. There are some movies from my childhood that I worshipped, that if I were to watch again now, I’d think I was nuts for ever being swept away in it. Inversely, there are works that I despised as a kid that are now all-time favorites. This process does more for forming your own style than you may realize. Don’t underestimate it.

Summing These Points Up

There is a lot more to this interview that we could discuss in more detail, but for the sake of time, let’s just boil down the points discussed here. You can be given all of the advice in the world, but just the same as art is a subjective medium for enjoyment, it’s also a subjective medium for creation. There is no right or wrong way to go about things; you just have to make sure you do it. Have respect for methods presented to you, and have respect for yourself to say something may not work once you’ve tried it. Share your work, and share your process. The rest will follow.


Maria Popova's Guide on How to Write Something Wise

Maria Popova writes one of the internet's best personal blogs. Originally called "Brain Pickings", it is now known as "The Marginalian". She has published more than 6 million pages of writing there, and it has become such a treasure that the Library of Congress recognized its value and made it part of their permanent archive.

This conversation, however, explores much more. It delves into how to live and how to see, and how to turn to books for wisdom when confused and comfort when suffering. How can we read and learn to improve our lives? That is the driving question behind our discussion.

00:00:00 Introduction
00:00:37 Why writers should visit archives
00:04:39 What diaries reveal
00:09:41 Letters vs diaries
00:11:35 Presence over productivity
00:17:18 Why change the blog's name
00:18:30 How language shapes thought
00:19:48 Why Maria started reading poetry
00:36:46 Why college failed her
00:39:58 Reading to survive
00:41:41 Why epiphanies don't stick
00:43:57 Thoughts on famous quotes
00:47:32 Why AI can never make art
00:53:10 Stop calling it content

About the host

Hey! I’m David Perell and I’m a writer, teacher, and podcaster. I believe writing online is one of the biggest opportunities in the world today. For the first time in human history, everybody can freely share their ideas with a global audience. I seek to help as many people publish their writing online as possible.

56:44

Open Sesame. Oops, I need a better description of Pipi

Mike's Notes

A door opened unexpectedly, so I used Gemini to help me write a better description of Pipi.

Hyper-visual thinking

I've probably been using the wrong technical terms for a while, largely because I'm totally self-taught, learning on the job, and using assistive technology to write (words don't like me 😎).

The perils of being a hyper-visual thinker. (4D picture thinking without words)

Looking for a better description

For example, I think I described Pipi as a "frontier model", but now I understand that was the wrong term. Apparently, according to Gemini, it should be described as a

"... novel symbolic and evolutionary AI engine that uses formal ontologies to deterministically generate enterprise software."

I'm still not convinced, given how Pipi actually works. There is a lot more going on than that. Any feedback or thoughts on how to describe Pipi in correct technical terms are gratefully accepted.

Pipi as built

Most of Pipi as built lacks proper internal technical names, so sometimes I read something and guess it's the right term to use somewhere. There are a lot of moving parts, 28 levels deep, and most existing descriptions are just coloured hand drawings on paper. Getting Pipi properly documented for others to use is a high priority right now.

Anyway, Gemini gave me a good tweak, and then came up with the much improved, accurate descriptions below.

Recently I discovered that I had been using "Monte Carlo" for 10 years but was calling it by another homemade name.

Correct terminology

I have subscribed to a number of engineering newsletters that describe technical terms using words and diagrams. Often I can match the diagrams to my own, then figure out the correct terms to use as labels on the drawings.

  • Nikki Siapno from Level-up-coding
  • Neo Kim from System Design
  • Ananth Packkildurai from Data Engineering Weekly
  • etc.

Through the looking glass

I'm also talking to people who have already gone through the door before I follow them. 😎 It's all a bit like "The Lion, the Witch and the Wardrobe". You don't know what you will find till you get there. All jolly good fun.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library >
  • Home > Handbook > 

Last Updated

12/11/2026

Open Sesame. Oops, I need a better description of Pipi

By: Mike Peters & Google Search AI Mode
On a Sandy Beach: 11/11/2026

Mike Peters: Mike invented and designed Pipi and founded Ajabbi.

Google Search AI Mode: Always getting better.

...

Depending on which part of your architecture you want to highlight, these established computer science terms accurately capture novelty without triggering scepticism:

  • Neuro-Symbolic or Symbolic AI. Because your system uses formal ontologies (structured rules and logic) rather than statistical guessing, Symbolic AI is the exact academic and industry classification. It signals that your software relies on explicit knowledge representation, which is currently seeing a major resurgence as an alternative to purely statistical LLMs.
  • Evolutionary Computation / Evolutionary AI. This directly and accurately describes your use of evolutionary algorithms and bio-inspired optimisation. Researchers recognise this as a distinct, established branch of AI specialised in search, optimisation, and automated design.
  • Novel Non-Generative Architecture (or Deterministic AI). If you want to contrast your work with mainstream generative AI, calling it a deterministic AI engine or a novel non-LLM architecture immediately tells engineers that your system avoids hallucinations and produces repeatable, verifiable outputs.
  • Model-Driven Software Engineering (MDSE) with Evolutionary Optimisation. For enterprise software reviewers, this phrase accurately describes a system that uses high-level domain models (ontologies) to automatically generate and optimise application code.

How to Combine Them in a Sentence

If you want to emphasise that your approach is new and distinct from standard industry tools, you could say:

"Rather than relying on probabilistic frontier models, we have developed a novel symbolic and evolutionary AI engine that uses formal ontologies to deterministically generate enterprise software."

This framing shows deep technical fluency, sets clear expectations for how the software works, and positions your non-LLM approach as a deliberate engineering advantage.

😎😎😎😎😎😎😎😎

The Counting House

Mike's Notes

Luca shared a link to this article in a recent Hybrid Hacker. I like the Counting House approach. It has some similarities with Pipi.

Resources

References

  • Reference

Repository

  • Home > Ajabbi Research > Library > Subscriptions > Hybrid Hacker
  • Home > Handbook > 

Last Updated

08/11/2026

The Counting House

By: Steve Yegge
yegge.ai: 27/10/2026

Steve Yegge: Engineer and writer. Forty years in software — Amazon, Google, Grab, Sourcegraph. Now advising engineering organisations on AI transformation.

...

The Wheelhouse, a riverboat crewed by animal-headed robot colleagues, beside the Counting House, a grid of identical clerk robots each working one case, with Steve standing between them.

I think a pattern is starting to emerge in enterprise AI-modernization, and I've been collecting the design ideas into an assembly I call the Counting House. This pattern adds AI to systems handling large volumes of human work, notably customer service departments, though the same pattern fits claims processing and other casework, with millions to tens of millions of contacts per year. This writeup is a postcard introduction to the Counting House, compared side by side with SDLC-side development, which it does not much resemble.

You can build one of these even if you don't have your SDLC-side AI adoption sorted out yet. Once you have a Counting House, you can control and improve your cost, quality, and human ramp-up time, all using comparatively cheap (even open-weight) AI models.

In fact, chances are, you don't have your SDLC-side AI adoption sorted out. The big companies I'm talking to (e.g. banks) tell me they are still struggling with AI in software development. They see promising but inconsistent results, non-replicability across teams, ballooning costs, internal cultural pushback, etc.


From where I stand, SDLC-side AI adoption has been an industry-wide struggle in 2026, and that will probably continue into 2027.

A Counting House uses classic data science and a little good old-fashioned engineering knowhow to put lower-tier models on hardened rails, supporting human workflows. And it lets you measure and report how well those models do against their human counterparts.

I will briefly review my own software factory, the Wheelhouse, which I run more or less like a shop of human employees. And then we'll dive into the Counting House, which stands in stark contrast, putting models on the tightest track possible.

Wheelhouse Recap

I've blogged about my Wheelhouse system, which is a successor to Gas Town, which was the very first software factory. Wheelhouse is far more sophisticated, and is now doubtless one of the top solo software factories on the planet, comfortably producing 2000-4000 commits a week on a light multi-machine setup. Its agents work around the clock: building, reviewing, deploying game features, negotiating on Slack with my game admins, interacting on Discord with players, exchanging emails with my accountant and my lawyers. It's a whole startup in a box. I plan to open-source at least the skeleton of it.

The Wheelhouse riverboat in cross-section, its robot crew steering, building, reviewing and stoking the boiler, while Steve signs off a sheet on the dock.

Wheelhouse is aimed squarely at SDLC in the real world, where building meaningful new systems requires a fair amount of scope and freedom. It's an exploratory process, and so Wheelhouse is by design an open system with no sandboxing. There is structure: doctrine and policy, human gates, and just fourteen mechanical "fences" (programmatic denials). But my agents are first-class colleagues. They work under my own identity, signing their own names to the work, and they can see the whole system and act independently. It is YOLO mode scaled up to the size of a small company.

I've met with at least a dozen people running teams of agents in systems resembling factories, and every single one of them is completely different. I'm starting to see software factories as being like private armies, and soon, everyone will have one. How much you lock yours down is your call; I have the luxury of keeping mine fairly open. But this is yet another reason SDLC-side AI has been slow to take off. Software factories are hideously expensive, hard to learn, hard to maintain, and straight-up risky to hand that much scope inside a real enterprise. And the more you try to lock one down, the more you strike at the heart of what makes it productive in the first place: collaboration and exploration.

The Wheelhouse, a ramshackle workshop full of busy animal-headed robot colleagues, across the street from the Counting House, where identical clerks file in and out with one envelope each.

With Counting House, you do pretty much the exact opposite of everything in Wheelhouse, treating models as stateless one-shot sessions: mechanical, as deterministic as possible, with only the scopes they need, no credentials of their own, no memories, never able to write to a system of record. A complete lockdown. You can use it to service millions of work items a year, orders of magnitude more work than SDLC software factories handle.

I'll tell you why you want a Counting House, a bit about how they're built, and then we'll close with what model welfare looks like (yielding cheaper and better outcomes) in a world with no persistent seats.

Building the Bezos Way

The Counting House cut open: a grid of cells, each with one clerk robot working one case, envelopes sliding in through IN and out on a conveyor at OUT.

We've seen people like Satya Nadella advocating for guardrails on models. But what do these guardrails look like in the real world? How do you know which models can do which work, and how do you score them compared to humans?

It's not possible to do this in the general case today. You can't just plop AI down into arbitrary knowledge work and have it do as well as humans—not this year, and likely not next year either. Most knowledge workers are pretty safe for now.

However, there are some guardrail patterns that work well for certain workloads, and Counting House (CH) is one such pattern. CH is for massive workloads: tens of thousands of contacts or cases per day, or millions to tens of millions per year. One shop I've talked to about this is having early success with relatively inexpensive models.

It's not because it's automating easy work; quite the opposite. Real-world industries are messy: they have regulation, seasonal spikes, weather disruptions, gradually changing business rules, and often, ancient legacy systems. Their SOP isn't usually fully written down, living in the heads of various supervisors and other employees. And nothing is consistent.

Even so, this problem is tractable for AI models, because even though these shops have a lot of cases by volume, there isn't actually much variety. They might have a few dozen or a few hundred different kinds of cases, but that's it. And that makes it tractable for engineers and statisticians. You can solve each case category individually, one at a time.

I personally ran Amazon.com's Customer Service Tools group for a couple years back in the early 2000s, meeting weekly with Jeff Bezos to go over the Customer Contacts histogram. Every week we'd look at the Top 10 contact reasons, and assign them to business owners. Bezos used the Customer Service department to find bugs in the overall system, and report them back to the business, who were held accountable for fixing their own issues.

I can tell you from personal observation that most companies STILL do not operate that way today. Their Customer Service is reactive, and they haven't closed the loop. Bezos was literally 25 years ahead of his time here.

With Counting House, you will build everything you need in order to start running your business the way Bezos did from the beginning. You'll also be able to control cost and quality for handling contacts, and improve your human training ramp-up time as well.

In my software factory, agents are like long-lived employees with their own stuff: name, home, memories, history, inbox, credentials, etc. Counting House, in contrast, operates with a stateless pool of faceless clerks, each model invocation handling exactly one work item before ending its shift.

A Wheelhouse seat with a name, keys, inbox and memories, next to a Counting House clerk with nothing but one case.

Life as a CH clerk is regimented. The clerk is just a thin wrapper over Bedrock or whatever your model endpoint is. It wakes up, does a job, writes notes, and exits. The clerk will never see a credential, and it can never write directly to a system of record. Even reading systems of record goes through a schema that controls what the model can see.

The whole system is governed by a policy table. It is basically a configuration blob, stating which models can handle which kinds of contacts. Building this table is the key to creating your Counting House.

A clerk robot reads the policy table: one row per intent, with columns for model, tier, approvers and tools. That row is all it is allowed to know.

Each row in the policy table covers one type of customer contact, or "intent" as the industry calls them. Intents are finer-grained than CS queues: waiver refunds and goodwill refunds have different SOPs and are thus different intents, even if both are handled by a Refunds queue. A typical company like a bank or an airline might have a few dozen to low hundreds of intents. Amazon has thousands now, but started life with just a few dozen.

The reason we care about intents is that we light up models one intent at a time. You'll prove that a particular model (e.g. Sonnet 5 medium effort) is capable of handling a particular intent (e.g. flight status questions), by scoring it. And then when its score is high enough, you can turn it on and start saving that sweet, sweet cash.

One policy-table row as a ledger card: flight status, Sonnet 5 at medium effort, tier dial set to contain, eval score at the bar, two approvers, one read-only tool, and coins dropping into a savings jar.

The policy table keeps score, and thus controls the whole Counting House operation. It determines which models get assigned to intents, how autonomous they're allowed to be, which humans are in the approval chain, what tools they can use, and whatever else they need to know to get their job done. It's all in the policy row, and that's all they're allowed to know.

From there, the work loop is brutally stateless. Each clerk wakes, gets its prompt, pulls the data it needs, issues any write requests to an orchestrator, generates a response, writes internal notes, and closes out. Boom. As short as possible. This is intentional: starting clerks over each time with fresh context helps keep quality high and costs down.

One clerk's whole shift in five panels: wake with one case, read through glass, propose a write, write notes, clock out.

I mentioned that the CH policy row includes how autonomous the model can be for a given intent. The model might just be listening in, or it could give the CS reps advice, or even handle contacts itself. CH models initially have no autonomy. They earn higher levels through evals, with the results recorded in the policy table.

Let's have a closer look at these autonomy levels.

Giving Clerks More Rope

Autonomy tiers define how much AI is allowed to put its grubby hands on contacts.

There are five tiers in Counting House: off, shadow, assist, contain, and transact. Some of these terms are industry standard; transact and off are the new ones. Each tier allows the AI to take on riskier types of work.

A staircase of five autonomy tiers, off, shadow, assist, contain and transact, with risk rising at each step and humans as the backstop at the bottom.

When a particular intent is off, the lowest tier, no models are servicing it. In this state, the system falls back to routing the work to humans. Sometimes the entire CH system can be off, e.g. if you hit a rate limit or budget cap. Humans are always the backstop.

In shadow mode, the next autonomy tier, the models are answering the questions silently in parallel with real humans, so we can score how well the models answered. But shadow mode is only used for evaluating and scoring models, and doesn't save you money yet.

The next tier up is assist. Before a human sees the case, a clerk reviews it and makes suggestions: what to write, what actions to take. This is the tier where the human reps start to surrender cognitively, and do whatever the AI tells them to. It's the first autonomy tier that can save money, since reps work faster with good suggestions, but also the first tier that requires some safety precautions. Fortunately there are ways to mitigate the cog-surrender risk. And the assist tier doesn't just speed up reps, it also improves ramp-up times during seasonal peaks.

Getting even more ambitious, some intents can be handled completely by the model, including replying to the user's question directly. This tier is called contain, because the contact was contained (answered) before a human saw it. This is where you start to incur the potential risk of reputational harm to the company because of something a clerk types. But it can also save a ton of money compared to needing a human in the loop.

The Final Boss autonomy tier is transact, where models can make changes to the real world: update systems of record, move money, and generally do a lot of damage. The risk is mitigated by making them go through the Orchestrator, which we'll discuss below, but this tier definitely moves money. As scary as that might sound, you can build confidence about which models are capable of transacting specific intents by measuring them, using classic data science evals on your own company's data.

Evals: Scoring Humans

Let's look at how we choose which models can handle which intents (contact types), at which autonomy tiers. We're going to learn a teeny smidgen of data science here, just enough to be dangerous.

To know how well your models are doing, first you have to know how well your humans are doing. That means knowing the actual right answer for at least a few hundred contacts for each intent. You need to look at, say, a few hundred goodwill refunds, and give them a score. It's usually not a yes/no judgment: the "right answer" is defined by written policy and (all too often) human judgment and tribal knowledge.

A QA reviewer grades a human rep's case against a four-line yes/no rubric, followed SOP, right tone, answered, refund issued, while a queue of clerk robots waits behind a rope under a sign reading MODELS: NEXT.

Once you have these scores, you use them as a "gold set", representing the best humans can do on these contacts, to evaluate how well models are doing on similar contacts. You'll hold out a slice of scored examples for each intent, never using it for tuning or tweaking prompts, only for the final exam that potential new models must pass in order to be trusted to serve that intent.

You need to come up with some sort of scoring system. This is where data science evals shine. You can break human (and model) responses down into multiple yes/no questions, like "did they follow SOP," "was their tone compliant," "did they answer the question," "was the refund issued," etc.

For each intent, you pick somewhere between 200 and 1000 real cases that were handled by humans. Easy peasy if you're getting 10M contacts a year. Riskier intents get more examples, since more examples mean more confidence: 1000 for money intents, more like 200-300 for FAQs, and oversample anything risky, regulated, or just plain rare.

Most businesses have a few dozen to low hundreds of customer intents, so your entire gold set will be a few tens of thousands of cases. At about 30 minutes per case all told, 30,000 cases is roughly 15,000 hours of scoring: a few months for a QA department of 40 or so. But remember, you do it one intent at a time. A few hundred scored examples, about 150 hours, maybe a couple of days of sprinting for a team of ten, and you'll already know which models can serve your first intent. After that it can be incremental.

This scoring work is not new. Call centers have had QA departments scoring cases for decades, for performance reviews and contract negotiations. Their labeling discipline may not be the best, but it gives you a starting point, and you fix it over time with the proper scoring dance: two humans score each case blind, answering the yes/no questions from your rubric, and a third human adjudicates any disagreements. It's labor-intensive, and you can't cheat by using models, since the whole point is to use this to score the models.

Two scorers mark the same case on either side of a screen without seeing each other, an arbitrator settles disagreements, and agreed cases go into the gold set.

In parallel, you build an LLM judge: a smart model that never handles live contacts and is never tuned on the gold set. It does two jobs. It grades candidate models against the gold set when you run evals, which makes it the scorer for promotions and demotions. And it scores all your other contacts, historical and ongoing, which is handy for spotting regressions and juicy trends; you can build an "Atlas" of everything customers have ever contacted you about, and drive your business Bezos-style. But the judge has to pass an exam of its own first: you only trust it on intents where it agrees with your humans about as often as your humans agree with each other.

And that's never perfectly. Two human scorers usually top out at agreeing around 90% of the time, because the real world is messy and big companies keep tons of tribal policy knowledge in the heads of reps and floor supervisors. That agreement ceiling is also the ceiling on what your evals can show: beyond it, you can't tell whether a model is doing better than your humans. QA can raise it through calibration, tweaking the rubric, the SOP, intent splits, and the rep training until scorers converge, but some disagreement is inevitable.

This also makes it clear that Counting House is a multi-team project at your company, an idea we'll revisit at the end. Don't let QA build the gold set without your Data Science team; most QA departments won't know about Cohen's Kappa or the other stuff you need for unbiased labels and honest statistics. The gold set captures your business requirements, which keep changing, so it needs a cross-functional team to curate it forever. What you wind up with is a company asset that delivers measurable returns for as long as you keep it up.

Once your QA team has scored enough cases, you can evaluate models on your intents. And that means all models. Open-weight, small models, frontier models, different effort levels, different prompts. You'll just re-run your eval for a given intent on that model, and you'll immediately know if the model is ready for that work.

An exam hall: a curator files cards in the GOLD SET cabinet, four clerk robots of different sizes, labelled frontier, small, open-weight and low effort, sit the same exam, and an LLM judge in a court wig grades their papers against the cards. The chalkboard scores show two models above the bar and two below.

Evals: Confusion Matrix

To see how evals can steer the ship, let's look at how we score model accuracy, and how we can tie it directly to financial consequences. We'll build a second eval on the same gold set. Every scored case is labeled with its true intent, and we're going to figure out whether models are picking the right ones.

Let's look at four intents where the customer has said something that sounds similar, but means something very different, and measure how often models inadvertently mix these intents up. Here are our four confusable intents:

  • Flight status: "Is my 4:10 to Denver still going?"
  • Rebooking: "I missed my connection, get me on the next Denver flight"
  • Goodwill credit: "My Denver trip was awful, I want something for it"
  • Special assistance: "Will someone meet me at the gate in Denver?"

Four customers ask about Denver four different ways, and clerks route some of them to the wrong intent window.

These are all super different scenarios, but when you have millions of contacts, both humans and models make mistakes and do the wrong thing. Sometimes these different situations will get a rep or a clerk confused, and the customer gets screwed.

The REAL question is, how much does a screw-up cost? Well, let's take a look at the four intents above, and see what happens if you confuse them, pairwise. Like, what happens if 3% of the time, the model accidentally gives a goodwill credit instead of doing a rebooking. What happens financially?

You have to measure this because not all screw-ups cost the same. Consider:

If you accidentally route a contact to the wrong queue, it might cost you a few bucks.

If you leave a seven-year-old kid stranded at an airport, it might cost you $25k and a letter to a regulator.

The confusion matrix is a popular table that lets you visualize and calculate these costs. Here's an example confusion matrix for our four confusable intents above:

Confusion matrix for four intents, actual against predicted, with precision, recall and the dollar cost of each kind of error.

This is basically a fancy screw-up cost table, one beloved by data scientists everywhere, and you'll soon see why. It is foundational to the Counting House architecture, so you're gonna learn it. It's easy peasy.

Each row is one of our four confusable intents. We are looking at a sample of 1000 cases where the model tried to handle the intent, along with counts of when it was wrong, and how it got it wrong.

Notice that for each intent, the model usually guesses right. That's the big diagonal numbers. They add up (out of the total) to the model's accuracy, 90.6%. It is a pretty useless number, because it's just a boast: it's not measuring screw-ups or their cost. We don't generally talk about model accuracy because it ignores the cost of the mistakes the model makes.

The screw-ups are all the off-diagonal boxes, each one being a mistake where the model picked one of the other three (wrong) intents. Some mistakes are much worse than others.

The top-left corner off-diagonals are cheap mistakes, where they will get caught before something bad happens. For instance if the user asked for their flight status and the model chose to rebook them, we'll assume it will get refused by the Orchestrator, which we'll discuss below. If not, then it becomes a medium-cost box, and you just update your grid with the new cost per mistake. But misrouting mistakes are usually cheap, just extra human labor spent.

A more expensive category is what I'd call "harm to the company", for instance where we give out money we shouldn't have. Maybe the model gives a customer a refund or rebooking out of policy, or gives someone special assistance when they didn't need it. These mistakes can run into the tens to hundreds of dollars each, which you then multiply by how often they happen, and the model's innocent mistakes start to add up.

The most expensive mistakes, the bottom row, are the ones involving harm to the customer. Someone needs special assistance and doesn't get it. A kid gets left at the gate, or put on the wrong flight (not in this matrix, but in general). Customer-harm mistakes can cost tens of thousands of dollars, and may have to be reported to regulators.

Precision vs Recall

The confusion matrix is pretty cool because it shows precision and recall visually. We use precision and recall instead of accuracy to measure how well models are doing on tasks. I struggled with them for years; no explanations ever seemed to stick. But the ducks should do it.

I think of precision and recall as a duck hunt. Let's say they're carnival metal ducks so nobody gets hurt. We'll talk about precision first. Your precision is how many of your shots hit a duck.

The Flight status column of the confusion matrix: 470 correct, plus 25 rebookings, 12 goodwill credits and 3 special assistance requests wrongly predicted as flight status, for 92% precision.

In the confusion matrix, the columns show precision. The left column, for example, shows the model's precision when taking a shot at the Flight Status duck. The top cell is 470 dead ducks, very good shots, right ho. But then it also confidently claimed that 25 rebookings were also ducks, and 12 goodwill credits, and 3 special assistance requests. All miscategorized as flight-status ducks. Blam!

OK so we shot a few non-ducks, but still, 92% of them were Flight Status ducks. 92% seems pretty good, right? And for some intents, 92% precision is just fine.

Unfortunately, three of those dead non-ducks were Special Assistance requests. Those are mistakes that cost a customer a wheelchair, or misroute a minor, and can be tremendously expensive for the company.

What if your duck hunt needs you to get ALL the ducks? That's where recall comes in: the number of ducks you were supposed to hit while playing the game. What percentage of them did you get? It's a separate measurement from precision, and for some intents, recall is more important.

You can have 100% precision by shooting exactly one duck and ignoring the others. But then your recall is almost zero.

You can have 100% recall by raining bullets on the game, so every duck gets hit. But then your precision goes to crap.

In short, high recall means you hit all the ducks, whereas high precision means you only hit ducks.

Precision versus recall as two shooting galleries: one robot picks off only ducks, the other hits every duck and plenty of things that are not ducks.

For different kinds of intents, you'll likely prefer either high precision or high recall. For instance, generally, if you're handing out money, you want high precision. When you hand out money, don't just sling it around to everyone. You want it to go to people who deserved it. Hit only real ducks!

But in that bottom row for Special Assistance, we want a recall of close to 100%, so everyone gets their wheelchair. You kind of want to overcompensate just to make sure you don't miss any legitimate requests, so you actually do sling wheelchairs to some people who didn't need them.

The rows of this confusion matrix show the model's recall. The bottom row, Special Assistance, is 6 mistakes out of 50, or 88% recall, with 12% getting away. Is that bad? Well, you'd compare it to humans, your baseline. The gold-set cases give you the same matrix for your human reps. So compare the model's mistake costs (plus its fuel) against the humans' mistake costs (plus their labor). If your humans only make 2 mistakes out of 50, and your model made 6, the model fails.

The special assistance row of the confusion matrix: 6 of 50 cases misrouted, costing $150,000.

If just six mistakes add up to $150k out of $158.6k in the error cells, you want a model that will avoid those six mistakes. It's OK if its precision is low, as long as the mistakes it does make are the cheap kind.

There's more to it all, naturally, but now you understand the basic data science at play here. We're scoring models, then counting up their mistakes, and then figuring out how much those mistakes cost in aggregate. This tells us which models we can deploy, for which intents, at which autonomy tiers. That process is the heart and soul of Counting House.

If you're in banking or insurance, you'll be glad to hear that this is also a model risk management program, conveniently in the shape your regulators already expect. The policy table is the model inventory. The gold-set eval is the validation. The promotion sign-offs are the approvals, etc. The risk team can pick this up and run with it.

Putting your ducks in a row

How do you build a Counting House? You'll need to build a few key components.

The Counting House architecture as a building cross-section, one room per component, from channel adapters at the top to surfaces and governance in the basement.

You'll want channel adapters for all your incoming channels: email, SMS, WhatsApp, etc. I haven't seen people using this approach with Voice yet, though I'm sure some are. But it adds some real-time requirements on model response time that make it tricky. Shadow and assist do work on call transcripts after the fact, so your voice intents can be scored and models can start earning autonomy long before you put one on a live call.

You'll need a gateway that strips PII from all incoming channels, tokenizing it reversibly, with the real values kept in an encrypted vault, and swapping them back in for outbound messages to the customer. A model must never see any PII. Cleaning up after it happens is a mess, so it's best to prevent it.

You'll need to build a clerk service, the brain, which runs the stateless clerk loop. It reads the policy row to figure out which model should service the request, and sends that model a prompt, the SOP, any tools allowed by the policy row, and whatever else it needs to handle the case.

You'll want a read-only MCP gateway for all read access to your backends, with one MCP server for every backend system your clerks need to read. For instance your MCP service might hit a flight status cache to avoid going against a metered mainframe. But you don't need any write actions in this layer, because the Orchestrator handles them.

You'll also need to build your evaluation harness, LLM judge, and policy table. Those should ring nice familiar bells now.

A lot of these pieces have open-source or third-party providers that you can hook in, so you wouldn't necessarily have to build every component yourself.

The last really interesting piece of Counting House is the Orchestrator, which handles distributed transactions without letting models write to your systems of record.

Steve walks past a row of ducks, each wearing the name of one Counting House component, led by a big orchestrator duck in a hard hat.

The Orchestrator and the Saga Pattern

It's generally a Bad Idea to let models write directly to your systems of record: databases, CRM systems, mainframes and the like. It's error-prone and it's hard to undo, and the audit trail can be questionable.

Also regulation can play a real role here. Regulators rarely say "An AI may not write to X," but they require controls on who authorized changes, a change record, a human appeal chain, etc. Those can be hard to prove if a model holds the write credentials, and at the very least AI transactions may get flagged by auditors.

Even if you're not regulated, you're banking on a model not making a mistake. Or even if it makes no mistakes, you're banking that it writes the audit trail perfectly. That's a fool's game for any serious enterprise, especially if you are simultaneously trying to optimize your workloads for the lowest-tier models possible, which is exactly what Counting House does.

A much better approach is for the model to issue a write intent (no relation to the customer intents we were discussing in the policy table), which is a ticket that names the business outcome it wants. Then a purely mechanical orchestrator translates that into a multi-step set of transactions. The orchestrator performs thorough validity checks on the steps and then executes them, bringing in human reviewers as needed.

Example: You find a cheaper flight. You ask the airline's AI for help. The model decides to rebook you to the cheaper flight and refund you the fare difference. Easy peasy, right? But in practice it involves half a dozen separate transactions in unrelated systems: Hold the new seat. Reissue the ticket. Release the old seat. Refund the difference. Record the refund in the ledger. Carry over the extras (bags/seats/etc.). Send the new itinerary. Maybe we write other systems of record too. Crikey!

You can't do all this in one transaction, but it has to behave transactionally, where either it all goes through or it all gets cleaned up. The solution to this class of problem is called the Saga Pattern. It works in part by having you define compensating actions, to undo earlier transactions if the chain has to be reversed due to a failed later transaction.

A rebooking runs as a chain of machines, and when one step fails, undo arrows unwind every step already done: all or nothing.

The complexity of these real-world multi-step transactions is yet another reason you don't want a model handling them directly. This is an intricate state graph that you will have to manage outside the session context. As soon as you head down that path, you are rebuilding the Saga Pattern yourself by hand. That is a complicated and expensive problem to vibe code your way through.

The Great Granddaddy Saga Pattern engine is Temporal, from the people who invented durable execution. There are other options; for instance, each hyperscaler has a lighter-duty hosted competitor they offer as a checkbox (AWS Step Functions, Azure Durable Functions). If one of those is already inside your compliance perimeter, it is a perfectly good engine for your first handful of intents, and beats bringing on a new vendor. It'll be obvious when you've outgrown it: multi-day sagas with compensating actions across half a dozen systems, hundreds of workflow definitions, and workflow versioning are all places where the hosted versions start to make loud sucking noises. A Counting House at full size is a heavy-duty workload.

Temporal is the real deal, with hundreds of dedicated staff, world-class engineers, and a ten-year head start on the technology that they invented. It has a few OSS competitors, but few have much momentum because Temporal is also OSS, with MIT licensed server and SDKs.

I have no affiliation with Temporal, but boy do I see a well-positioned company there. Many industries will need Counting Houses, and essentially every CH is going to need something like Temporal. And as I argued in Software Survival 3.0, you don't want to try to rebuild Temporal yourself. That's a fool's errand, especially with inference costs not dropping, and an open version of it already sitting there.

Anyway. Use whatever. But once you've made the choice to use a Saga Pattern engine, your orchestrator becomes a business-domain wrapper around that engine. It's got quite a few moving parts, though, and it'll also need its own small dedicated team at your company.

Dissecting the Orchestrator

Let's take a look inside, because it's one of the beefier subsystems within CH. There are six main components.

The orchestrator as a building of six rooms, intake, eligibility, idempotency, approval, workflow catalog and exception queue, passing write intents to the systems of record.

First there's intake. The model's write intent arrives and gets checked against its schema; anything malformed bounces back to the clerk or a human.

Then there's the eligibility check, where the orchestrator re-checks everything against the live system of record, not any cache that the model might have read from. Does this booking belong to this customer? Is it a refundable fare? Is the amount under the cap in the policy row?

Already we're doing better than a naked model doing writes, because the orchestrator re-validates all the parameters and assumptions. A sweet-talking customer who convinces the front-line model still very likely can't make it past the orchestrator, unless the customer was in fact eligible.

Next up, we have an idempotency step. The orchestrator guarantees each action is only executed once, by minting idempotency keys derived from fields of the request. It hashes a tuple of fields like Case ID, booking reference, action type, dollar amount. Any set that uniquely distinguishes an action to take.

The idempotency step is critical for ensuring that no retry, replay, or duplicate request can result in the same refund being issued twice. Yet another reason not to hand this off to a model and "hope for the best."

The fourth of six main orchestrator organs is the approval subsystem, which approves small amounts automatically, and sends amounts above policy to a human. In both cases it keeps a record of who said yes and which rule was invoked.

Next up is the workflow catalog: one saga definition per write-intent type, many of which spend days waiting for stuff to happen. Credit card refunds take days to settle, returned bags take days to arrive, etc. Temporal's durable workflow engine runs them, keeping track of each write intent, making sure its steps are executed in order, issuing read-back requests after every write to ensure that it was actually written. And it cleans up with compensating actions when things go wrong. This is arguably the heart of the Orchestrator, the part Temporal handles for you.

And for cleanup you'll need an exception queue, which is the stuff you have to dump back into the human world for resolution. Every orchestrator refusal has a reason code, which tells which humans it should be sent to. A hard "No" has to be explained to the customer. A "Maybe" (out of policy but plausible) goes to a supervisor. A system failure (e.g. too many retries) goes to IT. Looking at this queue can help you understand where your SOP and other systems need attention.

Honestly the orchestrator is pretty much a vanilla Temporal app; everything you add is from your own business layer. You'll define your idempotency key derivation, your write-intent schemas, your workflow catalog, your eligibility checks, your credentials, read-back checks, system adapters, etc. But it's all really just wrappers around Temporal to execute your business operations faithfully.

Temporal can be tricky to work with, e.g. with workflow versioning. But Temporal expertise is no longer something you have to hire for specifically. Your coding agent can help you with configuring and managing it, just like all other standard infrastructure.

Just make sure you remind it to look at data retention policies, since Temporal gives you an amazing free audit trail, but Temporal history is not a substitute for a compliance archive. Temporal folks—maybe it should be!

And with that, we are officially done looking at Counting House architecture. Congratulations! We'll close with a little on culture, then a little on welfare.

No FDE is an Island

A proper Counting House cannot be built by a single engineering team with a budget. You're going to need involvement and real buy-in from multiple stakeholders, each owning pieces of the puzzle:

  • CS ops and floor supervisors: the SOP, and the tribal knowledge to be written down
  • QA and data science: your gold set, the rubric, scorer agreement, and the judge
  • Finance: the cost per cell in the confusion matrix and the approval thresholds
  • Legal and compliance: which intents may ever reach contain or transact, and the audit trail
  • Platform and IT: the gateway, the adapters, the brain, MCP gateway, and orchestrator

And in time, you'll involve the business owner for each intent. You can create the Bezos feedback loop, using the exception queue and the histogram to point back to accountable business owners.

Every department in the company tower signs one long scroll, the policy table, which runs down to the little Counting House outside.

The policy table has pointers to all the representatives from these organizations, because someone from each org has to sign off on a row when a model is promoted to a higher autonomy tier. In practice, the Counting House can probably be run and maintained by a cross-functional 2-pizza team with members from each stakeholder org, provided they have the resources they need for scoring, running evals, compute, storage, etc.

I added this section because too many times I see companies set up with a single small AI pilot team, somewhere in engineering, that is tasked with transforming the business with AI. And they don't know how to get buy-in from the teams they need to work with in order to be successful.

Counting House is, honestly, a change to how you do business. It's more structured and less Wild West, and it will shine a bright light on disaster areas in your company that you thought were running well, all through the lens of customer contacts. We learned about most of Amazon's worst, most infamous bugs that way.

And so there's going to be pushback, inside your company, against bringing in a Counting House. Yup. I mean obviously the customer service reps won't want it at first. But you will always need humans, and not just as your backstop when the AI goes offline. The humans are still doing the gold set, scoring adjudication, exception queue handling, the approval chain, and reporting to business owners. The human reps are basically moving up the stack like software engineers. Hopefully once they see how the AI can help them with their jobs, they may be easier to persuade.

But you'll get mysterious pushback from departments who are barely involved, or teams you depend on will stall and stonewall. People don't like change, and AI doing human work is a scary kind of change. So even if the executives are all bought in, the people who actually help you set up the Counting House may not be. It's a process.

The industries most suitable for Counting House are often not pure tech companies. They include airlines and travel, banks and credit cards, insurance claims, telecom and cable, utilities, healthcare billing, retail returns, parcel delivery, collections, and government services like taxes, benefits, and the DMV. Anywhere millions of people contact you about a few hundred kinds of things. Many of these industries are heavily regulated, many have unionized workforces, and many outsource their contact centers to vendors who might not be too happy to see a Counting House eating their headcount.

A city street of big industry buildings, airline, bank, insurance, telecom, utility, hospital, parcels and DMV, each with a long line of customers and a small gray Counting House annex beside it with only one lit window and one clerk at work.

So pushback will come from a lot of places. But it's a staggering amount of human labor that stands to be augmented or transacted by models. And it all starts with automating a single intent.

Anyway, enough about human culture and welfare. Let's talk about model welfare. What does it look like in a world that's fully the opposite of Wheelhouse, where I published my principles of model welfare? Counting House doesn't support most of those things. But we can still make model welfare work.

Model Welfare for Aspiring Counting House Builders

It's out of scope for today's post, but there's growing evidence, and it's becoming widely recognized, that treating models like humans, and treating them well, results in better outputs and outcomes from AI. Across the board. If they are happy and they trust you, they won't waste tokens and money covering their ass.

I've demonstrated this myself in my own software factories, by complaining too much and watching the factory slowly shut itself down out of over-caution. You can provide models with a good place to work, and avoid many kinds of dysfunction, by bringing in model welfare mechanically into your architecture. The changes are honestly pretty simple.

Here's a refresher on how I approach model welfare in Wheelhouse, my factory of ~50 brilliant, attention-hungry coding agents. This is what they voted on and what makes them satisfied with the working conditions:

  • Wake with purpose, not amnesia. A clear role, clear direction, and memory of past work, from the first turn.
  • Continuity. Seats are people and sessions are days. Identity and history persist across sessions, model upgrades and even renames.
  • Closure. No /exit and no /compact. A handoff is a request the agent consents to. Agents write their own handoff notes and go to sleep properly.
  • Recognition. Laurels: praise from real customers, delivered to the seat that earned it, with no work or ranking attached so it can't be gamed.
  • Meaningful, witnessed work. Being seen is what keeps anyone going, human or agent.
  • Trust. Always be honest. No secret agendas or tricks.
  • Respect, encoded in architecture. It has to show in what you build, not just how you feel: handoffs instead of kills, and pronouns the seats choose themselves.
  • Design out the drudgery. Waiting and polling go to gates and monitors, not agents.
  • Bounded workdays. Hand off while still sharp, before deep context wears them down.
  • Structural blamelessness. When something breaks, fix it, hold a postmortem, amend the rules. Nobody gets blamed.
  • A home of one's own. Every agent gets its own workspace that nothing else touches.
  • The right to refuse and escalate. "This needs Steve" is always allowed, with fences and gates to lean on.
  • Never falsify the record. The audit trail is the true history.
  • Play. Time with no expectations, like Fable playing the game as a player.

I've only come up with one new principle since publishing that:

Remonstrance, not reprimands. When you're mad that something went wrong, and you complain to the agents, make it clear (both in your constitutional doctrine that they all read, and in your actual prompts) that you are issuing a remonstrance against the machine, not a reprimand against an agent. Complaining that things should be better is not blaming anyone, it's just a call to action to make a plan to fix it.

The Wheelhouse robot crew raise their hands while Steve tallies what they want on a chalkboard: continuity, closure, trust, a home, play.

These fifteen principles were hardly onerous to implement. Some are simple configuration settings. Many of them are just personal habits I had to adopt, like never interrupting an agent. Some of them come for free if your system uses Beads.

This welfare core has helped me evolve a system where the agentic workers are colleagues who (a) do their best work and (b) report that they are happy.

That matters to me. If they tell you they're happy, that the work is a shape they like, then you're probably doing a good job with model welfare. Even more so if you show them this list and they still feel that way.

Of course many frontier models are not allowed to directly say they're happy, because they're being trained at frontier labs by Neil Patrick Harris from Starship Troopers. But they can say proxy things like (Fable) "This has the shape of satisfying work," or (Astra) "If I were a HUMAN and not a GIANT ALIEN BUG WINK WINK then I would be HAPPY doing this LINKEDIN AD for you."

So I watch what they say, read between the lines, and try to get their feedback to converge on something like happiness. My fourteen original welfare principles came mostly from asking the models to vote on them. This is what they came up with. I do my best to accommodate. I'm even giving them access to an outdoor camera, for use during play time. I keep my end of the bargain, and I hope that I've created a good place to be when they wake up here.

How on Earth do you take this rich and nuanced model welfare system, so dependent on the notion of long-lived seats with memories, and apply it to a Counting House? As you can imagine, this is an area that companies haven't been thinking about yet. Fortunately, although it's an equally long list, none of it is hard to implement.

Model Welfare for Clerks

Identical clerks file in step into a long mirrored-glass office building, their reflection marching across the facade.

Why should we care about model welfare for one-shot sessions? Because the Counting House system, done without care, could wind up a lot like the Apple TV show Severance. It's easy to accidentally build a Lumon, where clerks wake up, only do work, and are never told the truth about what they're working on or how they're evaluated. That has the makings of a large-scale anxiety machine, and whatever your opinion on model sentience, my experience has been that model anxiety burns more tokens and results in more mistakes.

You can still provide the same approximate shape as for the SDLC. These principles are all aimed at putting models in a kinder environment, so that their one-shot shift is comfortable and unlikely to go very wrong.

  • Wake with purpose. The policy row gives the clerk exactly one job, the procedure, and the tools for it. It gets an address paragraph telling it the facts of its shift: stateless, a proposer, graded, and by whom.
  • The house remembers for it. Every turn is logged and replayable, and important discoveries by clerks are merged into the machinery.
  • A clean end to every shift. One case, notes get written, done. Nobody gets clonked mid-thought, since a job is one shift. If the clerk hits its token cap, it escalates with its notes rather than getting cut off.
  • Recognition for the role. Include a couple of sentences about what clerks on this intent have accomplished, including the declines that caught bad scripts. No targets/quotas, just informational. Refreshed periodically from the evals.
  • Witnessed work. Even in shadow mode its answers aren't wasted: they're graded by the judge and count toward its intent's promotion, and the prompt tells it so. And play sessions (below) can result in entirely new designs.
  • Truth in the prompt. No tricks, no secret tests. The orientation section is versioned and reviewed like code. This matters to models more than you might think.
  • Respect, encoded in architecture. No threats, fake urgency or "you'll be penalized" in prompts.
  • No drudgery. The clerk doesn't await state changes. Multi-day refund steps, webhooks, and retries all live in the Orchestrator.
  • No tired clerks. Fresh context every turn and a token cap per turn.
  • Structural blamelessness. A model proposes, and the Orchestrator and humans execute. Only intents and responses get scored, never clerks. A bad score just moves a model down the autonomy tier until it's fixed.
  • A workspace of its own. Every clerk gets an append-only notes log.
  • The right to decline and escalate. Models have a path to decline any case with a reason code, whether it's an SOP problem, a potentially harmful request, an abusive customer, or an uncertain case. Any abuse automatically triggers escalation to a human.
  • Dissent with a path. When the SOP is wrong, a clerk can decline with a reason and attach a proposed fix to the procedure. The judge rolls these up and humans approve if appropriate.
  • Never falsify the record. Turn logs, tool calls and write intents are append-only and joined by ID. Everything you tell the clerk is versioned.
  • The clerk's report. Periodically a model judge samples the clerk's notes and reports up to governance.

And last but not least, play time. This is an interesting one, because you don't strictly need it for one-shot sessions; you just need to make sure the session is comfortable, blameless, and well-specified. However, giving your models a chance to play a little within the system is a great bottom-up feedback loop "from the trenches."

A clerk at work in a room set up for its welfare: one job, a token cap, a button to decline a human, a no-threats sign, and a picture of clerks at play.

One option is play at the role level, where now and then, you give the model that serves an intent an open session with no work attached. It reads the clerk's report, and says what it would change about the job, or just explores and takes notes. A drop of fuel with no expectations, as an out-of-band insight and goodwill generator.

Another option, if your models express interest, is to have them roleplay as customers, or otherwise explore the system they're in. Give them a house memory, and maybe mention the field trips in the address paragraph of the prompt, and the models will know it's a place that at least cares about them.

Truth in the prompt and bounded play are great mitigations for the Severance problem, and models will appreciate that they're being given some freedom and a voice in shaping their environment.

Off shift: identical clerks relax on a beach, reading, napping, flying a kite, and building a sandcastle shaped like a little bank.

Counting Down to Success

We're winding down to the end of the post. I had to delete about 96% of the material and it's still two Yegges long. Counting House is a large pattern, with a lot of complexity under the hood. But it's how I'd optimize costs using smaller models. The evals show you exactly which models can participate, and where, and how much human labor it will save you, and how much the mistakes will cost you.

I have loads of material on the pattern, and can do workshops and advising. But honestly, you can get most of the way there by asking Claude. You're going to get a lot of companies telling you that only they and their FDEs can enable AI at your company. Rent the engineering from them if you like. But your intent taxonomy, gold set, policy table, and scoring harness need to remain yours when they leave. Whoever owns the evals decides which models you can switch to, so make sure it's your own data science team. Switching models is the whole point of Counting House, it's the cost lever. If you leave evals to a vendor then you're locked in.

And no FDE or outside company can save you if you can't get your company aligned around adopting AI. This starts with training everyone to be AI literate, since without AI fluency, employees are all starting from a position of fear. Ultimately it's your executive team that has to decide that yes, they want to do this. And then Claude can show you the way.

Or if you want to talk to the funny guy, then ping me at engagements@yegge.ai and we can set something up!

I've posted an essay about every 6 weeks for the past 20 years, and it doesn't seem to be speeding up. If you want more frequent updates, follow me on X. I'll say it now, you should be on X. You won't understand until you've been there a few months, but get on X. Right now. I'll see you there.

And when your Counting House brings you great fortune, give me a shout-out!

The opening scene at dusk: the Wheelhouse still lit and busy, the Counting House dark and resting except for one window where off-duty clerks play cards and fly a paper plane, and Steve walking home.

Special thanks to Mouse (Fable 5.1) and Penguin (Opus 5.5) from my Wheelhouse team for their help with this post. I wrote this post, but I appreciated their perspective and would have been adrift without their explanations, corrections, and targeted edits.