Tag: ai

  • Judgement > Output

    Danbo and a bird looking at each other. The bird has it's beak wide and looks like it's yelling or demanding things of danbo.
    Credit: Flickr / Matt Cooper

    The benefit of conference season is talking to a lot of people (the downside: needing antibiotics), and of course many of those conversations came to AI, explicitly or not. But underneath the obvious shift there’s an industry one, which started with the end of ZIRP and AI exacerbated. An industry that used to be quite cushy has become more cut-throat. You could call it “higher expectations”, which is often understood to mean higher output.

    When output was expensive, we used the cost of it to sidestep a lot of poorly judged ideas by saying they were too much work – less drama and less work than explaining why the ideas were bad. When output gets cheap, one, we can’t hide behind the cost anymore, and two, you discover that increased output is not a pure positive. The person who would go on some kind of crusade for their pet project had human limitations. Now they have a machine and are DDOSing their colleagues.

    Lack of judgement used to mean not producing; now it means 10+ pages. Low impact becomes negative impact. In code, that cost shows up as incidents, which at least get measured. Wasted human time we could measure in tokens spent summarising long documents that nobody wrote and nobody read. But the bigger cost is the undermining of confidence and trust between coworkers. Handing your colleague a document you cannot explain and did not think through is something between negative impact and a no-op. And if it’s a no-op, that’s because people have learned not to bother reading the things you send them – arguably worse.

    So the expectation that has actually gone up isn’t output; it’s judgement. The value of a butt in a seat producing something, whether that’s code or a backlog, but lacking higher order judgement, is gone. That is the thrust of all these AI-driven layoffs, even if it’s not said explicitly. The companies that aren’t doing layoffs are still doing more intense performance management; with or without layoffs, expectations are higher.

    The problem is that judgement is hard to evaluate and critique – because it’s hard to define, and too often means “agrees with me”. “I know it when I see it” was the evaluation standard of the industry for a long time, and whilst improvements have been made, it’s always been the fallback. It doesn’t just mask bias; it makes “the best so far” the proxy for good, which underestimates what good could be. Performance management is no more a science than hiring is, and like hiring, has always been wildly inconsistent even within a single organisation.

    You could say engineers were judged by impact, but output was the driver, and “impact” was a bit of a nonspecific term – especially once that impact had to be apportioned across everyone it takes to get something done in an organisation of any size. Non-specific “more”, “better” or “faster” doesn’t work any better now.

    The vaguer the role, the harder it gets. Product, a less defined role closest to engineering, is illustrative. Managing a backlog is work, but it only has value if it causes more to get done, which isn’t something we can empirically prove. The real value is in whether the ideas are well founded in what users actually want. Diana Kimball Berlin shared a graphic of Gamma’s research loop which I like for its clarity on what a good product manager does: you can automate a lot of things, but there is judgement in extracting the insights that will actually improve things for the user. For PMs, judgement was always the driver, and the latency of output made it a lagging indicator. Faster feedback has made it easier to evaluate, and possibly explains why I’m hearing from engineers that they didn’t backfill the PM, but instead just used AI.

    Even where the standards are defined, they have to keep being aligned on as they get pushed down, which means telling some uncomfortable truths like “we’re not doing a layoff but we do plan to have fewer people working here, so…” The result is that people’s experience is diverging. Managers report feeling compressed by pressure from all sides. Leaders are frustrated that they know what they need, but are missing the tools to get there. ICs are experiencing the pressure as random and unrelated to actual goals, a product of having less power and little or no insight into what’s driving it.

    Now it’s not enough to notice whether someone is producing; you have to assess the judgement behind what they produce. Most metrics don’t help, because they measure output too. Shipping with an incident is absolutely not comparable to shipping without one, but we never did a good job of measuring and valuing the work that ensured the problem didn’t happen. Devex, or AI enablement as we’re mostly calling it now, was initially measured on adoption, and is slowly reorienting to the impact adoption drives. Internal teams rarely have CSAT metrics at all, which is why underperforming HR teams go unnoticed (and now is really a bad time for that). Anything that drives output needs to be balanced with effectiveness – in engineering, with reliability being core.

    If you’re a leader, this is a blend of new problems and old problems that just moved up the priority list. “Not great but manageable” has become a bottleneck. Teams used to be able to derive their direction from the roadmap; now they need something bigger (a purpose, a mission) to judge it against.

    If you’re an IC and the pressure feels random, this is what’s driving it, and you get to choose what to do in it. I see a lot of “this is a bad idea” from ICs, and maybe it is. But if you’re closer to the work, you should have better ones. That’s what judgement is.

  • Attempting to reverse engineer code review

    I keep hearing complaints about code review. About volume, about quality, about trying to take responsibility for code produced by colleagues who have abdicated their responsibility for thinking and are uncritically hitting enter.

    But how much of that is just grumpiness? Frustration? “AI slop” is the new “I didn’t like it.” Code review was always something I paid attention to as a manager, because when there was conflict on a team, it was the place most likely to show up. Code review is subjective work presented with the illusion of objectivity; everything unsaid can play out in it.

    In the software factory model, no one writes the code and no one reviews the code. These go together: once the code is generated, human review is the bottleneck, so you validate some other way, through guardrails and automation. Most teams aren’t building from scratch, though. Akin to the challenge of self-driving cars, they are on mixed roads, humans and machines sharing the same lanes, which is the harder case (I loved this podcast on the topic).

    So I wanted to see whether I could reverse engineer code review: some kind of machine gate a PR has to pass before it’s allowed to request actual human time and attention. Much of my career has been in open source, the write-code-in-public kind, so I picked a team with an open source repo I knew well enough to have context. I deliberately didn’t pick the highest-performing team I’d worked with: this one had mixed levels, engagement, and AI adoption, and I was curious how that played out in review.

    What I did

    First, I constrained Claude to be read-only on GitHub. The last thing I wanted was to leave a trail of inadvertent and likely weird AI-generated comments.

    Then I had Claude (Opus) find the strongest code reviewers on the team, seeded with three names I asked it to confirm or deny. Its reasoning tracked with what I knew. It also flagged some reviewers who were thorough but inclined to nitpick, which made me add a style guide to the scope.

    An aside on style guides: I started my software engineering career at Google, where the style guide was canon. I had readability in two languages, Objective-C and Java (basically, I jumped through a set of hoops to prove I could deliver high-quality reviews according to the style guide). I don’t really subscribe to any religions, in software development or otherwise, but one thing I took from that period of my career is that the true benefit of a style guide is that we do not discuss it, and spend the time on the things that matter instead.

    Claude (Sonnet) built a corpus of 4,176 review comments and summaries by twelve reviewers, on over 1,700 PRs going back about 20 months. A Fable agent per reviewer pulled out the patterns in what they flagged, with examples: ~300 of them, which another agent compiled into 67 themes. Then I had it turn the themes into checks. Finally it went through every PR merged since the start of September (~500) to work out which checks the version the first reviewer saw would have failed, and whether review caught it. As on many teams, there were already AI review bots on the repo, so “review” in everything that follows means humans and bots together, unless noted otherwise.

    This took real time, most of it reading what came back. It’s the kind of data I would have loved to have when I was running large orgs, and now it’s feasible.

    What the machine recommended

    What came out was a tidy recommendation: 57 checks:

    • 11 style guide rules
    • 8 test requirements, either tests for classes of issue or required test types for certain kinds of work
    • 26 CI and static analysis checks (lint rules and scripts that run on every push)
    • 12 instructions for a project-specific review skill that an AI reviewer could run, the judgement calls that don’t reduce to a regex.

    I’ll call those style, tests, CI and judgement from here on. It also set aside 16 norms that authors had pushed back on at least once, sometimes for good reasons, and said not to enforce those mechanically at all.

    Run against the last six weeks, more than half of PRs would have failed at least one check, with ~700 issues between them. 71% of those were never raised in review, by human or bot, and most of those merged without anyone mentioning them. Review raised 75% of the judgement issues, for example around metrics and concurrency, 32% of the style issues, 14% of the missing tests and 10% of the CI-level problems. Count only humans and it’s 43%, 22%, 9% and 5%.

    (Some caveats. The CI checks were simulated by agents rather than actually run. The agents could see the review history, which probably flatters the judgement numbers: the checks were mined from these reviewers’ comments, so “review raised it” is partly “the check describes what the reviewer said”. A second pass re-checked every eighth PR and agreed with about nine in ten of the findings it was confident about. And only GitHub is visible, which is obviously incomplete.)

    On paper, that’s the case for a gate. Write the bar down, have an AI teammate check every PR against it, and only then ask for human time.

    Very nice. But what does it say about code review?

    What it taught me about code review

    It’s inconsistent. 92% of PRs had exactly one human reviewer (6% had none), so whatever that person missed went through. Individual reviewers raised anywhere from none to 38% of the problems in the PRs they reviewed, with a median around 20%. And there were 65 cases where someone flagged a problem on one PR, then let the same problem through on another. Reading the examples, my impression was that the miss was often a side change rather than the point of the PR. That makes sense, and with more and bigger PRs, it’s a prime reason to move this work to automated checks.

    Some of that is just asking people to do a machine’s job. PR review was never the way to catch mechanical things. How do you hold test coverage in your head? That’s why teams that cared about these things always instrumented them. Looking back, readability at Google was kind of weird: a linter installed in my head. Test coverage requirements would probably have been more impactful.

    The misses don’t obviously cost anything. I had Claude trace the 400+ unflagged issues forward: did a bug fix, hotfix or revert later touch the same lines? Then it compared that against other new code in the same PRs, adjusted for size. About the same. 20 of the 345 it could trace were followed by a fix, against about 17 expected. Reading the 20, only about half fixed anything the check plausibly described. The clearest cases were missing tests, which fits my hunch about coverage, but on small numbers to YMMV.

    That’s a lower bound: each PR had between two and 42 days of follow-up. It also only measures bugs, not the slower cost of architectural mismatches that make the next change harder to test, or dead code someone reads around for years. But it was enough to shake my intuition that the misses mattered.

    So what is review doing? Enforcing things the team hasn’t necessarily agreed to. Some of what reviewers raised was really shared: everybody flagged leftover dead and debug code. A lot of it wasn’t. Dependency injection, the architecture rule that came up most, was raised by only half the reviewers. Asking for missing tests was something seven of the twelve did over the 20 months, but in the six weeks I audited, review asked for tests only about 14% of the time, mostly humans. And in 16 places a reviewer’s norm got pushed back on, and in a few of them reviewers wanted opposite things: for instance, whether a feature flag should be read at runtime or once at launch.

    These are all reasonable stances to take, and it’s possible they were situationally dependent in ways that I don’t have insight into. But from this data, and for an individual onboarding, it would be hard to distinguish a team norm from one person’s preference. It’s muddier still when, as here, some of what read as a reviewer’s judgement was actually AI output transferred via C+P.

    That’s where the conflict comes from, and where bias gets in. Google studied pushback in its own code review and found the amount authors get varies with their gender, race and age, at a cost of more than 1,000 engineer hours a day, borne by engineers who aren’t white men. It is very easy to shred a PR if the person who wrote it seems like someone whose competence is in question, whether that’s down to what we politely call “unconscious bias”, because they were annoying in the meeting yesterday, or because the perception is they have handed their judgement to the machine.

    So would I actually implement this?

    There’s a real cost. An AI review on every PR costs tokens, and new lint rules and required tests add CI minutes to every push. The tokens are largely a more effective use of the AI review that’s already in place, at about a dollar a PR. Implementing everything would add somewhere between 6% and 38% to the CI minutes the repo already uses, depending on how carefully the expensive checks are scoped. Then there’s time. A gate adds 5 to 30 minutes before a PR goes green, a few percent of the time a PR already waits for its first human review. The bigger cost is that 55% of PRs would fail a check on their first push, so more than half would go round a fix-and-rerun cycle before a human was even asked to look. Against a bug rate that, so far, doesn’t move, that’s hard to justify on bug prevention alone. The case for it is consistency, which I think has value of its own, but has to be balanced against the cost.

    When individual enforcement becomes team enforcement, the team needs to agree. The cheap checks, which are also the lowest leverage, are the ones nobody disputes. The highest-leverage ones, like dependency injection, or what counts as properly testing a feature flag, are the ones that need an expensive team discussion first. Skipping that discussion would turn one reviewer’s preference into a CI failure. But the value of that discussion could easily lower than what it would cost to have it.

    The biggest value of code review isn’t in the repo. The most valuable thing review does is ask whether we’re building the right thing in the right way. Nearly half of what review raised here didn’t map to any check: product and behaviour questions specific to that PR. Ideally, those come up before review, because review is the last gate before user impact. But they’re context-specific, and the data to answer them isn’t in the codebase. What they point to is structure: who gets to decide what. What can an IC decide about their own project? What needs a broader group? Google’s readability and owner split was a version of this: style was settled once, for everyone, and ownership said whose call the rest was. Without that structure, those decisions get made in code review, by whoever happens to be reviewing.

    So, would I implement it? The uncontested checks, yes: done efficiently, they add only a percent or two to CI. The rest depends on decisions the team hasn’t made yet, and depending on the team those discussions could be expensive. Right now code review is where those decisions get made, one PR at a time, by whoever is reviewing. Even if that worked, more or less, when humans wrote the PRs, I don’t think it works in an agentic world.

  • Nine Months of The Spreadsheet: What AI Taught Me About How I Work

    Credit: Google DeepMind / Pexels

    In February I wrote about The Spreadsheet: one tab per week, a column for each area of my life, and a green/yellow/red rating for impact and vibe. Every Monday I feed it to Claude and ask “how did I do?” and “what can I learn?”

    Nine months in, I’m still doing it. And with that much data, the patterns are inescapable. It’s also a nice grounding way to start the week – on Fridays, I plan the following week. On Monday I look back and see what I can learn.

    AI is a better coach when you play to its strengths

    Ask AI for productivity advice and you get the average of every productivity blog post ever written. Plan your week. Take breaks. Eat the frog. Many good ideas, I’m sure, but I think productivity has to map to the way your brain actually works – not how you think it should.

    What AI is actually good at is spotting patterns in a lot of data and reporting them (more) impartially. It’s also notably kinder than the voice inside my head can be. I’ve come to think the trick is giving it two things:

    Data. Not “I feel like I’m not getting enough done,” but months of what I actually did and how it actually felt. Real data keeps it honest. It also gives it a basis to call me out when I’m doing something stupid – like working 40+ hours whilst being quite sick.

    What good looks like. Sometimes I define that up front, but in this case “good” is the average of my good weeks, not the impossible idea of “productive” that lives in my head. That version of me ships every workstream every week, never gets tired, and has somehow also taken up running. She doesn’t exist (I really hate running). My actual good weeks do, and they’re a much more useful benchmark. Once Claude knows what those look like, it can tell me whether I’m on track or drifting, instead of measuring me against a fantasy.

    In coaching, I often think about it as reflecting someone’s best self back to them. With this much data, Claude can do that for me – at least in terms of what I’m getting done.

    So here’s what nine months of evidence told me. Some of it is stuff I mostly adhere to because I believe it, some is stuff I keep having to learn again and again – and some of it I learned through this process.

    I was more burnt out than I realized

    At the start of the year, I spent most weekends spaced out. Physically present, not really there, and pretty unavailable emotionally. I would not have told you I was burnt out; I was pretty high on creative energy (poor Jean). But every weekend, it caught up with me.

    What I was missing was what Jean calls space to just be. Rest was whatever was left over after the doing, and there usually wasn’t much. Gradually I started getting higher quality rest – the floatation tank as a Sunday night ritual. More consistent and longer workouts. Finally finishing the giant Monet cross-stitch.

    In February, 30 hours a week was my limit. Now 35 feels fine, and my weekends feel more like a different rhythm than a complete halt.

    My body keeps score, and it’s not subtle about it

    I started the year with the idea I’d work 30 hours a week, and gradually worked my way up again. But having that as a goal and sticking to it are not the same – periodically I went over, for one reason or another (always a good one! Honest!). But every time I pushed past my limit, I paid for it later physically. It’s so easy to think there’s no time to rest, but if you don’t take the time, the time takes you.

    Productive isn’t the same as progress

    Every Friday I make a list of what I want to get done the following week. And then, of course, things happen.

    How I feel at the end of the week? Directly related to that list. Turns out, I’m pretty much always “productive.” My output is pretty steady week to week. How I feel about it is not. On the bad weeks I still got plenty done; it just wasn’t the stuff I’d decided was important, or I added tasks faster than I completed them. Good weeks, more of the list is done than not. Bad weeks, I let other people’s priorities dictate mine and get to Friday feeling like my week happened to me.

    Which is probably also why days with no meetings are such a highlight.

    Away from the computer is where the ideas are

    When I get stuck or overwhelmed, the spin bike is the best place for me to be. I know this. I’ve always known this. The spreadsheet keeps reminding me anyway, because apparently I need it to. On days when I work out earlier, I don’t get less done – usually the same, or more, and the big unblock is more likely to happen. Again, the things I’m tempted to think I’m “too busy” for are what make hard things possible.

    What I’m changing

    Nothing as a result of this. Just the same things I’ve been nudging towards, week in, week out, all year. It’s not that every week has to be great; it’s that the average week can get better over time.

    In the first post, I said that knowing something intellectually and remembering it when you need it are very different things. Nine months later, that’s still the point. There’s the occasional lightbulb moment, but mostly the spreadsheet tells me, again and again, that habits matter, and to stay grounded in the good ones.

  • What do managers even do now?

    Credit: Paul Hudson / Wikimedia

    I watched an IC have a lightbulb moment: they’re a manager now. Running agents in parallel, giving direction, deciding what’s worth doing. Being responsible for what ships, without doing it all themself. My job is giving them the pieces of management that are useful right now.

    Not how to run a 1:1.

    How to think one level up, and systematically. Not the bug, the bug class. Not the problem directly in front, the one that comes next.

    The ones asking for this help are the ones who still feel responsible for what ships. How do I do this responsibly? I see pro-AI-development and cognitive surrender being conflated as the same position. They aren’t. The work is building systems where you can be responsible for what you shipped without being in the details of it.

    ICs spend – or at least used to – more time focused on the thing immediately ahead and as such are biased to notice that. It made sense when that took a while, but now it can be faster, the timelines are compressed – or they can be, if things are working well.

    Managers encode judgement through process and relationships. Relationships build judgement 1:1 – and that’s valuable – but process is how judgement scales.

    Scaling agents has really tapped into something for me, and it’s all the skills I used to use when I was scaling teams – many of the same ideas showing up at different times. Also things other people used to do and I never got into the depths of – I’m spending a lot of time in CI, lately. I never did that before.

    The human parts of management are important. Of course they were. But the people who thought managers did nothing but go to meetings always seemed to miss the pieces those meetings (and documents, and nudges, and questions) moved forward. Which was finding the things that were lossy – whether partial processes, or missing guardrails or understanding – and causing them to be filled in directly or indirectly.

    If you want to move at speed with these new tools, you can’t have 8 people on the team moving as fast as possible and one person trying to figure that out. You certainly can’t expect that from the person who has so many direct reports they can’t physically meet with them each week. Every IC needs to learn how to do it for themselves, and then collectively, so the team can move faster together.

    Management is learnable. When you scale a team you build a layer of management underneath you. You bring people in, or you lift people up, and you train them. You take the values of the team and encode them into process, because process is how you get multiple people to do the same thing consistently.

    ICs aren’t being asked to do this. They’re being asked – told – to go faster. But this is how teams go faster. This is the work good managers always did. Now they need to scale that work earlier, with people who didn’t sign up for it deliberately.

    If we’d ever really been clear on what a manager does, perhaps this aspect of management would be more defined and it would be easier. Oh well.

  • Skills for a more leveraged workflow

    Adapted from Mix and Match Robot expressions by anarres (Openclipart, CC0)

    I’ve been skeptical in general about sharing workflow skills. I have a bunch I share with direct collaborators, but there are a lot of “AI skills” repos out there and I’ve only found 2 of them really useful to me (#1, #2). I thought working out what you wanted and making your own was important – but I kept packaging up a similar set of skills to share with specific people, because I underestimated the value of having a starting point. So finally I accepted – I too would make an AI skills repo: ai-dev-starter-kit.

    The conversation I was having is how to go from leveraging AI to leveraged by AI. I think it’s a question of what do you hand off, and what do you keep? AI can do a lot of work, but it can also generate a lot of work for you. You have to figure out how to manage that so you’re not just scattered and getting less done (been there).

    At the core of it is a board configured a certain way, and three terminals. The board is the source of truth, and work is issues moving through a status workflow. Two of the terminals are the skills that generate and validate code: the orchestrator, which fans work out across the board, and the merge queue, which serialises the landings and does final validation before anything goes in – things that can run for hours with periodic checkins. The third terminal is the one that requires thinking and engagement, and the work I do in it is to make work agentable (i.e. push it into the queue the other two terminals run).

    The other thing in it is a sample CLAUDE.md – talking to people in mid-sized orgs, I consistently notice it’s under-leveraged, and it’s much easier to change robot behaviour than human behaviour.

    The others are smaller. One goes through what went in overnight and flags anything I should worry about – review isn’t required before merge, but I still want to know what landed.. fable-audit-prompts is my answer to “what is Fable even for”: I use it to audit things where I have vibes but not concrete ideas, and it generates a report that makes that work agentable by less expensive models.

    If you’re looking for a starting point for a more leveraged workflow, I hope this can be helpful – whether you take some pieces as they are or just as inspiration. Feedback or ideas always welcome.

  • Maturing our development process as an AI-first 2 person team

    Credit: Landiva Weber / Pexels

    We’re doing a big release this week for DRI Your Career. Mostly bug fixes and reliability work nobody will notice, but the two things worth mentioning:

    • Audio episodes are now available as a private podcast feed.
    • Made significant progress towards supporting multiple coaches (yes this implies some very exciting announcements – more soon).

    We’ve come a long way since it was just Jean and “Claude the intern”, but mostly very iteratively. We’d come to a point where it seemed worth rethinking. We wanted to ship some bigger things – this release was nearly 100 PRs over about a week, passing back and forth between Ireland and California – and to do that we had to mature our approach.

    1. Release branch and release process (vs pulling from main). I know this goes against conventional wisdom, but for a 2 person team with no on-call SRE, I think it’s better to be able to schedule your problems for a convenient time.
    2. Orchestrator + merge queue. The merge queue tends to be stricter, so everything gets 2 code reviews, follow-ups filed, and so on. It also handles sequencing and rebasing – the things I used to burn CI minutes on when I was chaotically working in 5 tabs.
    3. Board hygiene. P0/P1/P2/P3 issues, aligned with the orchestrator + merge queue setup. Everything related to features slated for the next release gets tagged so it gets prioritised.
    4. Devex and optimisations. Ephemeral DB targets, authenticated E2E tests, env var requirements, a full design system, automated nightly DB backups.

    What I find interesting is that these four pieces map onto four different phases of a team maturing.

    1. Overkill. Slightly anal for two people. But the natural progression for a 2 person team no longer wanting to deploy main on every PR.
    2. The AI-native piece. Only necessary because AI exists – also allows Claude to work overnight whilst we are sleeping (and don’t need the tokens for everything else we are doing).
    3. The setup of a well-organised 6+ person team. What keeps the orchestrator (and us!) focused on impact rather than maxing out token and CI spend. Brought in when I realised we could easily ship >50 PRs but no finished headline feature.
    4. A level I used to watch 20+ person teams struggle to reach. Necessary to support the velocity.

    All of this to say that a sensible setup for a 2-person team has shifted dramatically from even a year ago. If the shift is that big for a small team running a small website on top of other activities, it’s going to be even more dramatic for larger teams. The list above is our specifics, but the categories are transferable: even if your process was right sized for your team before, it isn’t – or shouldn’t be – once each person on your team is running a team of agents.

    1. What is your release process? How do you validate, find issues and monitor?
    2. How do your agents run, what guard rails are in place?
    3. How do you maximize value for token spend?
    4. What’s your DevEx setup and how does it need to evolve?

    Lately I’ve been thinking about the AI shift as upending the standard bottlenecks for team scaling. That’s moved me from “we need a new playbook” to “we need to update the existing one with different priorities”. All but one of these are things I’ve done as part of scaling a team in the before times. I just didn’t used to do them all in a week.

    This is why I stay optimistic about developers even as writing code gets automated. You can’t just “AI it”: getting AI to solve problems faster than it creates them takes judgement. Claude helps enormously in moving things forward, but knowing what to prioritise and what it should be doing is a critical input.

    This is the kind of work good managers do – evolving how the work gets done as the team evolves, to keep things moving. So we’ve decided, again, that we “don’t need managers”, at the exact moment there’s more of that work to do than ever… My hot take from years of experience scaling teams and being deeply immersed in AI-dev for the last 8 months or so? We’ll come back to the same conclusion – managers matter 🍿

  • The Value of Getting Closer to the Work

    A raccoon stands upright in a surreal, upside-down landscape — a spoon-armed column, a floating hand, an inverted figure, and blue hill-creatures with little doors — picking its way through.

    Scaling

    The core problem of scaling a team is that not everyone can know everything. Scaling is the work of systematizing that – so that things are understandable, knowable. It’s turning “tribal knowledge” into documentation, automation, and process. It used to be that only the automation was for the machine, but now documentation is machine critical and process is machine readable.

    With humans, things bubble up and down. You don’t – in general – make systematic fixes for the first complaint. First you validate, check the scale of the issue. You try the softer interventions – trying to get people to actually talk to their team mates – before you start building process. Maybe you make a change a month to the way a team works. Carefully, thoughtfully. Because no effective person wants to be on a team where the job seems more process than impact.

    Automation was always the most effective, but also usually the most expensive. The thing that gets put in when “please follow this checklist” fails frequently and severely enough to be worthwhile. An incident can be the gift that delivers the agreement on priority that always seemed necessary to those close enough to it, by making the cost of not doing it clear.

    As we are deep in replatforming Twill, I’m back to being a developer, and as I operated my orchestrator and dispatched agents on the backlog, even though in some ways everything is wildly different, some parts of it that I reached for out of habit felt very familiar: scaling.

    +AI

    AI feels like speed-running some of the team scaling stuff that previously I orchestrated over months – years even. The economics have shifted to make automation much cheaper, so it’s no longer deferred, but top of the list.

    With people, knowledge lives in two places – in the people and in the systems. With AI, only the systems part persists. Every session starts contextless. This is the problem that surfaces all the documentation that wasn’t written, and all the automation that was never built – a core part of why building with AI from scratch is so much easier than retrofitting onto an existing setup.

    On DRI, when Jean started building the platform, we talked affectionately about “intern Claude”. Over time, we matured away from that framing. We put the guardrails in. We stopped treating it as a novelty to be supervised and started treating it as a system to be designed. With Twill, I started from a higher level of understanding, and the baseline was higher.

    The snark, which I’ve also expressed: people will do for a machine what they were never willing to do for their team. Write the doc. Define the work clearly. Build the guardrail. Forgive the mistake.

    I stand by it. But that doesn’t mean those of us who were always trying to do this for humans gain some moral high ground by not doing this for machines.

    The rigour was always good practice. The humans operating the machine will have a much better time if those machines are set up to succeed. I think the call for EMs to “get closer to the work” has been poorly expressed and executed, but at the core of it – having got much closer to the work myself – I have come to see the validity of it. The scaling problem that managers used to deal with is now happening at higher speed, and the only way to have fidelity on it is to understand the work being done. Because so far, there is no playbook – we all have to write our own.

    The best example of this is CLAUDE.md. For both DRI and Twill, it’s a living team artifact, and the churn on it is a good thing – the way of working should keep evolving, as we evolve our understanding, our work, our effectiveness.

    Three terminals

    “Dispatch a review agent” was encoded in our process for months. It’s very helpful. But I would still find the odd issue or untracked follow up. Then, tired of Claudes colliding on CI and wasting build minutes, I made a merge queue. It serialises merges across concurrent sessions, files the follow-ups, and sends PRs back when they’re not ready.

    The next round of reviews I dispatched came back clean. And the next. And the next.

    Turns out merge queue Claude, reviewing work it had no part in, was stricter than the Claudes that had done the work.

    The reason is the contextlessness. Which gets talked about like a strength or a weakness as we look for metaphors about working with this thing that is new and we don’t yet fully understand. I’ve given up on metaphors, and think this aspect, like this shift in general, is best treated as a neutral.

    The reviewer had no investment in the work. Nothing to rationalise. No “well, I did it that way because.” It knew only what the standard said and whether the code met it.

    So many times, after reading a review or finding something that didn’t work as I expected it to, I’ve thought that something should be already prevented by CLAUDE.md and asked why it wasn’t. The answer was always clarity – fair. But turns out, like humans, the answer could also be process.

    The common manager refrain on AI is that it separates those who think from those who don’t. Fair. But one of the core skills you have to develop in leadership is knowing what to pay attention to and what can be ignored. The broader your scope, the more you need to be brutal.

    How many times have I told someone “that’s not making it onto my list”? Too many to count. Enough that people have been known to text me when they find themselves saying it. Now, having so many things in flight at once, I’m always trying to figure out what to pay attention to and what not.

    The mechanical guardrail is encoded in CLAUDE.md: hit the same issue twice, stop. A lesson learned the hard way after burning a bunch of build minutes on something stupid, because it only got intermittent attention and seemed like it should be fine. It’s inclined to keep pushing through, but telling it to stop also tells me to stop, to give that problem some attention, to think.

    Before, the discipline was: you found a bug, you wrote a test so it wouldn’t come back. Good practice. Now, I do that, and then I ask: how do we guard against this class of problem?

    Competent humans rarely make the same mistake twice. AI will make the same mistake endlessly.

    Making a class of issue impossible is not cheap. This is why we didn’t use to do it. A minor bug spawns five-plus follow-ups – graduated solutions of increasing complexity. A null-handling issue that threw 500s; fixed in one PR, but the follow up issues to eliminate the pattern and add a tripwire in CI so it can’t come back got added to the backlog, ready to be picked up, implemented by the orchestrator as a slot becomes free. Done within the week. Because now it’s possible. It’s tractable: it’s bounded, well-defined work, which makes it exactly the kind of thing you can hand to an orchestrator and let run in parallel.

    I keep reading all this stuff about “working with AI” and it’s so depressing that most of it seems to be about building things that run AI agents. What’s the point of it? What are you actually shipping? Is this just some game, like The Sims: But Make It Programming (and also really expensive)?

    Cranking through a bunch of stuff to launch, I’m running Claude in three tabs. The first is an orchestrator. The second is the merge queue.

    The third is the terminal where I actually define, decide, and unblock things. The most important one. In this one sometimes I feel like a product manager, sometimes an engineering manager, sometimes an architect. Sometimes I’m just wiring up secrets and things, feeling like some modern day version of the ENIAC women who used cables and switches. At least I get a manual.

    Scaling a team, you keep coming back to the same need: be clearer. Clearer about what’s expected, who owns what, what good looks like. Clarity. Clarity. Clarity.

    The work of the third terminal. Clarity.

    The tools have changed. The work is the same.

  • WordPress + Claude – a new look for cate.blog

    Created by Claude Design

    If you’re reading this on cate.blog, things look different. I’m pleased with how it came out. Getting there was more difficult than I expected.

    I came into this confident. I used to work on WordPress (admittedly, the mobile apps – but I did the support rotations and lived in P2 like everyone else). I have been running my own site for years. This was not my first visual update, it was just the biggest. I thought my expertise + Claude design would be a winning combination.

    It was not.

    I knew what I wanted: a stronger brand, something more visually distinctive, an information architecture that put DRI front and centre, more of a “website” than “just a blog”

    What went well

    The design. Yes, it has the italicised titles characteristic of Claude design. But overall the look is a nice balance of pops of color but not in the way of reading. The goals have been met, and I’m really pleased with how it looks.

    I love Claude Design, high on updating DRI Your Career I was looking for other things that could be prettier. I also used Claude Chrome to validate: visual detail at the level of font sizing and spacing is not a strength of mine, so it was incredibly useful to go through multiple rounds of checks and the entire site to ensure that everything had been applied just so.

    What went poorly

    TL;DR: WordPress.

    I started using Claude Code as an issues manager and lookup engine, but after a day of this, frustrated by how much slower the progress was than I’d expected, I connected Claude Code directly. This was a huge accelerator, although I had to pay attention that it built block components rather than making everything custom – I do want to be able to edit it, after all. But even with that working, CSS edits still had to be copied and pasted.

    I made a staging site, which I have never bothered to do before – probably something that had kept the scope of my previous redesigns small, ensuring that I could keep a working and visually acceptable website as I went.

    I’d decided against a child theme, on the reasoning that child themes don’t receive updates and those are important. However because all the customisation lives as configuration in the database instead of in theme files, appearance and content are in the same place. Deploying staging to production took two hours, took my site down while it ran, and broke Jetpack on the way. I knew this going in (well not Jetpack breaking, that I had checked a box to prevent), when I set up the staging site, and decided to go with it. But it is still horrifying.

    I moved from Twenty Twenty-Three to Twenty Twenty-Five, which is extremely minimal; very little is defined until you define it. As the definitions live in two places, Global Styles and a separate CSS block, any given margin might be set in either. Reconciling that was fiddly and irritating.

    I always heard web developers be dismissive of WordPress, and I didn’t fully get it, because what I have always loved about it was that it was easy to write in, and that was my goal of having a website. But having dealt with this I get it. The tooling is just so frustrating.

    Now that I have more of a brand, I decided to update What Raccoon to match – it’s more prominent on the site and felt jarring to switch, so wanted them aligned.

    Yes, it’s a much smaller site. But it took 10 minutes and had one minor bug: keyboard detection in Chrome. Done.

    cate.blog? Four days. Yes doing other things, but I maxed out my Pro plan twice in one day working on it. The last time I did that, I was making a 50 page slide deck (for the first time, inefficiently). My website is not that big. The difference in effort was wildly out of proportion to the difference in complexity. What’s interesting to me is the gap between where AI accelerates and where it doesn’t. WordPress – the speed increase did not match the increase in my ambition. But for a regular website? The speed increase is driving my ambition.

    What I’d do differently

    I did write GitHub issues, but they were the spec I handed Claude Code to work through, not a tracker I was keeping for myself. As things got more complex, I was missing one source of truth for me, across the three Claudes and all the manual steps in between, that stayed current as the work moved.

    So, next time:

    • Treat it like a project for real: not just a spec to kick things off, but a source of truth I keep current as I go, so the plan and the actual state don’t drift apart.
    • Content backed up to GitHub so there’s real version control on it.
    • Still build the staging site to test against, but deploy by connecting Claude to the live site and having it move the changes across, rather than deal with deploying and all the problems that entailed.
    • A better way to keep Claude Design, Claude Code, and Chrome-for-validation in sync with each other. A validation checklist living in GitHub would go most of the way.

    I’m keeping the new look, and I still have a deep love of Claude design. But my WordPress ambitions? Those will be tempered for a while.

  • Useful AI skills for Engineering Leaders

    The volume is up. More PRs, more pings, more incidents, more asks coming from more directions. I used to have 8 directs that dealt with most things, and organized and routed information to me for what I had to help them with, but since embarking on a new adventure, I’ve been rebuilding a different system that helps me stay on top of things and prioritize it.

    Three jobs, in order: 1. route the tedious stuff so it doesn’t waste my time, 2. report usefully so I can orient quickly, and 3. understand what’s actually going on.

    These all run on Claude Code as part of a broader chief-of-staff setup. I started with this repo and have expanded extensively. Here, I’ll focus on the set of skills that cover the core of EM operational work.

    1. Routing: get the tedious stuff off your desk

    Tedious but useful: maintaining a source of truth, and making sure that incoming requests get appropriately routed. Many EMs try, reasonably, to enforce a process here, but let’s be real; that process never applies to the C*O or your boss. They send the request as and how it works for them.

    I had a Zap set up for this, but it relied on a narrow trigger. Now, I have a skill that scans the channels where work comes in and reconciles them against the boards where work is supposed to live. It is also instructed to read the thread before flagging anything. If the ask is resolved in the replies (“we already have this”), flagging it anyway just makes noise, which is the thing I’m trying to reduce.

    That last rule is important because I don’t want one more thing pinging me. The point of routing isn’t to surface more; it’s to surface usefully and reconcile quickly, so the backlog reflects reality.

    2. Reporting: start the day knowing what’s going on

    The reports run first thing, and they run ops-first. Strict ordering: anything broken right now beats anything in flight, which beats anything in the backlog, which beats anything in the future. The daily version is deliberately short; a few lines, only the highest-signal items, and if nothing flags it just says “clear.” I don’t want a dashboard I have to interpret; I want one line that tells me whether my initial plan for the day has to go in the bin.

    At longer cadence: a weekly view for what’s moving and what’s stuck, a monthly one for the board update. Same sources, different altitude. Daily keeps me from missing fires; weekly keeps me from missing drift; monthly is a quick and useful way to keep other people in the loop.

    “I have a morning report” is the most boring possible sentence. But I think of it not as a report, but as an orientation to my day; something that looks through everything in a structured way, so that I know where I am and where I need to go, quickly. The terminal is also the feature: plain text on a yellow background (so I can distinguish that terminal from all the others), and I don’t risk getting distracted by “just this one small thing”, which is a) never just the one, and b) rarely that small.

    At a previous job, once I stopped getting pinged on every incident, I stopped knowing they were happening as they were happening; I learned about it when I reached that particular channel in Mattermost, or got through all the explicit demands for my attention in Asana. On some level, that’s fine and expected: if, as a director+, you’re in every incident, that is itself the problem, and incident response being able to run to your standards without you being aware is evidence that it has resolved.

    But then I had to look at everything. Now I can have structured, prioritized information without needing to. Cheap operational awareness is a win.

    3. Understanding: what’s actually going on, versus what we agreed

    Once you’re organized, and oriented, you come to the actual work of an EM: making the team better.

    It had been a long time since I consistently looked at PRs, but now it’s ~daily. I have heard that PRs are redundant now, and maybe I’ll come around to that point of view, but currently I see a PR as a queryable unit of work. It’s never been easier to know:

    • Is this code good? Dispatch a review agent.
    • Does this architecture make sense? Ask questions or for an architecture document.
    • Does this conform to our standards of ways and working? Encode those in claude.md and check.
    • Are the docs up to date? Ask, and then update.

    Better understanding of the work used to convert into process changes; tweak the review policy, change how we run standups. That work is still important, more so than ever. But also think about claude.md or equivalent. Did I learn something about how we work more effectively? Propose a change to it. The file is a really useful lever for team effectiveness and consistency.

    The bonus: the skills are alive

    I’m not sharing these skills publicly because I think the skill.md, tailored to my work situations, is much less useful than the way of thinking. But here’s a snippet I think you should take. All my skills have it, and it ensures they keep evolving as I do, and as the work does.

    ## Living document
    Update this skill when:
    - Triage patterns emerge from real audits
    - A new source of truth comes online
    - The way the work actually happens shifts

    Coming back to the start – I used to manage a lot of people, and now I use AI to manage (and do!) a lot of work and far fewer people. The thing I loved, and still love, about managing people is that they would learn and grow. Skills doing that is not as rewarding – but it is necessary to function.