A couple of weeks ago, the backburner project that had gotten half of my attention while life (and a whiskey collection) got in the way sat in a CLI tab. I was both a little embarrassed and mildly frightened of what state it might be in. Today it's live, and the speed-to-delivery isn't really the important part, though, really, the delivery time doubled since I first wrote this article because I almost thought good enough was good enough.

The build story was boringly short at first, and then I did that thing where I become really invested in an outcome and I realize that the solution that looks easy enough, or might be good enough in someone else's hands has now landed in mine. And I sometimes lose the battle of good enough versus perfect, so here we are.

Generative AI can be embarrassingly easy, and the AI hype crowd is doing their darnedest to race us right to the most mediocre possible outcomes with their scrape-and-regurgitate-automate that never sees the light of human eyes. Generative AI with a bunch of levers in place to drive the potential for embarrassment down a few notches, well, that takes some effort, and some time.

I've been actively building, and still actively improving, a side project that I believe, and sincerely hope, does some public good. Executive Order Translator eotranslator.com ingests every new U.S. Executive Order and translates it into plain language: what it requires, who's on the hook, and why a normal person might care. I even added in a bit of a compliance matrix-like table that can be downloaded as a CSV in case that helps someone along the way.

Why Executive Orders?

Executive orders move real policy in the United States. They can change how agencies spend money, who gets hired, and what gets enforced. And they're published in a format almost nobody reads: Federal Register prose, dense with statutory cross-references, wrapped in boilerplate. And oh my gosh, they're just boring and dry and can be exhausting to get through.

And because of that previous sentence, a lot of people learn what an order means or does from a news headline or someone else's sort of informed hot take. The distance between "what the government ordered" and "what people believe it ordered" is where a lot of our civic noise lives.

Plain language can close some of that distance. I've felt that for a long time; it's the same conviction that ran through my years working in the Federal government. Clear writing about government activity is a public service. And the problem with all of this is that it's not exactly simple to write plain language around a lot of government-speak, no matter who is in office.

Another part of the distance between meaning and belief is a lot newer. AI can summarize anything now (the hype crowd, again), instantly and cheaply (and when you one-shot your way through summaries, it's definitely cheap, if not cheap and dirty), which means a summary by itself is worth almost nothing. You have no way of knowing whether the summary is faithful to the source, unless you're putting in the time to validate claims, force reference citations, and check the work.

And you have to check that work yourself, if you're attaching it to your name in any way, shape, or form.

So the summary is table stakes, for certain. Anyone with a prompt box and a token pass can generate one.

The feature, well, that is that increasingly important receipt.

Check the Receipt

Every claim on EO Translator has a reference number, and that's one of the first receipts you see as a user. When you click it, you're shown the exact sentence from the executive order, and you can jump to that sentence highlighted in context inside the full official text, right on the page.

And, not for nothing, that's generally what my version of table stakes starts with. I need you to at least have the opportunity to see that I'm not just full of prompt and circumstance hype when you see something I've worked on.

That's the part you see. Here's the invisible machinery part that you don't:

• The plain language is written blind. The model that drafts the summary never sees the raw executive order. It works only from a structured distillation of the facts, who has to act, what they have to do, and whether it's required or merely allowed, with the original legal wording physically withheld from it. That way, it can't paraphrase its way into trouble because it never has the words to paraphrase in the first place.
• Every numbered obligation traces back, by construction, to a specific quoted span in the order. That is the receipt you can click on to check. The one-line summary and the "why it matters" note are built from the order's stated purpose and those same facts, so they answer to the meaning check below rather than a per-sentence quote.
• A deterministic check confirms every quote exists in the source, character for character. This is software string-matching, and there's no AI involved, and not one model grading another model's homework. This is much better math than I can do on my own.
• A meaning check reads the plain language against the order on six points: who has to act, whether the action is required or only allowed, how far the rule reaches, whether the work is finished, proposed, or still under review, whether a legal limit or condition got quietly dropped (the "subject to the availability of appropriations" kind of caveat that changes everything), and whether the summary slipped in any characterization the order itself never made.
• The model flags anything that reads differently, and the flag is a lead, not a verdict. The editor (and that's me, the owner of all the flaws and mistakes in all of this) makes the call.
• Anything incomplete announces itself. If an obligation didn't survive the pipeline, the draft wears a banner saying so. Nothing gets to look finished when it isn't. (This rule has a body count; more on that below.)
• Then I review it (because I am, again, the editor). Every claim, against the order, rewriting as much as it takes. And when I edit, my rewording goes through the same meaning check the machine's did. Nobody skips the line, including me.
• If the Federal Register amends an order's text, the summary gets pulled automatically until a human (it's still me, I'm still the human despite an over-subscribed supplement regimen) re-reviews it against the new version.

I wanted trust to be a simple interaction: a mouse click or a thumb tap that exposes the proof immediately, instead of a badge you're asked to believe. Making that happen meant we had to create a lot of hoops on the backend, then I doused them in kerosene, lit them on fire, and proceeded to attempt to jump through them.

By Now, We're All a Little Fatigued By "How I AI'd This Thing"

I thought I was going to be brief. It's new for me and a real struggle, and I sincerely had your best interests in mind. And then I started checking my receipts and realized I needed to tell you how I AI'd this thing.

The division of labor followed the framework I wrote about in "Apps at the Speed of Need": I made the judgment calls, Claude turned them into plans committed to the repo, Claude Code built from those plans, and Qwen generated content mapped to the guardrails we provided while I kept my hands on the gates. Nothing published without my approval, nothing got approved without evidence on the screen, and there was more content and prompt editing than you care to hear about.

There is a small sidebar here that's possibly uninteresting, however, it is truth in lending, and also just the way I work: I don't blindly direct Claude Code to "go do this prompt, robot" and that's very intentional. Given the harness I've built, and given that Claude Code knows the infrastructure and can see things that I have partly walled off from the Claude desktop app (this is just a preference), I've set this up as a 3-way collaboration to ensure we're thinking as big and broadly as we can without over-directing on how to develop and deliver within the infrastructure. It adds extra steps as we work through challenges and refinements together, and the differences in when either of them compact (run out of memory and summarize themselves) allow me to personally hold more in memory and check the plan just a little easier. This feels like I'm probably doing something wrong, or missing some efficiency that someone else has solved, however, it works for me!

One of my favorite (though not so awesome at the time) early moments was a failure. The verifier flagged 8 of 31 summaries as ungrounded, and for an hour I thought the model had been inventing quotes, which meant I had to read, in detail, more executive orders than I really wanted to. My investigation showed the opposite: the Federal Register's own formatting, page markers and line-wrap hyphens, was breaking the matches. The pipeline was more verbatim than its source and the only way to learn things like that is when the system is built to check itself.

Also for the record: the first version of the front end earned an F+ from me, largely because it matched my sketched wireframes. I slowed my own process to invest time in branding and sensible design elements, not allowing myself to get lazy and bring you an AI purple or beige monstrosity that looks like all the others. It's a green monstrosity that looks like all the others, however, it's mine.

Taste still has a job, and I'm Rick Rubin-ing my butt off over here.

And I'm telling you about this failure and the F+ because they were, in hindsight, the fun ones. The first one taught me the system could catch its own mistakes, and the second one taught me I still had a design job. What I didn't know yet was that the next three weeks would be a parade of quieter failures. Those were the kind that don't flag anything, don't look like anything, and sit there politely looking finished. Until I was in the role of editor, editing, and then finding one of those mistakes that makes you wonder if you're being gaslit. Then we fixed it, reprocessed 30 executive orders, and I started editing again so I could identify the next one.

Finding those, and the right way to fix those is what took the most time. While I was in pursuit of those receipts, I ended up finding out that all those failures turned out to be the same failure on repeat.

The System Knew. It Had No Voice.

The three weeks after launch were spent discovering, repeatedly, that the difference between a demo and a trustworthy tool is a long list of things nobody claps for.

I was in an editorial moment of frustration when that gaslit feeling hit, and I'm sure I loudly asked (typed) whether the model (Qwen) writing our plain language was ever seeing the executive order it was translating.

The answer turned out to be no.

The pipeline that fed it meaning, performed the checking, and passed the judgments had never been connected to the source of its own truth. Everything had been working beautifully in a test harness with a limited blast radius (see also: what looks good for a test of one may not scale to a test of many). There was a function with "testbed" literally in the name, and the production system had been routing around it the entire time.

And the best (worst, it was the worst) part: buried in the data was a flag that blatantly said as much. The system had recorded its own failure, accurately, into a field that nothing on earth ever displayed.

It knew. It just had no voice.

That became the theme of the next two weeks. A Regenerate button was on mute when it destroyed an evening of my editing because nothing warned me it would.

A choice log faithfully recorded every correction I made while nothing, anywhere, ever read it.

I had been told the system was learning from my edits, and there were plenty of times where I felt I was repeating the same substitutions across multiple executive orders, so I spoke up. And I learned that the system wasn't really learning so much as it was keeping a diary.

There is a difference between capture and learning, and I found it the hard way, at about forty frustrating edits deep.

We gave EO Translator a voice. I was getting tired of talking to myself, so I made sure that the button now tells me exactly what it's about to destroy and archives everything first. A defective draft announces itself with a banner instead of rendering politely and embarrassingly landing on a web page (though I'm sure some of my late night edits may eventually embarrass me).

And my corrections finally got wired into the drafter as teaching examples (distilled by my own hand and not some magic feedback loop), and I know it took because we tested it on obligations it had never seen: the exact governance details it used to drop on every single order, the "who must consult with whom, through whom" chains that plain language loves to erase, now survive without me. Grading that test is the closest I have felt to being a proud parent of a very large matrix of numbers. Wow, I really just wrote that. Dork.

Then I turned on the scoring and found out our readability rubric was, and I say this with love and a quote from Colonel Sherman Potter, "horse puckey." We asked the model to judge our content with seven questions that our persona would answer, and when we finally tested them against my own edited corpus, five of them were dead weight.

One question asked if the writing "avoids bureaucratic phrasing" and answered no to everything, including the plainest sentence in the corpus, because the sentence contained the Secretary of the Treasury, who is, to be fair, a bureaucrat.

The fix produced my favorite finding of the project: questions shaped like "Could a busy adult with no legal training explain this to a coworker coherently after reading it once?" measure something, and questions shaped like "does this text have good property X" get a cheerful yes for everything, including a 79-word run-on.

It's kind of wild; when you ask the machine to simulate a person it gets painfully honest, and when you ask it to grade an essay, it grades on a curve. Well, thanks, machine, now I am better at knowing how to grade you.

Every one of those bugs was the same bug wearing a different costume: something failing quietly and looking finished. Every fix was the same fix: give the failure a voice.

It's not like I wasn't planning for error states, or I was ignoring steps in a process. Validations and checks were in place; the voice just wasn't in the right place with them.

No, Really. Check the Receipts.

One more thing happened while I was down in the engine room (my office) being gaslit by my own pipeline: readers started showing up. I'd quietly launched the frontend for testing purposes and had made no announcement nor published an article like this. This was just Google quietly indexing pages full of the Federal Register's own vocabulary.

To my surprise, those readers skipped the flashy quantum computing executive order almost entirely. The handful of readers who found the site jumped into executive orders that were more focused on federal contracting, land access, and the financial system.

It turns out the people who go looking for an executive order in plain language are, disproportionately, the people the order obligates, and what they need is exactly the unglamorous stuff I'd been grinding on: the qualifiers kept, the consultation chains intact, and the "must" that stayed a "must consider." (Believe me: I know this is speculative with incredibly limited data, so enjoy the outlandish leap I'm making, however, at least give me credit for acknowledging it!)

All that fidelity work I occasionally (frequently) resented was always the end product.

Over 30 executive orders are live, browsable by topic, searchable, and syndicated by RSS. The next time a headline tells you what an order says, you've got a place to check: eotranslator.com.

I've got a pet project on the back burner that I'd kind of lost the thread on; I'd scoped it months ago, sketched the screens on paper, and built part of it with a coding agent before life pulled me elsewhere (and before the whiskey collection made me organize it). When I picked it back up, I handed the entire thing to Fable 5 (because why not give it a try - also: wow, just wow) and asked it to revisit what I'd built, rescope what was left, and get us realigned.

It did. (And again: wow.) It produced a multi-step plan across a sprint, that plan went to Claude Code running Opus 4.8 to build against, and inside the application itself, Qwen handles the generative work on my own hardware. By the morning the backend was fully tested and deployed, and I was frontend-ready.

Four different roles were involved in accomplishing that work, and three of them were models. I was definitely the fourth.

For most of my working life, there was a middle layer between the design work I do and any other function I needed handled when creating digital products, unless I taught myself to do it (which I frequently have). I needed someone to build the database, and then to connect to it and make the results display in the design. More recently, someone else built the software and if you needed or wanted to use it, you rented that middle layer, and those were the only two options (unless you had the rare capability to build it all yourself).

That middle layer of renting software is thinning fast. Now, with a single tool, anyone can write prompts that attempt to create apps. The catch, and it's the whole catch, is that the middle layer's disappearance hasn't made the work disappear. Being good at Figma never made anyone a good UX designer, and a subscription to a frontier model doesn't make any of us good writers, coders, or product teams. The functions required for success still need to be accounted for and the difference is that now we're the ones accounting for them.

After making this AI learning and exploration effort my second full-time job for the better part of a year, I barely put thought into who did what. My approach to routing has become instinctive. I scoped and decided and course-corrected, Fable planned and orchestrated, Claude Code reviewed, ingested the requirements, and delivered, and Qwen ran the tests that validated what the daily load would work like in production. Each did the job it was best suited for, if not perfectly. The handoffs happened without much ceremony, and maybe just a little copy and paste, and approving a lot of command-line prompts.

That division of labor wasn't new, especially when you take a look at any sort of software development lifecycle. I've been working this way through every app I've built this year and I've been circling it in these articles without landing it cleanly.

It's bugged me for quite a while, and I think I can finally name it.

Nearly Everything I Created Could Have Been a SaaS Pipe Dream

As a colleague told me recently, I've been burning the midnight token a lot lately, and the list of applications in my wake is admittedly a bit of a #humblebrag. From the coffee profile application to the spirits catalog to the workout creator application, the conference tools, the expense report tool, the audio and video creation tools, 5 Minute UX, The Feed & The Thread, Patterns and Signals, a couple of research tools, and on and on. I'm sure most of them could have been packaged, priced, and sold, or I'm at least sure that someone, somewhere, would have tried to.

And I get that. Most people are spending their own hard-earned money on inference for both building and maintaining, and those costs can get out of hand, especially when you're the founder and the investor and it's all the same wallet.

The thing is, though, I've been building software that fits one person exceptionally well, and doing it fast enough that the economics only make sense for that one person. These apps wouldn't have justified their existence five years ago. The cost of building them was too high to spend on an audience of one, and that cost has collapsed. In 1999, so many of these projects would have required special hosting configurations, a 1-2 pizza team, and a coordination of shared time to pull things together. The removal of those barriers changes what's worth the investment of building.

And, look, I get it: that doesn't sound very human-centered at all. That's not really the case; I don't know that I can even attempt to step away from doing the thing that I've been doing for decades. Anything with a publicly exposed interface still gets thoughtfully built, validated, and tested with real people, even a few begrudging friends. I can't switch it off. We've all got that one friend who's a little tired of being shown the next new project. I'm usually that first friend - the one with the annoying new project.

While I definitely think that what we currently see as SaaS likely isn't going to stay the same, I don't think it means that SaaS dies.

I mean, c'mon. I've heard that Design is dead at least 3 different times this year alone.

I do think it means SaaS stops being the automatic answer to "I have a problem, is there an app for this?" I think we're getting much closer to what I've frequently cited as "Apps at the speed of need" because more and more frequently, I've seen something that I've thought "Can I build that? I bet I can build that. Let me try that." and then before the morning is done I've got a knock-off NotebookLM Short Video Explainer MVP and I've saved a Vision document for future exploration.

So yeah, apps at the speed of need, and we each become our own small IP farm, and enterprises are heading in that direction, too. We no longer need the majority of subscription services when we can create what we need when we have the infrastructure in our clouds or in our basements.

Compounding Infrastructure

Every project I've worked on, from the first RAGs to the spirit collection, leaves new capability behind. Spirit Spy needed to read text off a curved label, so it got optical character recognition that's now available to the next project. Spirit Spy also needed web enrichment, and we pulled from research tools that had left behind search and retrieval. The design language, the deployment pattern, the local inference setup, and the data conventions have all been built up steadily since the beginning (and learned through some hard failures), so each app hardens a little more of the foundation.

Whatever the next project might be won't start with a blank slate at this point, and frankly, that'd be a failure on my part. It will start with a foundation built from everything the previous projects learned, added, or adjusted. That's an economy of scale, and it only grows. Last night's project deployed a non-trivial backend in an evening because almost nothing about it was new except the domain.

I'm noticing a small shift in my own language that captures what this is. It's still exciting to say "I built an app" from time to time, however, the framing that I'm starting to view this all through now is "I extended my infrastructure." The app is almost an emergent property of the platform underneath it. Through some mix of happenstance and unconscious planning, I've built the platform well enough that new apps feel like routine extensions of it. It's the point where the map and the territory start to meet.

Inference Architecture and the Division of Inference Labor

Twenty years ago we learned not to put every workload on the same server. Well, some of us did. I've been learning that for the better part of the past year, which is why I now do things like "fleet management" and have things like "network topology" that I seem to reference more than I'd thought possible. This is, of course, good hygiene if you're going to head down this path of local inference. It's also a thing you don't have to consider all that much if you're subscribing to frontier models and those do all of that thinking, balancing, and overall heavy lifting for you, and all at the price of an ongoing subscription that has limits.

The lesson that I'm learning now is not to put every workload on the same model, or sometimes not even on a model at all.

Some work needs frontier reasoning: the planning, the hard architectural decisions, and the orchestration. Some work needs to really know how to weave code together, how to test and validate code, understand design systems, work with compliance as part of the output, and commit it all. Some work needs a capable local model running cheaply and constantly: reading labels, extracting metadata, generating and refining content, and all the other things that fall into daily work that is essential, yet still, work that keeps you away from doing the more creative and expansive endeavors. Some work needs no model at all, just plain deterministic code that saves, validates, and syncs without a shred of intelligence, because intelligence would only make it slower and less reliable.

And above all of it sits a person deciding what good looks like. Someone who orchestrates the orchestrator, watches the watchers, and ensures that we stay the heck away from Gas Town.

Last night's four roles were exactly that: I was there for judgment and scope, a frontier model was activated for planning, a coding agent executed for delivery, and a local model was engaged for the generative load. I never once considered asking the expensive reasoning model to do the OCR, and I definitely wasn't going to ask the local model to rewrite the strategy.

The work went where it belonged. And that's the important piece here: I was choosing which inference made sense for how I needed the work to flow.

The Blueprint Outlasts the Build

I've noticed that since I've been working this way, the app stopped being the valuable artifact. Apps are great; I still love creating something cool and useful, regardless of whether it's just for me. That said, the infrastructure has become so much more durable than any single thing built on it, and the plan that describes the app is more durable than the code that implements it.

And that's why I keep sharing product requirements documents (PRDs) instead of dropping links to github repositories. A repo is a finished build that someone has to trust, inspect, and maintain in an environment they may not control.

A PRD is a blueprint. You build your own version from it, with your own tools, in your own environment, on your own hardware, and you adjust, reduce, or improve it as you go.

The code is disposable. The blueprint compounds, the same way the infrastructure does.

The Last Subscription

The most valuable thing I built this year wasn't any of these projects. It was the infrastructure underneath them, and the realization that I've been LEGO-building foundational blocks until connecting the next app became routine.

I've told myself a lot of times that in my career I'm a pattern detective, and in this case, that pattern is the point. Creating apps at the speed of need isn't magic, and it most certainly isn't a one-shot prompt given to a companion that miraculously cranks out a billion dollar business overnight. (Just ask all of your billionaire AI buddies. You don't have any? Me, either!)

Apps at the speed of need are what happens when you invest in a foundation that compounds and divide work across the right mix of people, frontier models, local models, and plain code. Get those right, and building a small piece of software that fits one person exceptionally well stops being a project and starts being an evening.

Maybe that means that the definition of what it means to be an enterprise is getting a lot more local, and quickly. Well, if that isn't just in-the-middle-of-writing-this-piece thinking showing up out loud, I don't know what is.

I'm not here to pronounce the death of SaaS. I'm definitely SaaS-averse, especially given what I see in front of me and how I'm working today. SaaS isn't the default anymore, and I don't think you'll get far trying to sell something built from a prompt and a design-tool subscription. When you're charging a monthly fee for something you spun up in an afternoon, without much thought to whether it's safe, secure, or meaningfully different, that isn't a business. It's not entrepreneurial. And you're not a founder.

For me, and possibly for many of you, it seems like the default is becoming: I have a problem, and I have the infrastructure to solve it myself.

This was a project that, by all accounts in this stepping stone era toward whatever AI is becoming, was pretty straightforward. It reminded me of the first time I really created a project in HTML, or the first time I animated something with interactivity in Flash, or that first time I connected ColdFusion to MySQL and ran a query against a database. These weren't monumental accomplishments, yet each one was a moment of clarity.

That clarity for me is that SaaS, in the way I suspect many people are pursuing it, is going to require a lot more than someone who can few-shot their way through a handful of disparate technologies and weave them into something coherent, because that's not all that special anymore. And I'm not here for that. I'm here to learn, share what I can, and push myself toward the next thing on my list of things I'm interested in learning.

With that, if you skip to the end, you'll get the sharing part: the PRD to build your own collection management tool using computer vision, OCR, and web scraping. I hope you do, and I hope it helps declutter some of your shelves, too.

After a trip that included a visit to our friends at Hanalei Spirits, I came home with six bottles of rum and dropped them into a collection that had been comfortably bourbon-and-whiskey-heavy until that exact moment. Or so I thought. It turns out I hadn't done a great job keeping track of what I actually own, which is dangerous territory. The shelf had tipped from "collection" into "wait, do I already own that?" It was time to organize and catalog what I had on hand.

That collection has grown a bottle or two at a time, picked up on trips and at tastings, each one a little more about the memory than the market. I don't collect as an investment; every bottle connects to a place or a moment along the way. It's the experience, really, and that's the part I wanted to memorialize and do a better job of keeping track of.

When I was looking at the app landscape, I found the usual stuff that makes me cringe: someone else owns the data, there's a pay gate with a monthly or annual fee, and there's no good way to easily show and share the collection, or just parts of it. I felt like what I wanted was a fairly simple record of what I own, without pouring time and energy into maintenance, and without adding the cost of a micro-subscription that gets used sporadically.

Like so many things lately, from coffee profiles to deep research tooling to automated heuristic analysis to custom workout programs, it was pretty obvious I already had the tools at home to build exactly what I wanted, without someone else's everything in the way.

So I built my own.

And so can you.

Read along if you're interested in the build, or skip to the end for a link to the PRD. The product requirements document is available so you can hand it to your own AI tool to build whatever you want or need, without relying on someone else's system, someone else's inference, and someone else's subscription fees.

Threading the Pieces Together

The infrastructure was already in place for me. This was less a quick win and more a matter of connecting things I'd already built. I'd used computer vision models before in a heuristics analysis tool, and I'd built web search and retrieval into both a deep research tool and a UX sentiment research tool. Reading a bottle label from a photo is essentially OCR, and pulling supporting detail off the web was a solved problem. All of the parts existed, the inference is local, and the work was wiring it all into a single connected flow.

Capturing a label was the logical starting point, however, labels on bottles pose a bit of a challenge: curved glass makes for imperfect reads. Factor in the occasional handwritten batch number, labels that wrap around, or worse, an angled label and it's pretty easy to trip up even a good vision model, so OCR alone was never going to be enough. To fill the gaps, I added web enrichment: search for the bottle, fetch the maker's own page, and pull in only the facts the sources actually state, with a list of those sources stored on every entry. That turned thin label reads into rich detail pages with almost no manual effort.

The refinement was less about hard problems and more about noticing gaps and subtle improvements that I could make as I went. Some of the default, and easy/most sensible sources are aggressively anti-bot and block scraping, so I had to find ones that welcomed the traffic, had the same or more information, and let me bring their information in to the system. A few rounds of testing the capture, making adjustments, and locking it down got data enrichment to a fairly robust place.

Then there were the photos. Bottles shot on a checkerboard mat in my basement, couch and wall art in the frame, look like exactly what they are: an amateur phone photographer doing his best to match size and angle with a messy room in the background. I used a local background-removal process running on CPU that knocks out the background and leaves a (mostly) clean cutout of the bottle, which the catalog floats on a styled backdrop that matches the page design. While it's not professional-grade, these are product-looking shots in seconds, again with next to no effort on my end, and just for a personal vanity project, which makes it all pretty amazing.

And that's kind of amusing to me, and says a lot about me, to me. I started this out saying "my needs are simple" and then I went and kept creeping on my own scope. Probably because I'm a designer. Most likely because I'm rotten with perfection.

The Fun Part (And It Was All Fun!)

Once all the pieces were more or less in place, the workflow was pretty simple:

• I take a photo of the bottle.
• The vision model reads the details.
• I make any quick edits, and save.
• Then I take additional photos of the back, the sides, or the neck (or anything else like a label, etc.), and save again.
• I tell the model to read all the labels, and review the updated results for edits, and save.
• If I'd like (and I'd always like), I run web enrichment, review the proposed edits, and/or elevate web enrichment metadata, and save.

Beyond adding any cursory notes, like where I purchased the bottle, if it's open or sealed, or any other notes I'd like, I've got a complete entry: high-quality details, cited references, a cleaned-up product shot, tags, and a published card.

And this all takes under a minute per bottle.

I'm a bit of a local inference / self-hosting nerd, so every model in that flow runs on my own hardware. The vision model that reads the label, the language model that extracts facts from the web, and the background removal that cleans the photos. I don't make calls to external AI APIs, and I don't pay a subscription to someone else's services, and I definitely am not paying per-token billing. The only cost is the hardware running quietly in a little server closet that doubles as a sump pump closet.

And one thing it never does, on purpose: it never captures pricing or collection value. That's the one thing the subscription apps sell that is likely of value to some folks, and it's the one thing I didn't want.

About the only thing I'm quite literally adding as I write this: a wishlist functionality so I can take a photo of a bottle in the wild, enrich it, and have it sitting in a tab reminding my future self that past me thought this was a bottle worth sleeping on. This works on the same foundation that the platform works on; I take a photo, read the label, enrich the content, and the information is now saved to my Wishlist. If future Russ gets financial approval to make the investment in the collection, he can promote that bottle to the live list, update the details (better photos, most likely), and then it'll be part of the collection.

You Can Make Your Own; I'll Help! (Or: Why I'm Not Selling This)

I'm increasingly convinced that SaaS has to be genuinely special to be worth building, and a personal spirits catalog isn't that. I don't want to maintain a database for other people, I don't want to keep scrapers alive as websites change, I don't want to referee whether someone prefers a different model, different fields, tasting notes, a sip log, or whatever else they'd want their version to do. And none of those are negatives at all - they're maintenance and operations labor that require me to be at someone else's disposal for their happiness when it comes to how they want to maintain their collection.

So instead of creating a SaaS app with only one user (me), or signing up to spend unlimited time maintaining a passion project on other people's behalf, I'm giving away the recipe.

I wrote a full PRD for Spirit Spy: what it does, how the pieces fit, and the specifics about reading labels and enriching from the web that will save you the fiddly parts. There's no code or repository to wrangle, review, or wonder whether it's quietly tracking you. You hand the brief to your own AI tool, in your own environment, and build your own version, on your own hardware, owning your own data. And if you want to log every sip along the way and turn it into a full-blown diary per bottle, I think that's amazing. For you!

Please review it first to make sure it's clean and fits your needs, or that it gives you enough of a shape to modify. Talk it through with your AI, change whatever you like, and republish with all the cool things you've added that make it better.

Let me know what those are, because I'm sure I'm forgetting things that could make my own collection cooler, and I am, after all, rotten with perfection.

None of this is wholly about spirits, either. This is about a collection, and the same framing works for just about any collection you care about. Swap the metadata and it does what you need. My comics are next.

Read the build-it-yourself PRD ↗ Download the Markdown ↓

Enjoy!

At the end of 2024 I bought myself a Fellow Aiden, a fancy pour-over machine that makes great coffee once you tweak and tune a complicated (to me) brewing profile. Previously, my coffee ritual involved either French press or a drip machine with freshly ground beans of some sort. I liked pour over and never loved the labor nor the engagement of learning it, and as such, I spent a few frustrating weeks fighting with the new machine before I built a custom GPT to help me figure out which levers to pull in order to get a coffee profile that was at all drinkable.

The Aiden gives you a lot of levers, and that was the whole problem:

• Temperature and coffee-to-water ratio, the basics any brewer thinks about (I lie about the ratio and run the default, because the machine won't let me dose as little coffee as I need to report).
• Bloom settings: whether to bloom at all, the bloom ratio, how long it blooms, and at what temperature.
• Pulses: how many pours the machine makes, separately for single-serve and batch, the time between each one, and the temperature of every individual pulse.

That last one, the pulses, is where it gets overwhelming to me. A profile isn't just one temperature; it's a different temperature for each pour, and changing one by a degree or two changes the result in the cup.

A year and a half after going back-and-forth with ChatGPT to create some really great profiles that met my tastes, I had everything I needed to build a real app from my Pinocchio experience: a pile of profiles, the experience to know what I wanted, and finally the infrastructure to run it myself.

And so can you.

From GPT to Chat for Coffee

Back in early 2025 I built a custom GPT to tune coffee profiles, shared it on Reddit, made a few profiles for people who asked, then pointed folks toward using their own AI instead, because, frankly, it seemed like it'd be easier for them to have that discussion and tweak to their own preferences. Plus, I didn't want to be someone's go-between to my GPT to tune a profile that might be too bitter, too astringent, etc. Someone built a polished public site that did the same thing (and better), which was great to see.

At this point, I didn't have the infrastructure yet, so I just kept an ongoing conversation every couple of weeks when a new bag of coffee entered the mix. I'd feed it the details of the new coffee and my tasting (and sometimes smelling) notes, and over a year and a half we got dialed-in enough that I could grab a new bag and land a cup I loved in one or two tries, working from little more than the roaster's notes and my own history.

And that system has been working fine, without any complaints. It works fine. It also lives on someone else's servers, learning my palate in someone else's cloud. It's only coffee, so it never bothered me much, but I still wanted to do a bit more.

The MVP Was Easy

I had months and months (over a year) of dialed-in profiles ready, and ChatGPT knows my taste preferences, so I had the experience of tuning by taste. I discovered that someone in the community had already worked out how to talk to the Fellow cloud API, so device access was not an issue; the current only way to add or remove a profile is through an iOS app that isn't always convenient. Someone else shared how to get MCP access to the Aiden, which was also helpful.

What I assembled was a coffee profile archival system with machine retrieval and placement: a library of every coffee I've tried, profile generation, push to the machine, and pull back into an archive that outlives the device's 14-profile limit.

With over a year of trying a lot of coffees, I had built up a lot of preferences that allowed me to also build a personalized Taste Model too: my preferences as structured data on a 0 to 10 scale, learning from every brew I log, and visible enough that I can correct it. It keeps Qwen and me pointed at the same target.

That part came together fast. The build was easy, because the hard pieces already existed.

The Testing & Refinement Was Less Easy

Getting the archive working was the easy part; the application was up and functioning in little time. Pages were showing, content was seeded from experience and from the Aiden itself.

When it came to teaching the model my taste profile, things got a bit more involved. I started out on a frontier model and I was moving to Qwen3.6 on local inference so there was a lot more training and testing to do in order to prevent regression into muddy and undrinkable brews. And I didn't want to start the new week ahead with trash coffee, either.

Previously, every discussion was in a chatbox and it's easy to assume that any AI is your pal, your buddy, and they just know and remember things. You kind of set it and forget it, and as such, my early generated profiles looked fine and made sense. I hadn't previously been committing much to memory because AI did all the work and I just filled in the blanks, shared how things tasted, modified and moved on with my life.

Now, with a new AI that doesn't have nearly the same capability (nor cost, for that matter), I found I had no way to know if the model was reasoning or just handing me back my own data on a web page. It looked good, and I was afraid to waste good beans, so I built a blind test.

I hid an existing coffee's existing recipe and provided the model only the coffee's character and my taste model. I was able to see how close that profile landed to what I currently brew, and if it was close without seeing the current answer then I knew it was learning something, however, if it drifted pretty far away, I'd know that I was getting confident guessing.

It was guessing.

The model bloomed hot, around 188 to 190 degrees regardless of the coffee, when my own history shows I bloom much cooler on naturals and fermented coffees to protect the sweetness. It gave me a four-pulse batch where I'd hand-tuned five, plus a few other variables that were off just enough to notice.

The reason was simple: the model had nothing of my taste to reason from, so it fell back on generic averages. So I gave it something. I fed it my own dialed-in profiles as worked examples alongside the taste model, so instead of guessing from the middle, it could reason from the coffees I'd already gotten right.

Then I tested it the hard way, across all 29 of my coffees, one at a time, hiding each coffee's real recipe so it had to predict a profile it couldn't peek at. If it landed close to what I brew, it had learned from the other profiles. Likewise, if it missed, it hadn't learned much of anything.

The numbers split into two stories, because my coffees split into two types: ones I dialed from scratch, and Fellow Drops (an opt-in coffee subscription) that ship with a published recipe I then used as reference and had GPT adjust toward my taste profile.

The coffees I dialed from scratch were the first story. Blind testing revealed that the model predicted my ratio almost perfectly, nailing my exact pulse count 78% of the time, and my bloom temperature within about 1.4 degrees. It landed inside four degrees of my real bloom 89% of the time, without ever seeing the recipe. That might seem pretty close and accurate, however, when it comes to fine tuning a pour over coffee profile, single degrees within a specific bloom can change the entire taste of the cup you're drinking (he said, sounding like a coffee snob).

The Fellow Drops profile modifications were pretty compelling. A Drop comes with Fellow's own published recipe, which is what you'd use if you had no tool at all, and if you went by someone else's definition of what makes a perfect cup of coffee (and in my personal opinion: a very, incredibly, over-the-top strong cup of coffee!).

Fellow's base misses my bloom by 8.7 degrees, my pulse count by nearly two, and my ratio by 0.84, when measured against my real profiles. The grounded model, which never saw the Drop recipe, beat Fellow's preset on every dimension. Fellow blooms hot, around 200 to 205 degrees. I bloom the funky coffees much cooler to keep them sweet: 25 degrees cooler on a Grape Soda co-ferment that I found gross, 21 cooler on a Rwandan natural. The model picked up that cooling habit from my real profiles and settled right where I prefer.

A model trained on my taste predicts my preferred cup of coffee better than the roaster's own recipe does. And, in all sincerity, that's not an insult to the roaster's own recipe; they offer a great starting point that everyone should tweak to their own preferences, and I appreciate the baseline they provide. The model means I don't have to have a tiny sheet of paper taped to my fridge telling me how much to adjust each new profile that I want to try, and it's really great to be out of the 1970s.

The tests were incredibly insightful and they really helped close the loop on what it means to fine-tune a profile. We went from "pretty much guessing" to "this is how you apply the taste profile and preferences to get to something consumable by Russ" after a few rounds of rigorous testing.

When I started with GPT a year and a half ago, a new coffee took two to four tries to dial in. By the end I was down to one to three, and I'd often skip the first attempt entirely and jump to GPT's "if this doesn't land, try this instead" suggestion to get closer faster. The current model lands that close right out of the gate, before I've tasted a thing.

I want to be transparent about my testing: my from-scratch sample is small, nine coffees, and I'm reporting it with its size rather than rounding it into a tidy headline. Leave-one-out is a proxy for a brand-new coffee; it's a solid approach and, it's still a proxy. Bloom carries a lot of per-coffee judgment that no amount of metadata predicts, so the model gives a strong starting point and the tasting loop closes the last few degrees.

Why I'm Not Selling (You, Anyone) This

It's built, and it's tuned to me, and I was implicitly selfish about this from the start. Yes, I could add user accounts and allow people to customize their own taste models, and I could probably get some neat traffic and continually find ways to add feature bloat. Or, I could open-source the code and let everyone participate, or tell me what I left out because of how they drink their coffee. I could put a subscription model on it and chase that one-user SaaS joke into something real.

I just really don't want to.

I like being able to just continue to make adjustments to this for me and for my coffee and let it sit as a standalone app just for me, much like the at-home gym app that I have helping me (and only me) make custom workouts.

My coffee taste is finicky enough that maintaining this for other people would mean either compromising my own setup or signing up to be responsible for someone else's morning. If I tweak how the model handles a natural process coffee because it improves my cup, I might quietly ruin someone else's and that's a strange kind of liability to take on for a thing I built to make myself a better cup of coffee.

Instead of building a SaaS with one user, and instead of providing a github repo with open source code that someone can give a star to or not, I'm giving away the brief.

I wrote a full product requirements document for Coffee Compass that includes what it does, how the pieces fit, and the hard-won specifics about the Fellow integration that will save someone days. You can read it online or download the Markdown to hand straight to your own agent. There is no code nor repository at all to wrangle, review, or worry about whether or not there's something deceptive or hidden or tracking of use or any of that.

You can take the PRD and feed it right to your own AI agent, in your own environment, and build your own version, calibrated to your own taste, learning on your own hardware or cloud compute that you use.

Please inspect it first, or at least have your agent inspect it, of course. You'll want to have it consider your environment and needs and get things setup the right way for you, which is maybe on a local device or on a website somewhere. Or maybe you want to be the open source person, which would be cool, too. That's entirely the point; you build it yourself to meet your own needs, so you know exactly what it does and what it touches.

A Closing Thought

Software is getting cheap to produce and easy to print on demand. Rather than give you code, I'm sharing the blueprint, which is the part that lets each person build their own.

If you've got a Fellow Aiden and an AI you trust, take the brief and make your own coffee compass.

Read the build-it-yourself PRD ↗ Download the Markdown ↓

I want to start by adding that this is partially in response to something someone else recently wrote, I think. I haven't been able to find the article, however, it was referencing that AI is a solo sport, or at least I think it was. That premise, or my (mis)interpretation of it as intending that AI is a tool you are better off using alone, and is one that isn't good for / built for collaboration, has been rattling around in my brain since I saw it (and memory is a fickle thing; I'm not claiming to remember it's whole intent). What's following is what I have been thinking about, while working through a project that felt largely solo at points.

When I say "solo" in this article, I don't mean truly alone. I mean working with AI as the only other presence, without other humans in the loop. That's a real mode, and it's worth naming so the rest of this lands clearly.

Where the Team Part Starts

There was a recent inspiration for an MVP that had a quick turnaround. I'm certain the drill isn't entirely unfamiliar to a lot of folks: someone said something that identified a specific need and felt like it had some urgency implied to it.

It started at the end of the day with what I think of as the proper opening move: brainstorming with a couple of colleagues in different contexts to work out what a possible solution needed to look like.

In the first pass, there were more questions than answers, a lot of "could it, should it, why would it" type of things, and I was busy with sticky notes sprawling across doctor-signature-quality notes and awful sketches. I'm going to go ahead and call this the Discovery phase. We were trying to land on the actual problem worth solving, not just the one I'd assumed was in front of me.

Then a second pass with different colleagues, this one much closer to requirements gathering. We worked through existing infrastructure and recent delivery approaches, identified what data already existed and what needed to be synthetically generated for the MVP, sketched a definition of done, and made a handful of other less-important, yet critical decisions.

Those two upfront passes were rapid for me, in that they possibly took less than an hour. There was a shorthand that existed with the colleagues that comes from time spent working together, however, they carry more upfront weight than people realize. In this case, I wasn't really thinking of it as collaboration; I was getting to clarity as quickly as possible in an abstract way that I know how to do.

What struck me was how easy it would have been to sit down with a prompt and start typing in an unstructured and wandering fashion in this AI-assisted world. It's kind of like jumping right in to Figma or any other "drawing" tool instead of spending time with pencil and paper sketching through the thinking so that tool time is less, and better spent.

Those pencil and paper moments, whatever form they take, are where the work gets focused before any building happens.

Where I Went Farther Than I Expected

After working through fairly high-level Discovery and Requirements sessions, I then did the thing that I think gets mistaken for collaboration more than it is: I checked in with ClaudeCode to gut-check my development and infrastructure environment. Folks, I'm a designer, and I've been tinkering with tech since my TRS-80 on a 300-baud modem dial-up was the escape hatch from small-town life, and I'm here to admit that I'm scratching the surface on what goes on under the hood in my own environment, at least on a lot of things.

(Don't get me wrong; I've got some solid security, monitoring, and change control systems in place, however, they don't stay locked in my memory as well as the UX work that I grew up in industry with. I frequently need a refresher, and that's exactly why my heavily-documented system and agents exist.)

I needed to look at previously delivered systems that were similar, yet also different enough to be useful comparisons. Claude Code & Nucleus (my Qwen-powered internal agent; think OpenClaw without all the drama and/or snake oil) know the environment well, and I'm meticulous about documentation to make these types of situations happen smoothly, and accurately, so I can make informed decisions, and quickly. I don't see this as collaboration. It's engaging my confident interns to report back with qualified facts and evidence.

Once I understood the tech infrastructure and had made some development decisions:

• Flask app sitting on Postgres, specifically Postgres with vector search, meaning the database can find passages by how close they are to the question, not just by matching words.
• Embeddings on one HomeLab GPU, the language model on another, which is just how my internal HomeLab is set up; a lot of workload is separated out and points of failure are limited as much as possible.
• Hybrid retrieval; I can filter on structured stuff like document type, entity, or a contract vehicle, AND rank by meaning.
• Word and PDF documents accessible end-to-end, so users can find what they need with full context in a single click.

I kicked off deep research sessions to identify the right structures for the synthetic data I was going to need to get this off the ground. That allowed me to not only identify the document types I would need, it also allowed me to fill them in with synthetic data, and create the documents. That led to data ingest, chunking, tagging, and readiness for use in this RAG app.

Then I pulled from the requirements to get tasks aligned and understand what could be automated with minimal oversight. Over the course of an evening, I had most of the context I needed to validate data, map and re-map a data ingest process, and get ready to wire together a local LLM-to-RAG querying engine.

That meant I was ready to start building a front end and testing how my prompting skills would deliver synthetic content. Within 24 hours of scattered moments of focus, I had a real MVP. (Yes, that's easy to see as one of those AI flex things. I'm not trying to wow you with speed; it's not the point of this article. It is simply the reality of the situation.) I had delivered something that ran on real (albeit synthetic) data and could get real (and this means absolutely real) feedback, beyond what a wireframe or a clickable Figma prototype could ever offer.

I want to be transparent about the caveat: I have AI infrastructure that's been documented and tuned for the way I work. A lot of people don't have that yet, which means a lot of people would not move this fast on this kind of stack work. I paid the setup cost for that infrastructure earlier so I could move quickly later, and that's worth calling out.

This was the part of the work where it felt the most solo. I went farther "alone" with Claude (and Qwen) than I could have ever gone before. I relied upon my confident interns for infrastructure readiness knowledge, and then I "vibe-coded" (which, come on, is a bit bullshit; I planned and orchestrated this) my way through a remote-access dungeon and came out the other side with something that worked.

Where the Team Part Came Back

Once I had the initial prototype loaded with our synthetic data, I shared it out to a more technical colleague to understand how a development team might extend or rebuild this in production. The choices I'd made, from Postgres because I like it, to certain stack decisions because they were good-enough placeholders, were now visible, defensible, and ready for critique. The artifact did most of the explaining, however, no credible designer ships off a link without an explainer, right? Right.

The colleague's final statement was "I can align with that." And yes, I asked, to be certain. I wanted to know this meant it was a feasible way of working, and that the outcome could be replicated on systems not exactly like my own.

That kind of alignment matters. It came after looking at something real, and the conversation we had was specific because the artifact was specific. That's the show-and-tell moment, and it lands differently when the thing actually runs.

The Rhythm

When I trace the shape of this, and similar non-Russ-specific projects, they tend to follow:

Teamwork (Discovery) → Teamwork (Divergent, Convergent Thinking) → Solo + AI (System Scoping & Planning) → Solo + AI (Building) → Teamwork (Critique & Review) → Solo + AI (Iteration/Revision) → Teamwork (Delivery, Re-Discovery)

Two team passes happen upfront. The first is a quick-turn Discovery to get focused on solving the right problem. The second is more of the Design Thinking variety, where the solution starts to become more apparent.

Then two Solo + AI phases happen. The first is infrastructure and systems scoping and planning, where I get clear on what's needed to deliver the outcome and what's already in place, plus any compromises or new considerations. The second is the build phase, where everything that was planned gets orchestrated across a variety of agents. Data gets defined and structured, organized and created, ingested and implemented so it all lives under one roof.

This sets up more teamwork; MVP review and critique, followed by another Solo + AI round of iteration and revision, which can be cyclical. One more round may not be enough when you're coming in hot like this. And then, of course, more teamwork with MVP delivery. This, too, can be cyclical; it may turn into more iterative passes, or it may turn into another pass at discovery to ensure the right problem is still being solved and that the outcome aligns with the expectation.

Before someone says it: yes, this skips usability testing. At the MVP-validation stage, the question is whether the hypothesis is even worth putting in front of a user, and once it earns a green light, users enter the chat. Something often needs to exist before it can be good, and it definitely needs to be good before production.

The solo beats in there are real solo beats. I was working solo; I work through processes I've used since I was just a baby UXer, bringing the thinking, sketching, and defining what done looks like, and then deciding what to bring back to the team. Those parts are mine, and AI doesn't write them for me. AI is there, and AI will love to tell me how great and load-bearing my thinking is, however, I feel pretty confident in my ability to understand this current model's ability to placate.

Why It Feels Solo When It Isn't

Maybe I've got that backward. Maybe it doesn't feel solo when it is, because AI can seem like it's a real companion, and it certainly can do real, guided work.

However, here's the thing I keep coming back to: the reason any of this can feel like a solo sport is that I can now go so much farther before I need, or maybe it's want, to check in with others. The cost to getting "just one more thing" completed is low, and not at all low at the same time (ask anyone trying to tweak a layout about what happened to those 5 minutes they promised it would take, from over an hour ago).

The check-ins on incomplete work that used to happen at every stage can now get collapsed. Maybe that's ego, or maybe it's the benefit of new capability. I'm certainly guilty of the ego part and showing off my completeness of thoughtful thinking at an MVP, just because I can. That said, I no longer schedule a meeting before I've started working. I no longer share a half-formed wireframe just to keep people in the loop, though I still happily "share before I'm ready" with anyone available or willing to sit through a rapid-fire review.

In the old way (you know, 2023-ish), I'd have checked in more frequently and with less to show across more time. The New Way™ eliminates a lot of that, without removing that this is still an MVP. The end state is the same kind of artifact, with much more systemic thinking and planning baked in by the time anyone else sees it.

There's also the matter of availability, and it might be the biggest piece. AI is there when I'm there. I can be awake on my iPad at 10:30 on a weeknight, hit with an idea, and act on it right then. I don't need a colleague to be awake or available in order to run "hello world" on an idea when it strikes. A human collaborator has a calendar, a life, and perhaps more important than a well-defined supplements stack: a sleep schedule. AI has none of that, so it's available exactly when I am (token availability pending, of course). That's part of why the work feels solo: I'm never waiting on anyone. And it's part of why AI feels like a collaborator: it's always there when I want to think out loud.

There's a second piece here that's worth calling out, because it's where the "we" language gets misleading. Human collaboration has divergence built in. Your collaborators can identify blind spots you haven't named, or say "wait, you should consider doing this other thing" without being asked. AI doesn't usually seem to do that, at least not yet. AI follows the direction I set, faithfully, and it executes on the threads I pull. It doesn't pull threads I haven't pointed at. Oh, the time it would save me if it did...

A recent example: I have an AI agent called Nucleus running on a Mac mini, tapped into a Qwen model on a different machine. I'd been complaining about Nucleus being slow for weeks (so much for all the Open Claw hype). AI didn't tell me "what if we ran a smaller local model on the Mac mini itself for quick inference, and routed bigger queries outbound?" I had to pull that thread on my own, and once I made the connection to a previous way of working locally, the answer was obvious and the work to make it happen was trivial. However, surfacing the idea was on me.

That's the difference between human collaboration and AI collaboration. Both are real, both are messy in their own ways, and only one of them can pull me off course toward something better, with intention.

When I say "we," and I do say "we," I mean my AI counterpart and me. That's true, yet it's a different "we" from the "we" I use with another human, and collapsing the two of them into one obscures more than it reveals.

It's Just Me, Myself, and AI

The hype gets this wrong; none of this is a revolution.

This might be how we work now, or how we work next, because we can, and because we can shape it to fit how we need to. Working this way is a bit like a favorite pair of jeans or a pair of high-quality shoes; it takes some breaking in before it takes the right shape. Off the shelf, it's just a tool, and it'll stay that way until you take the time to break it in. Broken in, it's something else.

The demand to explain and defend design decisions didn't go anywhere, and no magical skills.md files are going to change that. It shifts; it is shifting now, and will likely continue to shift more and more as we all make our way through this.

All of this "we" business is the part that makes it easy to conflate what happens when you're going it alone with AI in the sidecar. It can feel an awful lot like you've got someone along for the ride, and it can feel an awful lot like you've got a real-life collaborator.

And you kind of do. Until you realize that you also don't.

As part of my Advisory role for Rockford University's Generative AI for Value Creation Certificate Program, I get asked to provide feedback on prompts to help improve the program. This article stems from some of those prompts, updated to fit this context better.

This is one of those "lessons learned" articles, however, it's not one of those where I imply or insinuate that the way I am working is the same way that everyone else works. I know what I'm learning and what patterns I'm seeing about how I work, and how the team I work with operates. If this resonates with you, that's great! And if it's contrary to how you work, I hope you'll share so that I can continue to get better and identify better paths, too.

For me, 0 to 75% is the easy part of generative AI. Any one of us can do that with speed and ease, and for me, it feels like the "hello world" of our era. Yeah, it's empowering and addictive, and sometimes it means there's a glowing screen keeping me awake late into the night to get just one more fix in.

The problem is that at 75%, people tend to feel this is an okay place to stop in their effort, and it isn't. I spend 25% of my time getting from 0 to 75% of a solution, and 75% of my time traversing that last 25% to completion of any AI effort.

That last 25% is where expertise, iteration, and judgment do their real work. And it's also where a lot of teams quit, or think that they've landed on a solution that is good enough. And yes, I'm stating this based upon any number of those amazing claims you see in Reels, in micro-content, and heaven forbid, Reddit posts. We're all still looking for that one-shot fix to financial freedom empowered by our own personal Open Claw with a credit card that does business while we sleep.

The gap between what AI makes look easy and what shipping AI-powered work at scale actually requires is the thing we should be talking about.

Guardrails & What They Might Look Like

The examples that hold up at scale for me share one feature: I made the deliberate choice to push past the 75% mark, and built guardrails to force that to happen. The most instructive cases are tools where AI augments my expert judgment rather than replaces it.

The clearest version of this in my own work has been with heuristics. The fast path is to grab Nielsen's ten, drop them into a skill or a prompt, and call it a day. I've not gone that route, and I'm a bit mortified by the assumption behind it, which is that ten general-purpose heuristics need no additional work to be valuable in any specific context. I'm sure that those skills will produce output that sounds confident and feels like rigor. That is, after all, what AI does very well: sounds confident and feels like rigor.

The problem is that ten general-purpose heuristics applied to any interface, any flow, any product context tend to produce ten general-purpose observations. Error messages should be helpful, well, no kidding, and what does helpful look like? Navigation should be clear, and clear without context is again not very helpful. While these are good overarching heuristics, they're not actually evaluating the thing in front of me.

I've found that it's not too terribly difficult to create sub-heuristics that can map categorically to the Jakob 10, or even to Morville's Honeycomb, or to the works of others, which can really help with the focus and granularity. It also requires more setup than I wanted to do initially; mapping heuristics into functional categories and subcategories, building context about the product type, the user, and the decision an interface is asking someone to make, and treating the prompt like a curation problem rather than a search query. That's the 75% to 100% work, and it's mostly invisible from the outside.

The same pattern shows up in the content guardrails I've built for tools that generate content with human editors. There are topic boundaries, tone calibration, what will and won't get published, and how to handle the moments where AI assistance is helpful versus where it would actively hurt the work. Those guardrails exist because I trust AI to do exactly what I tell it to do, including the wrong things if I haven't shaped the frame carefully.

The guardrail is the expertise itself, embedded in the system before the AI ever runs. The curation has had to happen up front every time. When I've skipped that step, the quality degrades the moment I get into testing. If I cannot get it to a point where I don't think any of you would make fun of me, I walk back through my process and do the work I tried to shirk off in the first place.

Or, even worse, I get fooled enough to try and ship something at scale, and get pretty terrifically humbled somewhere around the 10th or more prompt revision attempt, when I start to realize that the only way to fix the issue is to rewind the effort and get the quality guards in earlier in the process.

Ask me how I know.

What UX Professionals Need to Understand About GenAI & What GenAI Leaders Need to Learn from UX

I hate click-bait titles and sections headers like this, and yet...

What I've learned, watching myself and the team I work with operate in this space: AI does a remarkable job of fooling the ill-informed that it's doing a great job. On topics where I'm shallow, the output looks great to me, and I have to fight the urge to ship it. On topics where I'm deep, I can see what's off the moment I read it, and I know how to tune the prompting and the evaluation to get somewhere better. Domain expertise is the calibration mechanism that I must rely upon, and without it, I don't actually know what I'm shipping. That has made me a believer in leaning harder into the depth of my own discipline.

And it's made me an even bigger believer in the importance of Discovery, and in pairing the people who can turn concepts into outcomes with the people who know what those outcomes should be.

The lesson going the other direction is one I keep running into when I'm working on delivery rather than evaluation. Delivery at the scale of 1 is very different from delivery at the scale of 10. This is the big one, and my theory is that a lot of people kind of quit, or at least stop here longer than they should, because they haven't considered what happens when that single output gets repeated and repeated and repeated. And then somewhere along the way, that third, fourth, or fifth output starts to get glossed-over, then eventually ignored completely.

What sounds good for one piece of content starts to sound formulaic, basic, and without real intelligence the moment scale moves to multiple. I iterate to reach a steady state of 1, and then I iterate again, repeatedly, to reach a steady but variable state across many. UX has been thinking about consistency-without-sameness for decades. AI-first work that skips that lesson ships slop at scale, and I've shipped my share and fought with the machine to figure out what I was missing.

I keep coming back to the same conclusion: neither side gets to skip the other's discipline. UX work without AI fluency under-leverages the moment. AI work without UX rigor mistakes output volume for value. I sit in both rooms, and most of what I'm trying to figure out lives right in the space where they overlap.

AI Slop & the Trust Environment

The tension is real, and I'd reframe it. The deeper issue I keep running into is that the volume of AI output, where calling it mediocre or AI slop feels generous already, is actively eroding the trust environment for everyone, myself included. I've said it before, and I'll say it again: prompting without context delivers the mediocrity of the internet.

What I'm watching happen is user skepticism climbing rapidly, and the cause is straightforward. People are encountering more and more content that looks competent on the surface and doesn't hold up on a second read. It looks competent enough that any random person can feel confident dumping a wall of text into a forum online, right before getting rightfully lit-up by the resident experts who are already tired of getting their expertise AI-splained to them.

The lagging measurable is that the upper quality ceiling is coming down. If I put in half effort with AI, I get terrible results. If I put in maximum effort, I might maintain what audiences have been accustomed to. That's an asymmetry I haven't seen enough people factor into how they think about AI-powered work, and it's one I've had to factor into how I budget my own time. It also, frankly, kind of sucks to know that because others are taking shortcuts the rest of us have to work harder to maintain what might feel like a reasonable standard.

The principle that has held up for me is something I'm quoting my supervisor on, and it's incredibly simple: verify, then trust. AI doesn't have all the solutions, and it won't fill in the gaps out of the kindness of its heart (it's why your billion dollar idea didn't magically Open Claw itself into your wallet just yet). It won't identify the strategy I should pursue, and it won't see the blindspots I'm overlooking, at least not without a lot of poking, prodding, and asking on my part. It produces plausible output and lets me decide whether to ship it.

My best work in this space has come when I've treated AI output as a starting point that has to clear an expert bar before it goes anywhere. My worst work has come when I've assumed AI cleared that bar by default. It doesn't, and I've found that the biggest mistakes happen when I've been too comfortable, lazy, or both.

Investing in the Final 25%

Four investments matter most for getting past the 75% trap, and I notice them being underweighted in many of the conversations I'm in, or sometimes just having with myself. AI makes it seem like there's a shortcut to be taken for everything we do, and I understand the hype. I also understand that this is a longevity game, and the temporary players won't have the stomach for it.

1. Domain expertise in the loop, not adjacent to it. The calibration mechanism I keep talking about only works when real subject matter experts are actually looking at AI output and saying "this is right" or "this is off." That's a fundamentally different job from data scientists evaluating model performance, and the two don't substitute for each other. When I've had domain experts inside the workflow (admittedly, many times this alleged expert is me), AI work has held up; when I've relied on technical evaluation alone, or assumed AI was checking itself, the results haven't been worth shipping.
2. Budget for the last 25%. Most AI initiatives I've seen, including some of my own, are funded as if 75% complete is shippable. They generally aren't, and it's challenging because this is an exciting time and getting something right is less exciting than getting something MVP'd for the first time. That dopamine hit you get the first time your "Hello World" happens is addictive (and your friends are already tired of your neat little incomplete idea and don't want to see it every step of the way from there, either). The plan needs to include the iteration cycles to get from 75% to actual delivery, governance of the content over time, and the discipline to resist the cultural pull to call it done early.
3. Scale testing as a discipline. If a solution works for one output or outcome, that tells me very little about how it works for 10 or 100. AI is amazing in that, when it sounds and acts just like us, we want every output to be uniquely the same as the last. I've learned to build in evaluation at the scale I intend to ship, far beyond the scale I can demo. The scale-of-1 demos are easy; they're also where I've fooled myself most often.
4. "Verify, then trust" as a governance principle. I quoted this phrase from my supervisor earlier in the article, and at the org level it has to be a real workflow requirement to mean anything at all. The question I want every AI-powered touchpoint to answer is: who verified this, against what, before it reached an audience? When the answer is "no one, by default," I know I've got a problem upstream of the touchpoint.

The organizations that get this right will pull away from the pack. The ones that mistake speed for progress will spend the next few years watching trust erode under their own output.

The 75% keeps getting easier. The 25% is where the work lives.

The commercial deep research tools are genuinely good. Perplexity, GPT Deep Research, Claude Research: they index hundreds of sources, synthesize with depth, and return results in minutes. They also send your query, your context, and everything they crawl to someone else's infrastructure. For most questions that's fine. For questions about your own systems, your own security posture, your own infrastructure topology, or for questions you just don't want to put out in the open, it's less fine.

That was the starting point for my exploration. I had no grand thoughts about building something better because, really, there's no way that I can. This was much more about being able to do this on my own hardware, with my own inference, in the privacy of my own environment, and I'd like to understand what it takes to get to a reasonable outcome.

Start With the Tool Itself

The first thing I did was ask Qwen what I was about to ask Qwen to do.

Let me unpack that a bit. I wasn't being clever in the least; the LLM knows what it is and its own limitations and who better to ask than the source itself? Before I even started to build anything, I wanted to understand, from Qwen directly, what would be the best prompting frameworks for deep research queries when asking it to handle the task. I wanted to understand from its knowledge of itself what constraints matter the most when working with it and how I should structure a research question to get the most useful results from it.

Qwen handed back a set of research frameworks, each suited to a different kind of question, and explained which search strategies would focus the results most effectively given the constraints of local inference. What I keep coming back to is this: there isn't just one kind of deep research question, and treating them all the same is where most approaches fall short.

Before building the intake, I used Qwen to help me design the intake. Before I started writing the prompts to get to the results I wanted, I asked it what prompts would work best for the desired outcome.

This is the same instinct as interviewing users before designing for them, only the user is the model I was about to deploy.

And make no mistake here, I don't think I did user research in the way we do user research - not by any stretch of the imagination. I don't know what the hell this was, however, I know that I was given information that I could use to fuel the next steps and the insight allowed me to do something that I wouldn't have considered otherwise. That was a bit of a level-up moment for me.

The frameworks Qwen identified weren't invented for this project. They come from research methodology: SPICE* for comparative questions, PICO for causal ones, PICo + Entity Extraction for exploratory ones, step decomposition for procedural ones.

Each maps to a different kind of question and drives a different decomposition strategy. A question like "should I use Tailwind CSS for this project" is a comparative question that needs setting, perspective, comparison criteria, and evaluation dimensions. A question like "how do I deploy SearXNG" is procedural and needs prerequisites, steps, and validation criteria. Treating them the same way produces mediocre results for both.

*SPICE: Setting, Perspective, Intervention, Comparison, Evaluation; PICO: Patient/Problem, Intervention, Comparison, Outcome; PICo + Entity Extraction (Population/Problem, Intervention, Comparison, Outcome)

The Intake Is the Work

Every commercial deep research tool has the same weakness: the intake is a text box. The human types a question, the robot generates sub-queries, it searches, it synthesizes. There's no question classification, no framework mapping, and no understanding of what kind of answer the question actually demands. To be fair, I suspect the commercial tools do some version of this behind the scenes; that black box is bigger than I have the ability to crack.

I built the intake as a design problem. The form is your standard text area input asking for the research question. I added another text area for context - Why are you asking this? That's good to know, no matter what, and we're not always driven to over-share like that, so this is a reminder. When the query gets submitted, Qwen classifies the question type and selects the appropriate prompt framework for the best results from the query.

And here's the cool part: Qwen asks for a contextually-relevant clarification or additional details for each part of the framework that could use bolstering to get the best results.

I mean, c'mon. How cool is that?

Qwen also recommends how many rounds of research are needed to get the best results and aligns the query to a research strategy that winnows our 100+ search engines down to the most-focused and best-suited for the query so as to not burn extra cycles on sources likely to give us repeated results or those that are not even in the same ballpark.

Qwen tells me: Here is what I understood. Here is what I inferred. Here is what I still need from you.

That read-back pattern changed everything. It moved the quality gate from the output end to the input end, which is where it belongs.

Working Within Constraints

Qwen 3.6 at 27 billion parameters is not a frontier model. (What's a frontier model? It's a model that is the most advanced, large-scale, and right out there at the bleeding edge) It has a practical context window, occasional quirks with output formatting, and it performs better with focused single-purpose prompts than with multi-constraint instructions. I don't see those as challenges to overcome when they're more like the constraints within the design brief.

Every decision in the pipeline was shaped by working with those constraints rather than against them. Query optimization transforms natural language sub-questions into search-engine-optimized queries before they hit SearXNG (a free, open-source, and privacy-focused metasearch engine for aggregating results from a variety of search engines), because a 20-word conversational question is a terrible search query.

Confidence scoring runs after each round so the system knows which sub-questions have enough evidence and which need another pass. Ongoing summary compression keeps context bounded across rounds while preserving citations and topic-relevant specifics. The research stops when confidence is met, instead of running a fixed number of rounds regardless of what the evidence says.

The benchmarking against commercial tools was humbling in the right ways. Their breadth is better, their model reasoning is deeper, and they index more sources faster with more sophisticated content extraction. And I will still use the heck out of them when the need calls for it.

And what I've managed to build that they don't have is full pipeline transparency, per-sub-question confidence scoring, privacy, and the ability to run 9-12 Qwen calls per session without a cost review. And for good measure, I added a Research Analyst that allows for semantic querying of the results (full transparency: this is a pattern I've invoked multiple times across a variety of projects and products so it's not like this is some great feat; more like a light-touch humblebrag, I suppose).

When inference is free, you optimize for quality instead of efficiency.

What It Produces

After 41 completed research sessions (which for what it's worth, took anywhere from about 8-20 minutes each and is an awful lot of time watching paint dry), the output is focused rather than exhaustive. A typical session returns a 2,000-word synthesis with 10-15 sources, per-sub-question confidence scores, a full audit trail from claim to source, and explicit evidence limitation disclosures where the research came up short. It flags unsourced claims, notes where sources disagree, and qualifies vendor-reported data as vendor-reported. And if you'd like, it'll show every aspect of the pipeline log for verification; no black box.

It's not trying to be GPT Deep Research. It's trying to be the best research a 27-billion-parameter model can produce on a single GPU without sending a byte to anyone else's infrastructure. Within that constraint, the design goal is the same as any other research tool: get the most useful, honest, well-sourced answer possible, and make every step of the process visible.

This started by asking Qwen how to get the best out of Qwen. It turned out that insight ended up levelling-up everything else I did in the whole project.

The material changed and the design process somehow managed to stay the same.

Every few years, something arrives that designers are told will replace them. Desktop publishing replaced the typing pool. The web replaced desktop publishing. Online advertising reshaped broadcast. Each time, the fear was real, the disruption was real, and the discipline survived by adapting to the new material while holding onto what mattered most about the work.

We're in one of those moments again. I've lived through at least one of the big ones, and this does feel different, perhaps intensely so. The stakes feel higher, or at least they feel closer, possibly because it's far easier today to hear about how disruption lands on real people.

The pattern is familiar enough that it's worth pausing to notice: the people who navigate these transitions well are usually the ones who understand the discipline deeply enough to recognize it in new territory, not the ones who either reject the new material outright or abandon the discipline entirely in favor of it. There is, and always has been, enough room on the lawn for the damn kids.

I've been working through the exploration and build of an AI-powered lesson production pipeline (see previous Show and Tell article for more details). The pipeline has fourteen stages, with twelve different prompts to power it, twenty-one evaluation signals to measure it across three quality gates, and a content calendar of over a thousand lessons drawn from purpose-built RAGs. At no point during the build did I think I was doing anything other than design.

It Starts With the Map

Discovery efforts help us understand the landscape and make sure we're solving the right problem. A lot of design engagements start by understanding what exists, which could be a content inventory, a systems audit, a stakeholder map, a variety of processes and workflows that make sure the work gets done. In AI-augmented work it's the same exercise with different nouns: what data do we have, where does it live, who owns it, and what can we do with it.

For my build, the systems discovery revealed something I suspect is true for more than a few organizations right now: the pieces were already there. They were just living in different systems, built at different times, and for different purposes.

The RAGs were already running: Books, Heuristics, Research Methods, Compliance, Conferences, Courseware. Feed & Thread (feedandthread.com) had already forced us to solve Run of Show complexity, the multi-block episode structure with intro themes, narrated sections, musical stingers, and conversational segments that became the foundation of the podcast template. Tugboat (tugboathq.com) had already solved automated publishing to Castopod. The Courseware knowledge base had Gagné's Nine Events, Bloom's Taxonomy, and ICAP theory indexed and retrievable. Qwen3-TTS was already rendering spoken audio from text. The one genuinely new piece of infrastructure was Remotion, a React-based video renderer we brought in specifically for this.

The question wasn't "what do we need to build?" It was "what do we already have, and what's the stitching pattern?"

I suspect a lot of organizations are sitting in exactly this position without realizing it. Knowledge bases, content pipelines, rendering tools, evaluation frameworks, publishing infrastructure, scattered across teams and platforms, built in isolation, never connected. The stitching pattern becomes the design work. Designers are well-suited for identifying the patterns that exist, seeing the connections between existing pieces, understanding what feeds into what, how output moves from one stage to the next, and where evaluation happens along the way.

And that's not a technical problem; that's systems thinking. That's what designers do.

The fourteen-stage pipeline that resulted from that discovery is shown in the diagram above. Every stage, every decision point, every failure path, and every recovery flow. I prefer to enumerate when I map for traceability, and because explicit scope is manageable scope. And a pipeline with undocumented stages is the AI equivalent of a product with no sitemap.

The Sketching

This was one of those cases where I realized I wasn't starting with a framework. I was starting with a question: Will this even work?

The first prompt I wrote for 5mUX (5mux.com) was something close to: "You are an instructional design expert. Create a 5-minute lesson about Usability Testing based on this source material." It was just about one paragraph of instruction with no constraints and no evaluation criteria. Just: give me something and let me see if the idea has legs.

This is whiteboarding and sketching with a Sharpie on a Post-it before anyone opens their favorite bloated middleware (Figma, etc.). The output is rough and unfinished, and that's exactly the point: We're testing the concept, not the execution.

What I didn't expect was how quickly the prompt sketch and the interface sketch would start happening simultaneously. The moment I was thinking about an automated pipeline, I was also thinking about what happens when it doesn't work perfectly, or what happens when I want or need to manually invoke the process out of cycle?

These things can happen constantly, especially early on, and in environments that aren't like my own and that don't have a dedicated prompt support person, the human-in-the-loop isn't just a philosophical commitment to keeping people involved and engaged; it's a real design problem. What do humans need to see when the pipeline routes something to review? What context do they need to make a good, informed decision? What actions do they need to take, does someone else need to be involved in the workflow via escalation? How do they get back into the flow when they're done? What happens to the current queue of work when something stalls or breaks?

Sketching the prompt and sketching the administration / review interface are the same activity. You (or at least I) can't design one without implicitly designing the other. At least not today. Not yet.

I ran the rough prompt against a handful of topics, read and reviewed the output, and continually refined, focused, and adjusted. From simple edits like "Be more specific" to "Expand beyond general knowledge and use examples from the source material" to "This section is about gaining attention; write it like a hook that focuses the learner."

Each adjustment was a Post-it on the whiteboard: a direction, not a specification. I was using synthetic data and was working pre-API, pre-evaluation, pre-everything except me, the human, with a prompt and a question that I wanted to get answered as well as I could.

The "Hello World!" moment was seeing a generated lesson about Think-Aloud Protocols appear and thinking: this is teaching the concept! It's rough, needs work, and the direction and concept are sound. And then being able to pull that lesson's content into an administration view, read it in context, make a manual edit or kick-off a regeneration of the content, and then watch it move forward in the pipeline: that was the moment that the human-in-the-loop stopped being a concept and became a designed experience.

Both of these elements mattered and were of near equal importance. The prompt got us to content that actually taught what it promised to teach. The interface got us to turn a black box into something you could actually understand and reason with.

Complexity Has a Calling Card

The rough prompt worked for one lesson. As I've seen multiple times with content creation, a pattern of one does not a pattern make, so of course, when I tried to make it work for fifty, it fell apart.

The narration starts to become formulaic and generic, sounding like any model summarizing a topic rather than an expert teaching from specific source material. Opening and closing sections had the same register as core content sections, which made every lesson feel flat and sections repeated the same phrases across different topics. The word count varied wildly which made the promise of five minutes start to feel pretty far away. And there was no way to know whether the output was worthy of an audience; "does this feel right?" is not a quality system.

Vibe coding is one thing, vibe quality is an entirely different thing altogether.

So I started researching and adding rules. At the current tempo and temperature settings for my voices, 190 words per section landed closest to the five-minute target. That became the first hard constraint:

• "Target 190 words."
• "Anchor every paragraph to a specific detail from the source material."
• "Vary your sentence openers."
• "This section is tagged as Gagné Event 5: Provide Guidance; the entire section must be examples and demonstrations, beyond the opening."
• "Use contractions naturally."
• "Spell out acronyms with periods."

Qwen (my local LLM) tends to treat negative instructions as patterns to follow, so I've learned to reframe restrictions as positive directives instead. Specificity, along with that reframing discipline, quickly shifts a prompt away from a handful of brief sentences to beyond 400 words of instructions, and without much of a formal shape. Rules started piling on top of each other in the order I'd discovered I needed them. The output improved with every rule I added, however, the prompt was becoming unmaintainable. If I wanted to adjust the variety constraints, I had to find them buried between the word count rules and the TTS formatting instructions. If I wanted to know why a particular instruction existed, I had to remember the failure that prompted it.

That was the moment where the complexity of what I was building started to exceed what unstructured instructions can manage, and the mess itself becomes the signal that you need sense and structure. Any designer who has looked at a research wall covered in sticky notes and realized it needs a synthesis session knows exactly what this moment feels like.

Frameworks as Lenses, Not Prescriptions

I keep a working list of about a dozen prompt frameworks that I like and that fit most of my general needs (I use a separate set for research and that's a different topic altogether). RACE, CRISPE, Chain-of-Thought, RTF, TCREI, APE, CARE, and a few others. I don't start out prompting by choosing a framework - though, really, after a period of time in working within the frameworks you start to inherently pick up the overlaps. That said, I start by sketching out what I'm trying to accomplish, the same way I'd approach any design problem. Once I start getting something rough on paper I can start to see if the concept holds. The standard overlaps happen naturally: most sketched prompts already have some version of a role and some version of context before you've thought about structure at all.

Once I get from that sort of "Hello World!" moment and start to have more fidelity, more data, and more understanding of what the prompt needs to do, I step back and look at what I've already written. The question isn't "which framework should I use?" It's "which framework does this already resemble, and where are the gaps?"

That's where the RACE (Role, Action, Context, Execute/Expectation) recognition happened. I looked at the Script Generation prompt I'd been iterating on for awhile and realized it already had a role, an action section with five distinct constraint systems, a layered context section, and an expectation layer. That's RACE. I didn't choose it; I discovered it in the work I'd already done.

Once I saw it, I could organize the instructions by element, put role instructions together, give the constraint systems numbered subsections under action, layer the context sources intentionally, and group the expectation rules by purpose. The prompt didn't change behavior at all; it merely changed its shape and structure. And the shape made everything that came next faster, because when you need to add a new constraint you know where it goes, and when you need to debug a quality issue you know which element to examine.

I reviewed the other prompts in the pipeline and found a similar pattern: each one had already gravitated toward a framework without choosing one deliberately. The evaluation prompts were Chain-of-Thought (for more complex/multistep reasoning through logical steps). The structure analysis prompt was CRISPE (Capacity/Context, Role, Insight/Instruction, Specification/Style, Parameters, Examples). The source extraction prompt was RTF (Role-Task-Format). Some were simpler, some more complex, and a couple needed enough modification that they were effectively their own pattern.

Seven frameworks across twelve prompts, and none of them prescribed in advance. The framework fits the task, the task doesn't reshape itself to fit the framework. If nothing fits, you roll your own.

Design systems and templates work the same way, or at least they should. They're guidelines and you get to choose what fits when it exists, and you add something new to the system when it doesn't.

Refine, Refine, Refine

I ran somewhere between twelve and twenty lessons through the full pipeline before I shipped a single one. This was all synthetic data, intended to mirror the actual process, with source material being generated from placeholder content, real prompts, real evaluation, and real audio and video rendering. The pipeline was connected end-to-end and the content was disposable.

It should surprise no one that this is what prototyping looks like. And like every good prototype, it existed to be thrown away.

The first version of the evaluation system sent all seven quality signals to a single prompt and asked for holistic verdicts. When the model returned malformed JSON, which it did regularly, all seven signals collapsed to zero. Every lesson scored exactly zero regardless of quality. The good news was that the evaluation system was running; it just wasn't evaluating anything.

The fix was architectural and played to the strengths of the systems in place: separate what code can measure reliably from what genuinely requires language understanding. Code handles eighty percent of the score through embedding similarity, verb classification, readability metrics, and repetition detection. The model answers focused binary questions for the remaining twenty percent, one at a time, with narrow context windows. If the model fails entirely, the code signals still compute and the lesson routes to human review instead of silently passing.

Three calibration rounds followed, each surfacing a specific failure mode that needed a targeted fix. Some were prompt adjustments, some were architectural, and some required Claude Code to dig into the evaluation logic in ways I'll freely admit were beyond my depth. What mattered was that each round produced measurable improvement, and we documented every change and the rationale for it.

Then I wiped out everything that had been generated; every synthetic lesson went from finely produced straight to the recycle bin. The pipeline was reset with all of the real lesson material and extraction prompts and was ready to go, and it was time to start over with real data now that all the bugs were hammered out.

Except, and this is the part that gets skipped with what I suspect is more frequency than it should: I built the pipeline to prove the pipeline worked, calibrated the evaluations based upon the synthetic data used in the prototype, and then reset, and started all over again, with the real data.

And I'm here to testify that running the pipeline again still found additional bugs to squash; some of the automations were off and the evaluations weren't triggering correctly on the real content, so more adjustments were made, and the pipeline torture continued until my morale improved (which was about a half dozen additional run-throughs).

Documenting the Work

Here's the part I don't see happening anywhere near enough in the AI space right now: people documenting their prompts the way designers document their decisions. And I get it: we're in a bit of a Wild West phase of it all. That doesn't erase the need for the rigor, however.

A prompt that exists only in someone's head, or in a chat window, or in a file with no context around it, is the equivalent of a shipped product with no design system, no QA plan, and no handoff docs. It works until the person who wrote it is unavailable, or until something breaks and nobody knows why, or until the next person who needs to work with it has no way to understand what it was trying to do.

The pipeline I built has twelve prompts. Every one of them is documented as a handoff artifact: what it does, what framework it follows and why, what each element of the framework contains and what failure mode each constraint addresses, what signals evaluate its output and what thresholds gate its progress, and a version history that traces every change back to a specific evaluation result revealing a specific problem.

To make this concrete, here's what that documentation looks like for Script Generation, the core creative engine of the pipeline.

Script Generation: RACE Framework, Version 6

The Script Generation part of the pipeline produces approximately 190 words of spoken narration per lesson section. It runs four to six times per lesson and is the highest-iteration prompt in the pipeline. Its output feeds a text-to-speech engine, which means every formatting decision affects audio quality directly.

The Role is "a technical educational content writer who teaches from extracted specifics." The phrase "extracted specifics" is load-bearing: it prevents the model from generating from general training data and forces grounding in the provided source material.

The Action section contains five constraint systems, each traceable to a specific failure discovered during development.

• Extraction-first grounding prevents hallucination: every paragraph must anchor to specific details from the source.
• Variety and freshness prevents repetition: each example used once, sentence openers varied across the section.
• Gagné event alignment prevents generic instruction: the tagged pedagogical event shapes the entire section, not just the opening sentence.
• Key point coverage ensures evaluation signals pass: every key point appears as substantive content.
• Objective language preservation creates a throughline from structure to narration: the exact phrasing from enabling objectives appears in the scripts so the listener hears the vocabulary they're expected to learn.

Context layers four sources in intentional order: extracted specifics first, raw source material second, supplemental RAG retrieval third, and the previous section's script fourth, which serves double duty by enabling narrative continuity and preventing cross-section repetition.

Expectation covers TTS-optimized formatting rules, word count boundaries, and a format constraint that is plain spoken text only, no JSON, no headers, and no metadata.

The version history is the governance record. Version 2 fixed hallucination through extraction-first grounding. Version 3 fixed repetition through variety constraints. Version 4 fixed robotic TTS output through formatting rules. Version 5 fixed disjointed transitions through continuity context. Version 6 fixed objectives missing from narration through language preservation. Every change was motivated by data, and every motivation was documented.

This is prompt governance. It's not glamorous and it's not optional if you're building something that needs to be maintainable, extensible, or transferable. The prompts are part of the design and they deserve the same documentation discipline we'd apply to any other design artifact.

And to summarize: When providing formal documentation, I highlight the purpose of the prompt and the intended outcome. I explain each section (in this case each of the elements of RACE) in plain language and with a rationale, if necessary. And then I provide the prompt. Depending upon the need, I have plenty of room and liberty to provide more details along the way.

Would I do this for every case when I'm working alone on a side project? Probably not, if I'm being wholly transparent. Because documentation is less fun when you're also the client. Except that working with Claude Code makes it inherently easier, which shifts that probably not toward probably yes. Especially when I want to return to a prompt weeks later and regain context quickly.

Design Is Still Design

I want to be transparent about what I'm trying to say here, because the AI conversation is full of people being indirect about it.

Designers being told to just use AI without direction, watching their organizations adopt tools nobody understands, uncertain what their role looks like in AI-native work: none of them need more theory. They need to see what the work actually looks like when someone does it with rigor and documents the process.

I don't think that I've got this perfect, and I certainly don't think that what I'm doing is necessarily the only way, nor entirely the right way. There are people far smarter than I am at this, and I'm sure there are people who are annoyed that I've done this at all. I get it, and I understand, and I know that I learn a lot more by doing than I do by speculating.

This is what it looks like to me, for me, right now at this moment in time. I sketched until I had a "Hello World!" moment and then continued adding complexity and let the mess point me toward structure. I applied frameworks as lenses, not prescriptions, and discovered them in work I'd already done rather than choosing them in advance. I prototyped with synthetic data, calibrated from the results, threw away the prototype, and started over with real data. I documented every prompt as a handoff artifact with version history and rationale.

That's an awful lot like the design process I'm familiar with: Diverge and converge, sketch and prototype, map the territory and identify the patterns, apply structure, deliver with confidence (though am I at all confident? I don't think so; sharing this is incredibly daunting).

The pipeline is the design, if not the map to the design. The prompts are the wireframes, and every step of a pipeline can have a UI that needs to be available to investigate and interrogate what's being invoked, so the black box of AI isn't a mystery and can be verified until it's trusted. The evaluation signals are the quality criteria, and they need to evolve as the volume of content increases and new patterns emerge. The framework taxonomy is the design system. The version history is the changelog.

This hasn't felt like learning a new discipline, at least, not really. There are new techniques, certainly, and it requires applying the same skills we already have to material that feels unfamiliar until you realize it isn't. It's a new substrate, that's all.

The interface may be shifting, and that's not new. Voice interface design has happened longer than you can say, "Did you say 'Call Whole Foods?'" Interfaces have shifted before and we've navigated desktop publishing to the web to Flash (long live Flash!) to application-based interfaces on devices with different viewports galore.

What hasn't shifted is the need for someone who can look at a complex system, understand how the pieces connect, define what good looks like, build a way to measure it, and document the decisions well enough that the next person can understand and extend the work.

That's still design. It has always been design.

It seems to me, in my quite limited view of all of the internet, that every week or so someone asks how they can get up to speed on UX quickly. The answers are generally the same: read this book, take this course, follow these people. Good advice, generally. Also advice that assumes you have the time and attention to absorb a full curriculum before you need to apply any of it.

I've been on the other side of that question for most of my career, as an author and co-author, a conference speaker, a workshop facilitator. I know how hard it is to make practitioner knowledge transferable. And I've watched a lot of content, including plenty of my own, get consumed without landing.

The field isn't slowing down for anyone to catch up; it never has and it never will. And since I had the materials, the curiosity, and the infrastructure, I realized the only thing missing was the pipeline to put it together.

And that's the constraint I've been building around.

And the part I forgot to add, the lede I apparently buried: while creating 5 Minute UX.

The Corpus was Calling from Inside the House

Early last year, I'd already been torturing prompts long enough to know the ceiling. The results from general models were shaped by general knowledge, and no amount of prompt engineering was going to change that. I recently typed that "Prompts without context yield the average of the internet" and that is indicative of the experience I've had.

What I wanted was a model that knew my material, specifically, not an approximation of the field I'd spent decades working in. So I built a RAG from content I'd co-authored across six books and an unfinished manuscript, along with conference talks, workshop materials, and evaluation frameworks accumulated over years of practice.

Getting the materials together was its own exercise and ultimately the easy part. Getting that content into a formatted output so I could actually find it useful was the hard part. The paid models already do that for you behind the scenes and make it look inherently like that's just how it works. It's not.

Once it worked, something shifted. I could quick-ask a question and get a coherent response grounded in something I'd actually written and not a vague or hallucinated summary of someone else's blog post. I was getting answers with citations back to chapters I remembered writing and content I'd done the research on or had the lived experiences with. That was incredibly useful on its own.

Then the second lightbulb: this content had been used to teach people. Some of these books have ended up in collegiate settings, which is beyond pretty cool in and of itself and something I generally don't think about. They'd never been fully orchestrated to facilitate learning, beyond creating some light touch instructor guides with learning objectives.

A book teaches. I've been turning those chapters into lessons with clear objectives, a structured progression, and guided examples, followed by practical application. Adding in the ability for self-assessment is something similar enough, yet different for folks learning on their own and not in a workshop setting. I had the raw material at my disposal and it was a privilege and an opportunity to build the pipeline to extract and format it all in a formally structured way to facilitate learning.

Knowing Your Content Isn't the Same as Knowing How to Teach It

I've presented at my fair share of conferences and facilitated plenty of workshops across my career, including a few that deserved better orchestration than I gave them. Which means I understand, maybe more than most, that knowing your content isn't the same as knowing how to teach it. When I started automating lesson production, I didn't want to rely on what I thought I knew. I wanted something I could measure against.

That's where Gagné's Nine Events of Instruction and Bloom's Revised Taxonomy came in. Both have decades of evidence behind them, which matters when you're automating decisions that would otherwise require human judgment.

Gagné's Nine Events of Instruction is a bit of a workflow that mirrors how people learn: gain attention, state the objective, activate prior knowledge, present content, provide examples, elicit practice, give feedback, assess understanding, and support transfer to real contexts. Every lesson section is tagged to one of those events, and the pipeline generates differently depending on which event a section serves.

Bloom's Revised Taxonomy provides the cognitive ladder: from Remember at the bottom through Understand, Apply, Analyze, Evaluate, and Create at the top. Every enabling objective is classified by level, and the system verifies the lesson climbs that ladder rather than bouncing around randomly.

Together they answer a question a lot of automated content never asks: is this actually teaching something, or just presenting information in a tidy format?

Local Inference, Not a Phone Call to Someone Else's Model

I want to be transparent about how this all works and works together. For starters, everything runs on local hardware and inference. Qwen 3.6 generates content on a dedicated GPU, handling RAG extraction, objective shaping, and script generation. Qwen3-TTS renders audio on a separate dedicated GPU, keeping inference calls from stacking up on each other. A third evaluation pipeline scores the output independently.

None of it touches an external API, and that's more than a cost decision, though keeping experimentation costs controlled is genuinely nice. It means:

• The quality bar doesn't shift because a vendor quietly updated their model.
• The evaluation pipeline isn't just the generation prompt turned back on itself. It's a separate system with different logic, different signals, and a different job: not to create, to verify.
• The content stays internal to my infrastructure and isn't used to train someone else's model.

I've been continually assessing and iterating on the evaluation system since day one. And really, that's the point. While I perform Discovery upfront and create my own Vision documentation, I also have done this long enough to know that in many cases, the definition of done needs to have a bit of a sliding scale to it. Independence from external APIs provides control over the system that controls the quality bar, and allows for ongoing iteration without concern for token use impact.

How the Pipeline Works

I did three passes through the full corpus of multiple RAGs and catalogued 1,047 extractable lessons across eleven UX categories, each one assigned a priority score, a lesson type, and an extraction prompt tailored to the relevant RAG. There are lessons focused on concepts, skills, judgments, and critiques: each type has a different pedagogical emphasis and a different generation approach.

When a lesson enters the pipeline, the extraction prompt goes to the relevant RAG first: Books, Heuristics, Research Methods, Compliance, Conferences, or Courseware, depending on the topic. The RAG pulls source material, significantly more context than a cold prompt would produce. That extracted content grounds everything that follows; Qwen generates from the source material, not from the broad approximation of the field that a general model carries.

From there, Qwen produces learning objectives, then a Gagné-aligned outline, then per-section scripts at a roughly guided 190 words per section. Each stage is shaped by the lesson type and the Gagné event tagged to each section. When the pipeline generates script for a section tagged "Provide Guidance," the entire section is examples and demonstrations. Not just the opening sentence. Every paragraph.

The Evaluation Gates

There are three evaluators, twenty-one signals total, assessing everything generated. The Structure Evaluator checks the outline before a single script word is written:

• Do the enabling objectives support the learning objective?
• Does the lesson progress upward through Bloom's taxonomy?
• Does every objective have a section teaching it?

The Script Evaluator checks whether the narration delivers on the structure's promises:

• Key point coverage: are the promised points in the scripts?
• Objective delivery: does the narration deliver on every enabling objective?
• Variety: does the lesson avoid repetitive phrasing and monotonous sentence patterns?
• Gagné alignment: each section fulfills its tagged pedagogical event?
• Narrative flow: reads as coherent progression, not disconnected blocks?
• Readability consistency: consistent reading level across core sections?

The Assessment Evaluator checks the quiz questions:

• Are correct answers grounded in the lesson content?
• Are wrong answers pedagogically useful?
• Does the difficulty distribution reflect the right cognitive mix?

Seventeen of the twenty-one signals are deterministic code, things like embedding similarity, verb classification, word count, and n-gram repetition detection, which means the bulk of the scoring happens through logic that doesn't change based on how the model is feeling on a given Tuesday. The remaining four signals use Qwen answering focused binary questions, one at a time, with narrow context windows, and the LLM never exceeds twenty percent of any evaluator's weight. If you run the same evaluation twice, you get the same score.

Lessons scoring above 0.70 auto-publish, and anything below routes to a human-in-the-loop review, which in every case means me. I'm the human and I get to decide what happens next based upon what failed and why. A structure problem might get a prompt adjustment before the lesson regenerates, while a script problem might get a manual edit followed by re-evaluation. A Gagné alignment failure is usually the most instructive of all because it tells me something specific about what a section was trying to do and fell short of. The diagnostic output is in plain language so that the fix, more often than not, becomes obvious.

Every evaluation surfaces on the lesson page organized around three questions: does the lesson teach what it promises, is it structured for effective learning, and do the quiz questions test the right things. The scores are a weighted composite of all the signals, broken down by category, with per-section Gagné alignment shown individually so you can see exactly where a lesson succeeded and where it needed more work.

If you want to understand why a lesson scored the way it did, the answer is right there waiting for you.

From Evaluation to Episode

Once a lesson clears evaluation, the production aspect of the pipeline runs. Qwen3-TTS renders audio across two synthetic voices, audio assets are rendered and layered according to the Run of Show, our episode template, and a Remotion renderer produces 1920x1080 video with word-by-word captions synced to Whisper timestamps. Every lesson closes with The Take: a two-voice conversation between a Host and a Learner, both practitioners, working through the implications of the concept together. It's the kind of conversation that happens after a good workshop session, including, more often than not, a genuine question about where AI fits into the practice being discussed. Research calls the underlying technique the Muller effect. I'd argue it's the part of the lesson people actually remember.

Every lesson publishes across three modalities: a full transcript with key points per section for those who prefer to read, a podcast episode for those who listen on the go, and a video for those who learn best by watching. The goal is to meet practitioners wherever they are, not to pick a format and hope it works for everyone.

Lessons Learned from Inside the Pipeline

The pipeline was fully functional, in parts and pieces across my own current infrastructure, for months before a single lesson was published. I've been exploring and creating personal tooling to help with organization and efficiency, product-like projects that keep me learning, self-hosted applications and services to aid in the ability to continue to explore and expand my knowledge, and infrastructure that requires its own monitoring and maintenance to keep it healthy.

All of those pieces are a part of the cost of doing this with any real rigor. I needed the pieces from many different projects, reaching all the way back to that very first RAG and a glimmer of a vision, to get to the clear definitions of what I wanted to build before I could measure anything. Gagné and Bloom's gave me the frameworks to know what good actually looks like, and I ran the pipeline repeatedly on synthetic data, then on real content, then again after every calibration, until the confidence scores meant something I could trust.

The corpus that made all of this possible started coming together early last year, long before 5 Minute UX existed as an idea. The scraping, the RAGs, the intelligence feeds, all of it was already running. The AI amplified what was already there and brought it all together to match the vision. Even the related articles that surface alongside each lesson come from the same intelligence feeds I've been building for months, content from practitioners and community members creating work worth sharing alongside our own.

The name sets an expectation worth addressing directly: none of this is trying to teach all of UX in five minutes. The point is to give practitioners something genuinely useful in the time they actually have, grounded in real practitioner knowledge, built with enough rigor that we're not embarrassed by what comes out the other end.

And I'm still watching the queue today, still finding opportunities to improve, and still iterating.

I wrote this just a little under a year ago and it's never seen the light of day until now. I'm certain it's not all-inclusive, however, it feels more and more relevant with each passing day.

"AI First puts Humans First."

— Tim O'Reilly

We build and implement AI solutions across complex environments, from mission-critical platforms to customer-facing systems. In every case, we believe effective AI starts with people. If AI is to serve users, teams, and outcomes, then the design must center on usability, trust, and context from the beginning.

UX design has long relied on clear, intentional principles. These still apply, especially as we adapt to meet the evolving demands of AI-enabled systems. Clarity, consistency, usability, and control remain essential. Our updated design principles build on that foundation, adding the specific considerations that emerge when AI enters the equation and begins shaping interactions.

Transparency, timing, trust, and task alignment are not just technical or interface-level concerns. They are questions of orchestration, decision-making, and delivery. Our job is to ensure that AI supports value without introducing friction or disrupting flow. As AI becomes more deeply embedded in everyday tools and systems, design must adapt to meet users where they are and bring them forward with confidence.

This is a starting point: a set of UX principles for AI that reflect how we design, deliver, and validate human-centered AI in real-world systems.

Principle 1: AI Should Augment, Not Replace

"Keep the human in the loop, always."

AI should augment human capability and amplify output, not displace human responsibility. In many environments, AI is most valuable when it acts as a partner, offering insight, support, or acceleration with human expertise still guiding the outcome.

Human involvement should be clear, and when appropriate, controllable. Users may choose to stay fully in the loop, step in only at key thresholds, or delegate routine tasks entirely, with confidence that they can re-engage and review when needed.

Implication: Designs must support traceability, override options, and clear decision boundaries. In high-stakes or high-trust systems, human validation must be easy to initiate and meaningful to act on.

Principle 2: Inject with Care, Integrate with Purpose

"Disruption without design is just chaos."

Integrating AI changes how people work, make decisions, and interact with systems. These changes can be positive: faster insights, smarter tools, more adaptive workflows. These benefits must be introduced with intention, context, and design.

AI should be added when and where it improves outcomes; poorly timed or ill-fitting integration can erode user confidence and create friction in daily operations. Successful injection is thoughtful, aligned to the flow of work, and respectful of the user's cognitive and task load.

Implication: Treat every AI introduction as a service design moment. Onboarding should be intentional, minimal, and measurable. Users should understand why AI is present, what it does, and how to work with it, ensuring that they don't work around it or lose its benefits.

Principle 3: Trust is Earned Through Transparency

"Black boxes don't belong in critical workflows."

People trust systems they understand. It means providing users with the right level of insight at the right moment. Whether the AI is recommending a decision, prioritizing data, or summarizing content, users need to know where the output came from, how it was formed, and what to do with it next.

Transparency is contextual; some users want to see the logic every time, while others may only want detail when reviewing an edge case or validating a recommendation. What matters is that when someone needs to see the work it's easy to access, easy to interpret, and easy to act on.

Implication: Build transparency into the experience in ways that support user preference, decision confidence, and auditability. Showing the work, when requested or needed, builds trust and keeps humans able to support, override, or adjust AI outcomes with confidence.

Principle 4: Visibility Has a Time and a Place

"Sometimes users need to see the work to trust the result."

AI systems should make it easy to surface the reasoning behind a result whenever a user needs it. In mission-critical contexts or high-trust environments, that visibility is essential for backing up decisions, initiating review, or sharing accountability.

Visibility should be purposeful, not performative. Many users won't need to watch AI processes unfold in real time, however, they should always have the ability to access meaningful reasoning, data sources, or confidence indicators when they choose to. That access should be smooth, timely, and easy to interpret.

Implication: Design experiences so users can see the work of AI, without friction, when it matters. Visibility should align to intent and decision needs, and extend beyond system output.

Principle 5: Context is King

"Prompts without context yield the average of the internet."

AI is only as effective as the input it receives. Whether that input comes from structured data, freeform prompts, or real-time workflows, systems must be designed to respect and reflect the context of the task, the user, and the mission.

Helping users provide better input is a design obligation, not just a responsibility. Clear instructions, embedded examples, structured fields, and thoughtful defaults all reduce ambiguity and improve outcomes. Teaching users how to shape questions, provide signals, or guide models is just as critical as the underlying technology.

Implication: Design for intelligent input. Provide scaffolding, education, and interaction patterns that support better context and reduce noise while improving results. The more effectively we help users shape the ask, the more likely AI is to deliver useful, differentiated, and accurate results.

Principle 6: Design for Seamless Integration, Not Interruption

"Users want outcomes, not interfaces."

AI should integrate into workflows in ways that increase operational efficiency while avoiding added burdens. Whether supporting analysts, operators, or executives, AI should reduce the steps between intent and outcome without the added friction of relearning, rework, or workaround.

Seamless integration doesn't mean invisibility. It means placing AI where it supports tasks without breaking flow, duplicating effort, or requiring users to stop and adapt. Effective design reduces onboarding time, minimizes switching cost, and ensures that AI feels additive to the way work is done, rather than becoming another source of cognitive overload.

Implication: Design AI to meet users in the flow of their work. Integration should respect the tools, rhythms, and mental models users already rely on, delivering value with minimal friction and clear purpose.

Principle 7: Deliver Value in the Flow of Work

"If it doesn't make the job easier, it doesn't belong."

AI should serve the mission and the users who support it. That means it must deliver clear, observable value where and when it is introduced. Whether supporting faster decisions, surfacing better insights, or automating routine tasks, AI should reduce complexity and elevate outcomes.

While AI may feel novel by nature, adding AI should never be about novelty. It must be grounded in purpose and evaluated by its impact. If AI increases friction, slows tempo, or forces new work without benefit, it becomes noise, distracting from critical mission delivery.

Implication: Design and implementation decisions must be centered around real outcomes. AI should be measured by how well it integrates into existing workflows and how clearly it improves the work being done.

A Note on Accessibility

Accessibility remains as essential as ever. As AI systems evolve, our commitment to inclusive design does not change. It deepens. Designing with accessibility in mind ensures that the benefits of AI are available to all users and that we continue to meet the standards of usability and trust that real-world systems demand.

Why This Matters Now

As AI capabilities expand across a never-ending supply of startup, government, and enterprise environments, the pressure to adopt quickly can outpace thoughtful design. It's not enough to deploy working tools; they must be usable, trustworthy, and aligned to human needs.

These principles reflect a commitment to delivering AI that works for and with people. When we focus on context, clarity, and meaningful integration, we ensure that AI strengthens the work being done.

Human-centered design is a force multiplier. Systems designed with users at the center reduce friction, avoid costly rework, and accelerate mission results. By designing AI with intention, we reduce design debt and avoid the costly technical debt that comes from rework or retrofitting systems after the fact.

Our goal should be to augment human expertise and advance outcomes, while avoiding noise or friction. Human-centered AI builds confidence, accelerates insight, and improves the flow of work.

At my core, I'm a UX designer. I've spent decades thinking about how people understand systems, and one evening last week, I realized I needed to understand my own much better. My HomeLab is running CrowdSec, AdGuard Home, and fail2ban because people smarter than me made some recommendations to help me keep things safe(r) at home. I saw that things were being blocked, and frankly, I had only a limited idea what any of it meant.

A little over a month ago, I set up Nucleus as my HomeLab orchestration agent, built on JC Grubbs' Nella (github.com/thegrubbsian/nella-bot), a security-first take on the autonomous agent frameworks that were making the rounds at the time. A lot of those agents had a habit of running too hot on API credits and taking instructions further than intended. Nella was built with guardrails first, and I rewired it to run on local GPU instead of external APIs, and christened it Nucleus, to match the cellular theme of the HomeLab.

With Nucleus up and running, and learning the ropes of our HomeLab, it started passing CrowdSec alerts through Qwen and sending me plain-language summaries when activity thresholds were met. That solved an immediate problem and started an education. Telegram notifications would come through like this:

After about a month of these digests, I understood something I passively always knew, however didn't really consider with as much weight as I do now. And I understood better why some of my friends in Cyber are, well, the way they are about Cyber. There is a constant, low-grade festival of snooping happening against anyone running public-facing infrastructure. Most of it isn't targeted; it's automated background noise. Some of it is more interesting than that, and that's what got me to sit up a little straighter and pay more attention to what is constantly happening around me.

Those interesting hits had me sitting down Friday evening and doing deep research into what I didn't fully understand yet, and by Sunday night I'd built a custom Security Information and Event Management (SIEM) system for myself. Or at least a very localized and light version of one; the real SIEMs used in real organizations have a lot more horsepower to them, for certain, and I did my level best to keep things following the same pattern: Informative, with constant learning for me as part of the theme.

Before building anything with an interface, I worked with Claude Code to build the intelligence layer first. I generally refer to this as "Brains Before Apps:" if the system can't reason about the data, a dashboard is just a prettier version of the same tables I couldn't read. This is a process I've put into place for nearly every application I've built; if there can be focused knowledge to augment the LLMs capability, I want to have it help make our outcomes sharper and with the right context.

We built a Security Knowledge RAG backed by nine corpus files covering CrowdSec operations, MITRE ATT&CK techniques mapped to HomeLab traffic patterns, our actual infrastructure topology, and CrowdSec scenario definitions pulled from production. The reasoning engine is Qwen 3.5 running locally on an RTX 3090 Ti. No cloud APIs. Every synthesis, every recommendation, every analysis runs on hardware in the HomeLab and doesn't hit the outside world or someone else's systems for any inference.

With the brains in place, the apps followed quickly.

The customized SIEM we built translates the raw CrowdSec alerts into something a non-security person can act on: geographic breakdowns of where traffic originates, which services are being targeted, reputation scoring on attacking IPs, and plain-language descriptions of what each attack type actually means. We built in a Live Feed to be able to track real-time activity on a world map and that allows for things I'd never considered before: I can watch http probes from Vietnam targeting a blog while SSH brute-force attempts from Brazil are attempting to hit a Git server.

It's fascinating and sobering in equal measure.

This was a lot, if not more than enough for "just me with my HomeLab" at home, where one typically has a HomeLab, after all. However, I have GPU access without token cost, and the notion of "custom dashboards from semantic query" has been on my mind for awhile, so I wanted to figure out if, or how, this might happen.

My instinct was to build a fully open-ended semantic query interface, and every time I sat down to define it I hit the same wall: I don't always know the right questions to ask. I'm not a security professional. So the Dashboard Builder became a hybrid: five pre-defined report types covering the analyses I found most valuable, with the option to add semantic context. The pre-defined options aren't a crutch (though they certainly made testing much easier). These are the training wheels that are teaching me the vocabulary.

I ended up with a Custom Report Builder, which isn't far from what a Custom Dashboard Builder would have been anyway. The Custom Reports are useful and I found myself still having questions, so once the report is generated, I can follow-up with free-text queries. That is: I can ask the reports questions about the reports to get clarification and/or more details; the report itself is a part of the conversation and not the final output.

To close the loop, we updated Nucleus to run the analysis automatically. Every Sunday at 5:30 AM, Nucleus reviews the last seven days of incidents, compares them against the previous week, extracts hardening recommendations, creates tracked findings, and writes a dated case study to our Obsidian vault. The first automated run found 599 incidents from 171 unique IPs across 31 countries and produced five actionable recommendations. All five became tracked findings.

What we have now isn't a static security setup. It's a feedback loop. CrowdSec blocks threats. The pipeline captures what was blocked. The dashboard makes it visible. The RAG makes it understandable. The report builder makes it analyzable. Nucleus makes it actionable. And the Obsidian case studies build a documented record of an evolving posture, not a snapshot of one, and that weekly record will allow us to analyze trends over time as Nucleus continues to evolve.

I know significantly more about MITRE ATT&CK techniques, IP reputation scoring, and GeoIP enrichment than I did on Friday evening. None of it is deep expertise by any stretch of the imagination, and I have even more respect for those fine folks in Cyber than I did before I gave this my weekend. All of it makes me a more informed operator of the infrastructure I run.

Did I overbuild this for my needs? Absolutely.

Would I do it again? Without hesitation.

This is not about attempting to build a final product; I'm learning, and I'm sure I've fallen short of what someone else would do (and I'm also sure I'm going to ask some of those folks for feedback so I can keep the systems protected). This is entirely for my own benefit and self-interest. And it's fun.

The Feed & Thread, available on iOS and at feedandthread.com, is the result of six days and six months. I never knew I wanted to launch an iOS app when I started trying to stay current with my industry and community while using AI to make that easier.

Six months later, I knew exactly what I wanted to build and how to build it.

I've been scraping UX industry feeds, Reddit communities, and CMS content for the better part of a year, building a personal briefing system on my own infrastructure. At first it was daily summary emails to myself. Then a dashboard. Then a more serious aggregation pipeline pulling from dozens of sources across multiple domains. I was learning what freshness actually costs at scale, how to balance GPU load across simultaneous jobs, and what it takes to make content worth consuming rather than just worth collecting.

Along the way, I was working with Claude Code to build a half-baked iOS app and shared it with a few friends via TestFlight. It was ugly and incomplete and it didn't know what it wanted to be when it grew up. And I learned the Apple Developer ecosystem well enough that it wouldn't slow me down later. That matters more than it sounds.

I also spent a lot of time with TTS models: Kokoro, Chatterbox, and a variety of one-shot others; the lower quality, faster-processing side of this space is a time vacuum. I was learning how to write for speech, how to chunk content so it doesn't sound like an AI Instagram voiceover trying to sell hair restoration products, and how to coax something close to a real voice out of a synthetic one. Then Qwen3-TTS came out and introduced a driver conflict with our RTX 5060 Ti that had me manually rebooting a server more times than I'd like to admit on a Saturday evening. Pro tip: a smart plug and a BIOS setting to power on after restore will save you a lot of flights of stairs.

When the driver issues resolved, Claude Code and I refined our infrastructure to include Voice Registry: an administration layer that houses voice derivatives with configurable tempo, temperature, and tuning. I recorded myself at different speeds and energy levels and fed those into the model to create derivatives. Some of them sound enough unlike me that they won't annoy you. The process of creating new voices entirely is more involved than I expected; our attempts ranged from pre-teen to uncomfortably sultry before we landed on the right audition: Seven out of ten. We'll take it.

The podcast came together from all of this. I structured a Run of Show template system, learning from the Chicago Camps Tent Talks podcast exactly what a broadcast needed. Templates meant I'd only solve the architecture problem once.

Then came intro and outro music, stingers, music beds, all prompted and rendered locally via DiffRhythm. You can lose an entire evening rendering two-minute audio samples and tweaking fades before you remember you forgot to add volume ducking to the Run of Show settings you're currently trying to stitch your first episode with.

Eventually we had four segments: The Tease, The Feed, The Thread, and The Take. Two synthetic voices covering what got published in UX yesterday, what the community is arguing about on Reddit, and a closing conversation where the hosts land in different places on what it means. And now we're at around seventy daily episodes produced and we've never used a cloud API, never invoked an external subscription service. Creation is happening in my HomeLab with two GPUs that keep the scripts flowing and the hosts palavering.

Then I looked at the half-baked iOS app still sitting in TestFlight and realized: this podcast just gave it a reason to exist.

One Sunday evening the sketches on my whiteboard turned into an App Map, and the App Map turned into an MVP before I went to sleep. The Feed & Thread app would be the home for all of it: a daily industry briefing, browsable articles from over 50 RSS sources, community discussions from more than a dozen Reddit communities, and the full podcast archive with in-app playback. Over the next few evenings, designs got refined, the podcast wired in, and Claude Code and I launched a lightweight website and a lightweight ethical ad platform, impression and click tracking only, no behavioral profiling, serving Chicago Camps promotions between content items.

By Friday I was submitting the package to App Review. It's live today.

Here's what I want to be transparent about: a lot of people are claiming to build apps with a prompt and 20 minutes of jittering code creation from their favorite AI tool. And sure, some of them did just that. A lot of those are already stale URLs or abandoned ideas with a Stripe connection and a pricing carousel to purchase a subscription.

Speed without infrastructure gets you a demo. And in many cases, probably not a very good one, or one that didn't consider error handling and error states, fallback mechanisms, basic level security requirements, actual adherence to design systems and global style sheets (ask me how I know!), and the impact that scale can bring. I'm sure this is the tip of the iceberg.

Speed with infrastructure can get you a product or an outcome. The six days were fast because six months of separate experiments, scrapers, voice models, audio pipelines, podcast templates, and a failed TestFlight app, all suddenly had somewhere to go.

Claude Code was the development partner throughout. And the reason any of this came together quickly is because I'd been building the pieces without knowing exactly what I was building toward from the beginning. Each piece along the way generally had a purpose, including that initial iOS app which had a purpose of "can I FAFO and make an iOS app? Because if I can, that'll open some future doors," which is an okay purpose, at least for me.

Generally, though that's not a workflow I'd recommend as a strategy. It's what sustained curiosity looks like when it finally has somewhere to go.

I wanted to understand a geopolitical conflict better than my usual news sources were letting me, and I wanted to see what I could draw across economics and government contracting. So I built Tugboat.

Tugboat is a self-hosted intelligence platform that tracks active conflicts (currently only 1) and publishes a daily briefing with an audio / podcast component. Throughout the day, a pipeline pulls from scraped news feeds and various open APIs (USASpending.gov, and SAM.gov), runs it through Qwen3.6:27b on my local hardware, generates consumable summaries, and then overnight creates four reporter-style segments, synthesizes them to audio via Qwen3-TTS, stitches the episode with intro, stingers, and a music bed, and publishes to Castopod. I'm not using cloud APIs or external subscription services. The inference runs on two GPUs in my HomeLab.

The MVP took a few hours to get online; that's not saying this was easy by any stretch. I sketched it out first: a product vision, then a PRD. I wanted the full picture before anything got built. I also had the benefit of having the infrastructure already in place. I've been scraping feeds, building RAG systems, self-hosting podcasts, building tools that turn submitted content into audio and video training materials, and experimenting with TTS and audio pipelines for the better part of a year. Tugboat was mostly assembly.

A few things I learned that didn't show up until it was running:

After about a week, it was obvious to me that the scripts were starting to sound too identical. Not wrong, just... very alike. Qwen had settled into a handful of descriptors and wasn't leaving them. The fix was prompt-level discipline around vocabulary tiers, making the intensity of language match the actual intensity of the event. Obvious in retrospect. Only visible in production. Catastrophically visible after a few briefings (the joke is in the word catastrophic because it showed up a lot).

Editorial control in automated content doesn't always mean reviewing before you publish. It means building constraints into generation until you have real confidence in what comes out. I built in attribution discipline, avoided data dumps, specified opening lines that could only come from each day's data. Those aren't suggestions in the prompt; they're hard rules that we follow during creation.

The difference between a briefing and a summary is almost entirely in how you structure what the model isn't allowed to do.

Adding a new conflict is forking what exists and flipping a switch. That was deliberate from the start. The infrastructure was designed to scale, not just to work.

I'm a recovering UX designer. I did this because I was curious and had the tools to act on it.

Tugboat is live and free: tugboathq.com.