Show & Tell: Building a Pipeline Worthy of the Content

It seems to me, in my quite limited view of all of the internet, that every week or so someone asks how they can get up to speed on UX quickly. The answers are generally the same: read this book, take this course, follow these people. Good advice, generally. Also advice that assumes you have the time and attention to absorb a full curriculum before you need to apply any of it.

I've been on the other side of that question for most of my career, as an author and co-author, a conference speaker, a workshop facilitator. I know how hard it is to make practitioner knowledge transferable. And I've watched a lot of content, including plenty of my own, get consumed without landing.

The field isn't slowing down for anyone to catch up; it never has and it never will. And since I had the materials, the curiosity, and the infrastructure, I realized the only thing missing was the pipeline to put it together.

And that's the constraint I've been building around.

And the part I forgot to add, the lede I apparently buried: while creating 5 Minute UX.

The Corpus was Calling from Inside the House

Early last year, I'd already been torturing prompts long enough to know the ceiling. The results from general models were shaped by general knowledge, and no amount of prompt engineering was going to change that. I recently typed that "Prompts without context yield the average of the internet" and that is indicative of the experience I've had.

What I wanted was a model that knew my material, specifically, not an approximation of the field I'd spent decades working in. So I built a RAG from content I'd co-authored across six books and an unfinished manuscript, along with conference talks, workshop materials, and evaluation frameworks accumulated over years of practice.

Getting the materials together was its own exercise and ultimately the easy part. Getting that content into a formatted output so I could actually find it useful was the hard part. The paid models already do that for you behind the scenes and make it look inherently like that's just how it works. It's not.

Once it worked, something shifted. I could quick-ask a question and get a coherent response grounded in something I'd actually written and not a vague or hallucinated summary of someone else's blog post. I was getting answers with citations back to chapters I remembered writing and content I'd done the research on or had the lived experiences with. That was incredibly useful on its own.

Then the second lightbulb: this content had been used to teach people. Some of these books have ended up in collegiate settings, which is beyond pretty cool in and of itself and something I generally don't think about. They'd never been fully orchestrated to facilitate learning, beyond creating some light touch instructor guides with learning objectives.

A book teaches. I've been turning those chapters into lessons with clear objectives, a structured progression, and guided examples, followed by practical application. Adding in the ability for self-assessment is something similar enough, yet different for folks learning on their own and not in a workshop setting. I had the raw material at my disposal and it was a privilege and an opportunity to build the pipeline to extract and format it all in a formally structured way to facilitate learning.

Knowing Your Content Isn't the Same as Knowing How to Teach It

I've presented at my fair share of conferences and facilitated plenty of workshops across my career, including a few that deserved better orchestration than I gave them. Which means I understand, maybe more than most, that knowing your content isn't the same as knowing how to teach it. When I started automating lesson production, I didn't want to rely on what I thought I knew. I wanted something I could measure against.

That's where Gagné's Nine Events of Instruction and Bloom's Revised Taxonomy came in. Both have decades of evidence behind them, which matters when you're automating decisions that would otherwise require human judgment.

Gagné's Nine Events of Instruction is a bit of a workflow that mirrors how people learn: gain attention, state the objective, activate prior knowledge, present content, provide examples, elicit practice, give feedback, assess understanding, and support transfer to real contexts. Every lesson section is tagged to one of those events, and the pipeline generates differently depending on which event a section serves.

Bloom's Revised Taxonomy provides the cognitive ladder: from Remember at the bottom through Understand, Apply, Analyze, Evaluate, and Create at the top. Every enabling objective is classified by level, and the system verifies the lesson climbs that ladder rather than bouncing around randomly.

Together they answer a question a lot of automated content never asks: is this actually teaching something, or just presenting information in a tidy format?

Local Inference, Not a Phone Call to Someone Else's Model

I want to be transparent about how this all works and works together. For starters, everything runs on local hardware and inference. Qwen 3.6 generates content on a dedicated GPU, handling RAG extraction, objective shaping, and script generation. Qwen3-TTS renders audio on a separate dedicated GPU, keeping inference calls from stacking up on each other. A third evaluation pipeline scores the output independently.

None of it touches an external API, and that's more than a cost decision, though keeping experimentation costs controlled is genuinely nice. It means:

• The quality bar doesn't shift because a vendor quietly updated their model.
• The evaluation pipeline isn't just the generation prompt turned back on itself. It's a separate system with different logic, different signals, and a different job: not to create, to verify.
• The content stays internal to my infrastructure and isn't used to train someone else's model.

I've been continually assessing and iterating on the evaluation system since day one. And really, that's the point. While I perform Discovery upfront and create my own Vision documentation, I also have done this long enough to know that in many cases, the definition of done needs to have a bit of a sliding scale to it. Independence from external APIs provides control over the system that controls the quality bar, and allows for ongoing iteration without concern for token use impact.

How the Pipeline Works

I did three passes through the full corpus of multiple RAGs and catalogued 1,047 extractable lessons across eleven UX categories, each one assigned a priority score, a lesson type, and an extraction prompt tailored to the relevant RAG. There are lessons focused on concepts, skills, judgments, and critiques: each type has a different pedagogical emphasis and a different generation approach.

When a lesson enters the pipeline, the extraction prompt goes to the relevant RAG first: Books, Heuristics, Research Methods, Compliance, Conferences, or Courseware, depending on the topic. The RAG pulls source material, significantly more context than a cold prompt would produce. That extracted content grounds everything that follows; Qwen generates from the source material, not from the broad approximation of the field that a general model carries.

From there, Qwen produces learning objectives, then a Gagné-aligned outline, then per-section scripts at a roughly guided 190 words per section. Each stage is shaped by the lesson type and the Gagné event tagged to each section. When the pipeline generates script for a section tagged "Provide Guidance," the entire section is examples and demonstrations. Not just the opening sentence. Every paragraph.

The Evaluation Gates

There are three evaluators, twenty-one signals total, assessing everything generated. The Structure Evaluator checks the outline before a single script word is written:

• Do the enabling objectives support the learning objective?
• Does the lesson progress upward through Bloom's taxonomy?
• Does every objective have a section teaching it?

The Script Evaluator checks whether the narration delivers on the structure's promises:

• Key point coverage: are the promised points in the scripts?
• Objective delivery: does the narration deliver on every enabling objective?
• Variety: does the lesson avoid repetitive phrasing and monotonous sentence patterns?
• Gagné alignment: each section fulfills its tagged pedagogical event?
• Narrative flow: reads as coherent progression, not disconnected blocks?
• Readability consistency: consistent reading level across core sections?

The Assessment Evaluator checks the quiz questions:

• Are correct answers grounded in the lesson content?
• Are wrong answers pedagogically useful?
• Does the difficulty distribution reflect the right cognitive mix?

Seventeen of the twenty-one signals are deterministic code, things like embedding similarity, verb classification, word count, and n-gram repetition detection, which means the bulk of the scoring happens through logic that doesn't change based on how the model is feeling on a given Tuesday. The remaining four signals use Qwen answering focused binary questions, one at a time, with narrow context windows, and the LLM never exceeds twenty percent of any evaluator's weight. If you run the same evaluation twice, you get the same score.

Lessons scoring above 0.70 auto-publish, and anything below routes to a human-in-the-loop review, which in every case means me. I'm the human and I get to decide what happens next based upon what failed and why. A structure problem might get a prompt adjustment before the lesson regenerates, while a script problem might get a manual edit followed by re-evaluation. A Gagné alignment failure is usually the most instructive of all because it tells me something specific about what a section was trying to do and fell short of. The diagnostic output is in plain language so that the fix, more often than not, becomes obvious.

Every evaluation surfaces on the lesson page organized around three questions: does the lesson teach what it promises, is it structured for effective learning, and do the quiz questions test the right things. The scores are a weighted composite of all the signals, broken down by category, with per-section Gagné alignment shown individually so you can see exactly where a lesson succeeded and where it needed more work.

If you want to understand why a lesson scored the way it did, the answer is right there waiting for you.

From Evaluation to Episode

Once a lesson clears evaluation, the production aspect of the pipeline runs. Qwen3-TTS renders audio across two synthetic voices, audio assets are rendered and layered according to the Run of Show, our episode template, and a Remotion renderer produces 1920x1080 video with word-by-word captions synced to Whisper timestamps. Every lesson closes with The Take: a two-voice conversation between a Host and a Learner, both practitioners, working through the implications of the concept together. It's the kind of conversation that happens after a good workshop session, including, more often than not, a genuine question about where AI fits into the practice being discussed. Research calls the underlying technique the Muller effect. I'd argue it's the part of the lesson people actually remember.

Every lesson publishes across three modalities: a full transcript with key points per section for those who prefer to read, a podcast episode for those who listen on the go, and a video for those who learn best by watching. The goal is to meet practitioners wherever they are, not to pick a format and hope it works for everyone.

Lessons Learned from Inside the Pipeline

The pipeline was fully functional, in parts and pieces across my own current infrastructure, for months before a single lesson was published. I've been exploring and creating personal tooling to help with organization and efficiency, product-like projects that keep me learning, self-hosted applications and services to aid in the ability to continue to explore and expand my knowledge, and infrastructure that requires its own monitoring and maintenance to keep it healthy.

All of those pieces are a part of the cost of doing this with any real rigor. I needed the pieces from many different projects, reaching all the way back to that very first RAG and a glimmer of a vision, to get to the clear definitions of what I wanted to build before I could measure anything. Gagné and Bloom's gave me the frameworks to know what good actually looks like, and I ran the pipeline repeatedly on synthetic data, then on real content, then again after every calibration, until the confidence scores meant something I could trust.

The corpus that made all of this possible started coming together early last year, long before 5 Minute UX existed as an idea. The scraping, the RAGs, the intelligence feeds, all of it was already running. The AI amplified what was already there and brought it all together to match the vision. Even the related articles that surface alongside each lesson come from the same intelligence feeds I've been building for months, content from practitioners and community members creating work worth sharing alongside our own.

The name sets an expectation worth addressing directly: none of this is trying to teach all of UX in five minutes. The point is to give practitioners something genuinely useful in the time they actually have, grounded in real practitioner knowledge, built with enough rigor that we're not embarrassed by what comes out the other end.

And I'm still watching the queue today, still finding opportunities to improve, and still iterating.