Spirit Spy: Vision & Build PRD

A build-it-yourself brief for a self-hosted, no-subscription spirits-collection catalog with photo-label intake.

Scope. This document is a build specification, not a supported software distribution. It provides the product model, requirements, sequencing, and implementation lessons needed to create your own version. Your resulting application, technology choices, security posture, deployment, and maintenance remain yours.

Version: 1.3. Covers the as-built system through the cocktails layer: photo→label intake, the expression-vs-bottle model, web enrichment that cites its sources (and fills blanks, specs included), the public showcase, local CPU background-knockout, a private wishlist, and cocktails (per-bottle real-recipe enrichment in labeled tiers, a mixologist browse across the whole shelf, and fenced off-menu AI invention). Built and running. The enrichment sections reflect real-world hardening (multi-page crawl, the "known context" trap, ambiguous-name search, a manual deep-dive escape hatch, and the recipe-page tuning that specs enrichment does not need); the data-model section reflects the modeling lessons that surfaced in use: size lives on the bottle, capture-loose-then-merge, and "same name is not the same product." Audience: someone technical-ish who wants to roll their own hobby-collection tracker, own their own data, and feed this brief to an AI coding agent on whatever stack they prefer.

How to use this document

This is a product requirements + vision brief, not a codebase. Hand it to a capable AI coding agent (or read it and build by hand). It describes what the catalog does and how the pieces fit, plus the hard-won facts about label OCR and web enrichment that will save you days. It deliberately avoids prescribing a language, framework, database, or model: those are yours to choose. Where a decision matters, the brief explains the reasoning so you can adapt it to your collection and your hardware rather than copy ours.

There is no repository and no shared code by design. The goal is for you to build something that fits your shelf, your taste in detail, and your setup, grounded in your data, rather than inherit assumptions baked into someone else's build.

Why a PRD instead of a repository

This is a deliberate choice, not laziness about packaging.

Software is trending toward disposable and single-use. A great deal of what gets published as a repo is something almost no one else will actually run, because running someone else's build means trusting it, and the thing you are building here holds a personal inventory of what is in your house, points photos of your basement at a vision model, and reaches out onto the open web on your behalf. Reasonable people are skeptical of letting an unfamiliar codebase touch any of that. No one wants a stranger's project quietly opening a backdoor or hauling in a dependency tree they will never audit.

A PRD sidesteps all of it. You hand this brief to an agent you already trust, in an environment you control, and it builds something you can read and own end to end. You adapt the data model to how you think about your bottles, swap the models for whatever you run, and decide your own safety posture. You get the design and the hard-won integration knowledge without importing anyone's dependencies, assumptions, or risk. The thing you end up with is yours, not a fork of ours.

And here is the part that matters most for a hobby tracker: there are perfectly good catalog apps out there, and most of them want a few dollars a month, forever, to hold a list of things you already own. The recipe is a more generous gift than renting you the kitchen. If you build something good from this, that is the entire point. There is nothing to star and no one keeping score.

The problem

There is no shortage of apps for cataloging a spirits collection. The trouble is what they ask of you, and it comes down to two real objections:

They charge you for the part you do not need. The pitch is almost always value tracking: what is your collection worth, what is this bottle going for at auction, is that dusty find an investment. That is a real feature with a real cost behind it (pricing data is paywalled, more on that below), so it ends up behind a subscription. But a lot of people do not collect to flip; they collect to pour and to remember. Paying a recurring fee for a market-tracker you will never use is a bad trade.
They own your data. Your inventory lives in their cloud, in their schema, exportable on their terms. The day the app shuts down, gets acquired, or triples its price, your carefully entered collection is theirs to hold hostage. For something as personal and as permanent as "what is on my shelf," that is the wrong arrangement.

What you actually want is to grab your data and own it. A list of what you have, made easy to build (you should not have to type every field off a label by hand), pleasant to browse, and yours to keep, running on hardware you already have, with no monthly bill and no external service that can disappear.

The vision

A personal catalog whose superpower is photo intake: snap a bottle's label, a local vision model reads it, you confirm, and it joins a private collection. That collection is browsable two ways, as a working admin where you edit, enrich, and photograph, and as a quiet, public showcase that is image-first and read-only, a conversation piece you can hand someone a link to.

Underneath, it is honest about what a thing is versus what you own: an expression (the product) and the bottles of it on your shelf (sealed or open, however many, which barrel). It fills in the blanks for you by searching the open web and citing what it found, but it treats the label on the bottle as ground truth and never overwrites your own words. Photos get cleaned up locally so a messy phone snapshot becomes a tidy product shot. And it runs entirely on your own hardware, no external AI APIs, no subscription, no telemetry, your data on your disk.

One stance worth stating up front, because it shapes everything: pricing and value tracking are deliberately out of scope. This is a conversation piece, not an investment tracker. That is not a gap we are apologizing for; it is a design decision, and the section below explains why it is the right one.

Core capabilities

Each capability below is described as an outcome, not an implementation.

1. Photo → label intake, with a confirm step

The everyday way a bottle gets into the catalog: you photograph the label, a local vision model reads it, and the add form arrives pre-filled. You glance, fix anything off, and save.

A few choices made this work rather than frustrate:

Front face and back face are different jobs. The front label carries identity (brand, distillery, the variant name), and these want to land in the right separate fields (the brand is not the distillery; the variant is not the brand). The back label carries the production details, the tasting notes, and the producer's own marketing prose. Capture both, tag each extracted fact with which photo it came from, and let the model merge them.
Pull the producer's own "from the label" prose verbatim. The paragraph on the back of the bottle, the maker's romance copy about the grain or the cask or the river the water came from, is worth keeping as written. Read it off and store it. (Exclude the government warning text; nobody needs that in their catalog.)
Reading is a deliberate action, not an automatic one. It is tempting to OCR every photo the instant it is uploaded. Resist it. A "Read this label" button (and a "Read all faces and merge" button when there are several) puts the human in control of when the model runs and which photos it reads. You capture carefully, one bottle at a time, and reading on demand fits that rhythm, and keeps a model call from firing on a blurry test shot.
Show confidence per field. Where the model read a value cleanly off printed text, mark it confident; where it had to guess (handwriting, glare, a turned bottle), mark it as needs you so your eye goes straight to the fields that need a human. Small thing, big trust payoff.

2. The expression-vs-bottle data model

This is the structural idea the whole catalog hangs on, and getting it right early saves a painful migration later.

An expression is what a thing is: the distillery, the brand, the product name, its category and subcategory, region, age, ABV, cask, flavor notes. One row per product.
A bottle is what you own: a single physical bottle of that expression. Its size (a 750 ml vs a 1 L), sealed or open, fill level, where it is stored, what you paid (owner-only), and, crucially, its batch / barrel / lot identifiers.
Quantity is just the count of bottle rows under an expression. The library shows "Blanton's ×3"; expand it and you see each bottle's own barrel number, fill level, and notes.

The batch/barrel/lot fields are what let you tell same-name bottles apart honestly. Two bottles of the same single-barrel product from different barrels are two bottles under one expression (record the barrel number per bottle). A genuinely different product line, the Gold release versus the Original, is a new expression. Encode that rule into your "you already own this, add as another bottle?" prompt at save time, and you will avoid both of the failure modes: spurious duplicate products, and quietly collapsing two distinct releases into one.

Two more lessons learned the honest way. First, size lives on the bottle, not the product: the same gin in a 750 ml and a 1 L is one expression with two bottles, each recording its own size; don't let a format difference spawn a duplicate product. Second, capture loose, then consolidate: when you are adding a whole shelf, it is faster to let each photo land as its own record and tidy up afterward than to perfect the match at intake. So give yourself a merge action (fold several records into one canonical expression, moving their bottles across) and lean on it. The corollary matters: a same-name match is not proof of a same product. Two "Blanton's" can be the Original and the Gold Edition; verify before you merge, and never auto-collapse on the name alone.

3. Web enrichment that cites its sources

When a bottle is saved with blank fields (and label reads always leave some blanks), the catalog goes and looks the bottle up on the open web, fills the gaps, and tells you exactly where each fact came from.

The shape of it: search → fetch → extract → cite. A search engine finds candidate pages; the results are ranked (the maker's own domain first, a short allowlist of review and retailer sites next, obvious noise dropped); the best official page is fetched in full; and a local LLM reads the gathered text and returns only the facts the sources actually state. It stores the distillery URL, the product-page URL, and a bulleted Sources list, so every claim is auditable.

The rules that keep it trustworthy:

It fills blanks; it never clobbers. Your label reads and your hand edits are sacred. Enrichment writes only into empty fields (and may correct a prior enrichment guess), never over something a human entered.
Blanks fill automatically; conflicts are advisory. A blank spec (ABV, age, cask, mash bill, tasting notes) fills straight from the web, because an empty field has nothing to defer to and a sourced value beats a blank. But when the web disagrees with a value already there (a different ABV, an age the label did not print), it never overwrites; it surfaces that as an admin-only parity flag ("web says 46.5% · Use") with a button you press if you agree. The line is simple: an empty field is a vacuum to fill, a filled one is a fact to defer to.
No pricing. Enrichment never searches for, extracts, or stores a price. (Again, the reasoning is below; it is load-bearing enough to repeat.)
The public detail page shows the cited sources as plain underlined links beneath the description, plus explicit Distillery and Product links, so if a visitor likes something on your shelf, they can go find it.

4. The public showcase

A separate, read-only face on the same collection: image-forward cards in a browsable grid, a faceted filter (category, distillery, region, age, ABV) generated from what you actually own, and a per-bottle detail page. The detail page leads with the photo, lays out the specs, shows the producer's "from the label" prose, and underneath it the cited sources. It is the version you send a friend a link to, a quiet, handsome catalog, with the owner-only fields (what you paid, where it is physically stored) simply absent.

A note on how to build that safely: the public surface should be its own process that registers only the public, read-only routes: the admin and every editing path physically absent from it, not merely hidden behind a login. "The code to delete a bottle isn't loaded in this process" is a much stronger guarantee than "the delete button is auth-gated."

5. Image handling

Two image jobs, both local:

Background knockout (the everyday one). A bottle photographed on a basement shelf has a messy background. A local model removes it and writes a transparent product cutout: bottle floating clean, no background. The showcase floats that cutout on a styled backdrop, so a quick phone snapshot reads like a product shot. Keep the original untouched; the cutout is a derived asset. The pleasant surprise here: knockout is a CPU job, not a GPU one. The good background-removal models run fine on a CPU, which means this can be a small always-on service that does not fight your GPU for VRAM.
Hero generation (optional, nice-to-have). If you have a GPU and a local image model, you can generate atmospheric backdrop images (for a whiskey collection: copper stills, a rickhouse, amber glass) and rotate them behind the showcase alongside your own photos. Pure polish; skip it in v1.

6. The private wishlist

A bottle you want is captured exactly like one you own: snap the label in a shop, the model reads it, you confirm, and it lands on a private Wishlist tab in the admin. The same intake, the same web enrichment, the same fields. The only difference is that it never reaches the public showcase, and the day you buy the bottle you promote it into the collection with one action, no re-entering anything, because the record is already complete. (After promoting, you usually only want to re-photograph it, since the wild shop photo is rarely the keeper.)

The privacy here is worth dwelling on, because the cheap way to build it is also the wrong way. Do not add a "private: yes/no" flag the public query has to remember to check: that is a filter you can forget, and forgetting it leaks. Instead, reuse the review-state the catalog already has. Every bottle carries a state: draft while you are still cleaning it up, confirmed once it is real and catalogued. The entire public surface already shows only confirmed bottles. So a wishlist item is simply a bottle in a third state, wishlist, and it is private by construction: it is not confirmed, and the public process only ever asks for confirmed, so there is no code path that could surface it. Promoting is then one field flip, wishlist → confirmed. You did not build a privacy feature; you reused an exclusion the architecture already guaranteed. That is the pattern to reach for: make the unwanted state unreachable, rather than adding a guard you have to maintain.

7. Cocktails: recipes for what you own, and originals from your shelf

A collection is for pouring, so the catalog earns its keep by answering "what can I make with this?" It does that in three layers, each more adventurous than the last, and one rule runs through all of them: a real recipe and an invented one must never be mistaken for each other.

Real recipes, per bottle. For any spirit on the shelf, go find actual cocktails that call for it and attach them to that bottle. Keep two kinds, and always label which is which. A Signature recipe names this specific bottle: the distillery's own house drink, or a bartender's recipe built on this exact expression. A Classic recipe is the canonical drink for the spirit's category, a Manhattan for a rye or a Margarita for a blanco, used as a graceful fallback when the web is thin on the specific bottle. What the catalog stores here is only recipes that exist in the world; the model extracts and selects, it never invents at this layer.
A mixologist view across the whole shelf. Once every bottle carries recipes, let the owner browse them as a single pool rather than bottle by bottle. Pick a spirit type, a cocktail flavor, a tasting note, or a tier, and see everything that matches across the collection. Treat it as a browse aid, not a precision search: the question it answers is "what can I make tonight," from what you actually own. Give the unfiltered view a sensible default order and group it under headings so it reads as something composed, not a wall of cards.
Off-menu invention. The adventurous layer, and the one to build last and fence hardest. Ask a local LLM to invent an original cocktail from bottles you own, using only the model's own knowledge of how spirits and flavors go together. It attaches the invention to every bottle it draws on. And it is badged as invented, always, because an invented drink is an untested idea and the interface has to say so out loud.

The honesty fence. This is the "label is truth" of the cocktails feature: an invented cocktail is never allowed to pass for a real one. Give every recipe an explicit tier: signature (real, names this bottle), classic (real, canonical for the type), and invented (AI-generated). Let the tier drive a visible badge wherever a recipe appears. A saved invention keeps its own tier, lists under its own "Invented" heading, never mingles with the real recipes, and wears copy that calls it what it is: an original idea, untested, pour at your own risk. The failure you are guarding against is a plausible fabrication getting made as though a bartender vouched for it.

Conceptual data model

Described as entities and the fields that carry meaning. Use whatever storage you like; any relational store is plenty.

Expression: what a thing is. Distillery, brand, product name, category, subcategory, region, country, age, ABV (proof is derived, never typed, US proof is exactly twice ABV), cask type/finish, flavor descriptors, the producer's "from the label" prose, and the enrichment bookkeeping (which fields came from the web, the distillery and product URLs, the cited sources, when it was last verified). Mostly public.
Bottle: what you own (or want). One row per physical bottle: foreign key to its expression; size (a bottle attribute, not the product's, two formats of one product are two bottles); seal status, fill level, condition; year / batch / barrel number / serial; storage location and purchase price/date/source; and a review state, draft, wishlist, or confirmed. That state is the access gate: the public surface asks only for confirmed, which is what keeps drafts-in-progress and wishlist items off the showcase. Identity fields public; storage location and money owner-only; anything not confirmed is wholly private.
Photo: image files attached to a bottle (the original and its derived knockout cutout and thumbnail), each tagged with which face it is (front / back / side) so intake can merge them.
Enrichment source: for each enriched expression, the title and URL of every page a fact came from, so every web-sourced claim stays auditable. Public (it is the receipts for what is shown).
Recipe: a cocktail attached to a spirit. Foreign key to the expression; a tier (signature = real and names this bottle, classic = real and canonical for the type, invented = AI-generated); title, ingredients, instructions, glass, garnish, and a one-line "why it fits"; a few flavor tags drawn from a small fixed vocabulary (sweet, smoky, spirit-forward, and so on) that power the mixologist filter; the cited source (URL and name) for the real ones; and, for an invention that draws on several bottles, a shared creation id that ties its rows together. Real recipes are public; invented ones are yours to show or keep.

The public/owner-only split is the only access rule you need: identity and story are public; what you paid and where it physically sits are not.

The capture pipeline reality

The intake path, end to end: camera → downscale → local VLM OCR → structured fields → human confirm.

Downscale before you send. A modern phone photo is enormous; the vision model does not need the full resolution to read a label, and a smaller image is faster and cheaper to process. Keep the original on disk for the showcase, send a downscaled copy to the model.
The "deliberate read" choice, again, because it matters. Auto-OCR-on-upload feels magical for about a day and then it is firing model calls on every accidental, blurry, half-framed shot and muddying your data. A human-triggered "Read this label" is calmer, cheaper, and more accurate, because the human only points it at good photos. We retired auto-read in favor of it and did not look back.
Merging faces. When a bottle has a front and a back photo, read each, tag every extracted fact with its source photo, and merge: front wins for identity, back wins for production details and prose. Surface conflicts for the human rather than silently picking one.
The model matters more than you would hope. Not all local vision models are usable for this. A weak one will confidently fabricate an ABV or an age that is not on the label, which is worse than leaving the field blank, because a confident wrong number sails past your eye. Test your candidate model on a handful of real bottles before you trust it: check that it reads printed numbers exactly, that it returns null for facts the label does not state instead of inventing them, and that it keeps the brand, distillery, and variant in separate fields. A good one will read clean titles, exact ABVs, and even handwritten batch numbers; a bad one will tell you a pale ale is vodka. Spend the afternoon to find out which you have.

The enrichment reality (worth reading twice)

This is the part that separates a useful catalog from a pretty one, and the part most likely to go subtly wrong, so it earns a careful read.

The pipeline: search → rank → fetch → extract → cite.

Search. Query a web search engine with the bottle's identity ("{name} {distillery} {category}"), and run a second pass adding something like tasting notes if the first is thin.
Rank into tiers. Tier 1: the maker's own domain (the result whose domain matches a normalized token of the distillery or brand). This is the authoritative source and the one you most want. Tier 2: a short allowlist of reputable review and retailer sites. Everything else: dropped. A naive "just feed the model the top results" approach will pull in a restaurant menu, a big-box store listing, and a vacation rental that happened to share a word with the bottle's name; ranking is what keeps that junk out.
Fetch: the content pages, not just the official one. This is the correction that mattered most. The early instinct is to crawl the maker's own page; in practice that page is often a thin JavaScript marketing shell, while the actual specs (mash bill, age, and proof) live on the review and database sites. Crawl the full text of the top few distinct domains (dedupe by domain, or an official site that returns five results will eat every slot), keep snippets of the rest, and cap how much text you keep. Two more realities worth designing for: some sites (big retailers, aggregators) sit behind bot-protection that hard-fails any crawler (detect that and fall back to their snippet rather than burning a fetch slot), and a brand-only search buries the good content sites under the official domain, so run a second, review-oriented query ("{name} review tasting notes") and merge the results.
Extract, and be careful what you call "known." Hand the gathered text plus the bottle's known facts to a local LLM and ask it for structured JSON of only the fields the text supports: description, flavor notes, category, region, ABV, age, cask, mash bill, and the source links. A strict prompt: only what is stated, no invention, ignore irrelevant snippets, and never a price field. And the trap that cost us a long debugging session: when you give the model the bottle's known facts for context, give it only the human-facing identity fields, never the whole database row. Feed it your own bookkeeping (internal IDs, timestamps, and above all the previous enrichment's stored JSON), and the model reads that as "this is already done" and returns nothing. The symptom is maddening and easy to misread: freshly added bottles enrich fine, but re-running enrichment on anything already touched reports "nothing to add," and a bulk backfill quietly fills zero. Keep the KNOWN context to name, distillery, brand, category, region, ABV, age, cask, mash, and nothing else.
Cite. Store the distillery URL, the product URL, and the list of sources, and show them.

The one rule that governs all of it: the label is truth, the web is a draft. What is printed on the bottle, and what you typed, outranks anything the web says. Enrichment fills blanks and proposes corrections as advisories; it never silently rewrites the bottle's own facts. Build that contract in from the first line and never relax it.

Why pricing is excluded. It is not an oversight and it is not a "phase 2." Two reasons, and they reinforce each other. First, the philosophy: this is a catalog of things you own and enjoy, a conversation piece. The moment it becomes a portfolio with a running dollar total, it changes what the collection is for, and not in a direction we wanted. Second, the practicality (see the data-source section): real, current secondary-market pricing is paywalled or ToS-locked everywhere credible, so "value tracking" would mean either a recurring paid data feed (the very subscription you left to build this) or scraping someone in violation of their terms. Excluding pricing is what lets the whole thing run free, local, and clean. If you ever genuinely want a value number, leave a slot in the schema for a manual observation you log by hand, but keep it out of enrichment entirely.

The failure modes, and how to guard them.

Wrong-bottle match: a different maker's product that happens to share a name. Guard: put the distillery in the search query, and rank the maker's own domain first, so the authoritative page wins over a same-named impostor.
Generic-token noise: a common word in the bottle's name ("Farms," "Mission," "Reserve") matching an unrelated domain and getting cited as a source. Guard: normalize and match on meaningful tokens, not partial words, and store the URL for every claim so any bad citation is visible and removable. Auditability is the backstop: if every fact links to its source, a wrong one is caught at a glance instead of becoming silent bad data.
A source service being down. Guard: degrade gracefully (snippets only if the page fetch fails), and if there is no usable source at all, leave the row untouched. Never write a guess. A blank field is honest; a fabricated one is a landmine.
An ambiguous name or brand: a generic expression word ("Blanco," "Silver," "Reposado") or a short/numeric brand ("1800") collides with something unrelated: a sink company named Blanco, the 1-800 phone brands, or the year 1800. The search comes back full of kitchen faucets and the bottle never surfaces. Guard: when the normal name-first query returns nothing on-topic, retry with the category leading ("tequila 1800 blanco") to anchor the search to the right world. Don't lead with the category by default, though: on some categories it drags in SEO award-site spam; make it a fallback, not the primary.

An escape hatch for the bottles the web can't reach. A small number of bottles defeat the pipeline no matter how good it is: a genuinely web-dark small producer, or a famous one whose only pages are JavaScript shells that crawl to nothing. For those, keep a manual deep-dive path: a person (or an agent you drive) researches the specific bottle by hand, identifies what makes it special (a commemorative edition, a limited release), and writes a rich, sourced record directly. It is the white-glove complement to the automated pipeline, not a replacement; reserve it for the bottles you care about that the automation leaves thin.

Run it as a background job. Search plus a page fetch plus an LLM extraction easily blows past the ~100-second timeout that a reverse proxy or tunnel (Cloudflare's default is the classic one) will impose on a single request. So enrichment must be a background job you poll: kick it off, get a job id, poll for progress, collect the result. A bottle can auto-enrich in the background the moment it is saved, with no one waiting on a spinner. (This same async pattern applies to the label OCR reads, for the same reason.)

The cocktails reality

Recipe enrichment reuses the same search → rank → fetch → extract → cite spine as fact enrichment, but recipe pages behave differently enough that they need their own tuning, and the invention layer adds a few rules of its own.

Recipes live below the fold, so read deeper. The text window that works for specs fails for recipes. A bottle's ABV and age sit near the top of a page; a cocktail recipe sits well below the marketing chrome, so a small text cap that is fine for facts will cut the recipe off and the model extracts nothing. Keep a much larger slice of the page text for recipe extraction than you do for specs. This one cost a real debugging session: the crawl looked healthy, the page had the recipe, and the model still returned empty, because everything it was handed was nav and banner.
A maker's own site will hand you the wrong drink. A distillery that makes several products publishes cocktails for all of them, often on one page, and a naive crawl of "the brand's recipes" happily attaches a gin cocktail to the vodka. Guard it: check that the drink actually uses the bottle's base-spirit family, and drop a recipe when that family is absent and a different one is present. Exempt liqueurs and mixers from the check, since they legitimately go into other-base drinks, and know that this exemption is itself a small leak, because a brand that makes both a base spirit and a cream liqueur under one name can still cross the streams. The catch-all for the rare dirty match is a per-recipe delete.
Strip the brand before you de-duplicate. Titles arrive in brand-prefixed and possessive variants ("Blanton's Old Fashioned," "Old Fashioned," "The Old Fashioned") that are the same drink. Normalize the title (drop the record's own brand and distillery tokens, possessives, and a leading article) before de-duping, or you keep three copies of one cocktail.
Seed from the maker's own page for search-invisible brands. Some bottles never surface in a plain search because the name reads as a common word and the results come back about something else entirely: a tea, a town, a color. But the maker often publishes recipes on its own site. Seed the crawl from the record's own product or distillery URL first, then fall back to search, and run the same base-spirit guard on the maker's page, because that page is the exact multi-product trap.
Validate any URL that came from a record before you fetch it. The moment you crawl a URL stored in your own data, you have a request-forgery surface. Confirm it is http(s), carries no embedded credentials, and resolves entirely to public addresses (reject loopback, private ranges, and cloud-metadata endpoints) before fetching.
There is a text-extraction ceiling. A brand that publishes its cocktails as a JavaScript widget, an image, or a PDF menu cannot be read by a text crawler; you get nav chrome and nothing else. Accept that some menus are out of reach, or add a vision/OCR path later. It is a real capability to build, not a tweak.
Cap the wall-clock per bottle. A single JavaScript-heavy page can burn a crawler's whole timeout on each of a dozen attempts and stall one bottle for ten minutes. Put a total time budget on the job: when it runs out, store what you have and move on rather than retrying forever.

For the invention layer specifically:

One call, a few originals. Ask for three distinct drinks in a single request rather than making three calls; it is faster, and the model varies them on its own when told to.
Ground it to the shelf. Pass the bottles the owner actually owns and require the invention to build only from those (plus common mixers), so it can never call for a bottle that is not there.
If it is public, bound it. An open invention endpoint is a model call anyone can trigger, so rate-limit it per client and send a monitoring ping per use. Key the limit on an unforgeable client identifier: behind a proxy or tunnel, the first forwarded-for hop is client-settable and trivially spoofed, so use the trusted client-IP header your edge actually sets, not the raw forwarded list.
Saving one must not lose the rest. When several inventions are on screen at once, saving one cannot reload the page out from under the others. Save in the background and mark that one card saved in place, so the remaining ideas stay put and still savable.

Data-source reality

A word on what actually exists out there, because the honest picture saved us from building on sand. The short version: identity, reference, flavor, and recommendations are all buildable on free and open data; current pricing is the one genuine wall.

What is open and useful:

Search snippets, the maker's page, and, for the specs, the review/database sites. The early assumption was that craft and local bottles "won't be in any database", and that is true of the structured databases, but false of the open web: a distillery page or a review surfaced for nearly every bottle tried, down to the most local small-batch ones. One refinement earned in practice, though: the maker's own page is great for the romance copy and the official line, but it is frequently a thin JavaScript marketing shell, and the hard specs (mash bill, age, and proof) reliably live on the review and enthusiast sites instead. Crawl both, and weight the content sites for facts; don't assume the official page is the richest source.
Wikidata: a free, queryable graph of producers, distilleries, regions, and brands with stable entity IDs, good for normalizing names. Note it is not blanket public-domain; check the licensing and attribution per use, and it has query timeouts.
Government label registries (in the US, the TTB COLA registry): free, authoritative label approvals and label images, good for verifying US-market bottles.
Barcode / product databases (e.g. Open Food Facts): a possible identity anchor from a UPC, but spirits coverage is spotty, so treat a barcode as a fast-path when it resolves, not the primary intake. Prefer their bulk data exports over hammering a live API.
A handful of whisky-specific review APIs and an open-source whiskey reference dataset: good for seeding a flavor-descriptor vocabulary (the SWRI flavor wheel and a 2024 ML-derived aroma wheel are the established taxonomies) and bootstrapping a "what to pour" recommender that runs locally with plain vector math, no learned embeddings required.

What is paywalled or ToS-locked:

Real-time secondary-market pricing: the load-bearing gap. Every credible source is paywalled (the dedicated pricing-data APIs) or commercial.
Whiskybase: a superb reference, but its terms block scraping and its export is your own collection only, not a reference dump, and not prices.
Aggregated auction stats exist free and no-auth from at least one source, but they are per-distillery aggregates, not per-bottle, so they are a weak pricing signal at best.

The honest gaps: collector-grade reference data for non-whisky (rum, agave, gin, brandy) is thin in open datasets, which skew heavily whisky; for those you hand-build the taxonomy from Wikidata and the regulatory bodies. And pricing, as said, you either log by hand or do without. Our take: do without, and let that be the feature.

Suggested build order

Each phase ships something usable and de-risks the next:

The catalog core, local-only. The expression-vs-bottle data model, a faceted library list, a manual "add a bottle" form, a detail page (view / edit / delete), and a merge action (fold duplicate expressions into one, moving their bottles over) for the capture-loose / consolidate-later workflow. No AI yet. This alone is a working inventory you own.
Photo intake. Wire in the local vision model: upload or capture a photo, a deliberate "Read this label" button, structured pre-fill with per-field confidence, the front/back merge, and the match-on-save prompt (same product → another bottle, or new expression). This is the superpower; get it solid before anything else.
Image knockout. Stand up the local CPU background-removal service, auto-fire it on photo ingest, and write transparent cutouts. Now your photos look like product shots.
Web enrichment. Build the search → rank → fetch → extract → cite pipeline as a background job, with the never-clobber contract, the parity advisories, and pricing firmly excluded. Auto-enrich new bottles on save; run a one-time backfill over what you already have.
The public showcase. A separate read-only process: image-forward cards, faceted browse, per-bottle detail with sources, owner-only fields omitted. Front it however you expose things to the world.
Cocktails. Per-bottle recipe enrichment on the same pipeline, tuned for recipe pages (a bigger text window, a base-spirit precision guard, brand-stripped dedup, a distillery-site seed, and a per-bottle time budget), storing real recipes in two labeled tiers. Then a mixologist browse across the whole shelf. Then, if you want the adventurous layer, off-menu invention behind the honesty fence, grounded to owned bottles, and rate-limited if it faces the public.
(Optional polish) Hero generation on a GPU, recommendation ("what to pour") on the flavor vectors, and a manual value-observation field if you decide you want one.

Roles you will need to choose

To keep this stack-agnostic, here are the roles, each one's job, not our pick. You may map two roles onto one piece of software, or split them across machines.

A local vision / OCR model. Reads a bottle's label into structured fields and prose. The job: read printed text exactly, return null for what is not stated (no fabrication), and keep brand, distillery, and variant separate. Test candidates on real bottles before committing; this is the role most sensitive to model quality.
A local LLM for extraction and enrichment. Reads gathered web text and emits only the structured facts the text supports, following a strict no-invention, no-pricing prompt.
A web search engine you can query programmatically (a self-hostable metasearch engine is one option): finds candidate pages for a bottle.
A page-fetch / extraction service that turns a URL into clean readable text (a self-hostable crawler is one option): pulls the full maker's page for the richest description.
A CPU background-knockout model. Removes a photo's background to a transparent cutout. The point of calling out the role: this is the one that runs happily on a CPU, so it can be a small always-on worker that never touches your GPU.
(Optional) a local image-generation model on a GPU, for atmospheric hero backdrops.
A relational store for the entities above. Any will do.

The whole point of the role list is that every one of these can be something local and free. No external AI API appears anywhere in the chain; the only standing cost is a couple of small always-on containers.

Principles that kept the build honest

The label is truth. What is printed on the bottle, and what the owner typed, outranks anything the web says. Enrichment is a draft that defers to it.
Fill blanks, never clobber. Web data writes only into empty fields and proposes corrections as advisories. A human edit is never overwritten by a machine.
Pricing is out, by design. Not searched, not stored, not displayed. A conversation piece, not a portfolio, and the thing that lets it all run free and local.
Run long operations as async jobs. Label reads and web enrichment both exceed a reverse proxy's request timeout; fire-and-poll so a tunnel or gateway timeout never kills the work.
Verify before you assert. A vision model that fabricates a confident ABV is worse than a blank field. Demand null-for-unknown, and put confidence badges where the human can see them.
Cite your sources. Every web-sourced fact links to where it came from, so a wrong claim is caught at a glance instead of festering as silent bad data.
Invented is labeled, never mingled. An AI-generated cocktail carries its own tier and badge and never sits among the real recipes. A fabrication that reads as vouched-for is exactly the thing to prevent.

Prior art and resources

You do not have to invent the taxonomy or the recommender from scratch; there is real public work to stand on, and pointing you to it is the opposite of exposing a private codebase:

Flavor taxonomies. The SWRI Scotch Whisky Flavour Wheel and a 2024 ML-derived distilled- spirits aroma wheel are the established controlled vocabularies; use them to define a ~50–70 term flavor-descriptor set, the backbone of any "what to pour" recommender.
An open-source whiskey reference API and dataset. A community project with a few hundred whiskies and per-item descriptor vectors, useful both as a reference architecture for the recommender (plain nearest-neighbor similarity, no learned embeddings) and as a seed dataset to bootstrap a flavor vocabulary.
Wikidata and government label registries for normalizing producer/region/brand names and verifying US-market bottles, as covered in the data-source section.
Self-hostable search and crawling tools for the enrichment pipeline (a metasearch engine and a page-to-text crawler, both runnable locally) are all the web side needs.

These are public community resources, not our build. Stand on them, then go your own way.

Working with your AI agent (starter prompts)

This brief is meant to be handed to an AI coding agent, not read like a manual. Below is a sequence of starter prompts that mirrors the suggested build order. Paste the whole brief into your agent first, then drive it phase by phase. Adapt the bracketed parts to your stack and your collection. These are a starting point, not a script.

0. Orient the agent and choose your stack.

I'm giving you a product brief for a personal, self-hosted spirits-collection catalog with photo-label intake. I want to build my own version for myself, with no subscription and my data on my own hardware. My environment is [language / framework / database / local vision model / local LLM]. Read the whole brief, then propose an architecture and a phase-1 plan that follows the suggested build order. Flag anywhere my stack changes the brief's assumptions, and ask me anything you need before you write code.

1. The catalog core, local-only.

Build phase 1 from the brief with no AI yet: the expression-vs-bottle data model (an expression is what a thing is; a bottle is what I own, with batch/barrel/lot to tell same-name bottles apart; quantity is the count of bottle rows), a faceted library list, a manual add-a-bottle form, and a detail page to view, edit, and delete. Generate the schema and the core screens.

2. Photo intake (the superpower).

Wire in a local vision model for label OCR. Let me upload or capture a photo, and read it only on a deliberate "Read this label" button (not automatically on upload). Pre-fill the add form with a per-field confidence badge: confident where it read printed text, "needs you" where it guessed. Read front and back faces and merge them (front wins identity, back wins production details and the producer's "from the label" prose; exclude the government warning). On save, match against what I own: same product → "add as another bottle?", genuinely different line → new expression. Have the model return null for facts the label doesn't state; never fabricate a number.

3. Clean up the photos.

Add a local CPU background-knockout service that, on photo ingest, writes a transparent product cutout next to the original (never modifying the original). Show the cutout floated on a styled backdrop in the catalog.

4. Web enrichment that cites its sources.

Build enrichment as a background job I poll (it exceeds a reverse-proxy timeout): for a bottle, search the web for "{name} {distillery} {category}" plus a second "{name} review tasting notes" pass, rank results (the maker's own domain first, a category-appropriate review/retailer allowlist next, drop the rest), and crawl the full text of the top few distinct domains, not just the official page, which is often a thin JS shell while the real specs live on review sites. Have a local LLM extract only the facts the sources state (description, flavor notes, region, ABV, age, cask, mash) plus the source links, and pass it ONLY my bottle's identity fields as context, never the whole DB row or any prior enrichment JSON (that makes the model think it's already done and return nothing). Store the distillery URL, product URL, and a cited Sources list. Fill any blank field, specs included; never overwrite my label reads or hand edits; surface web-vs-existing disagreements as admin-only "web says X · Use" advisories. Never search, extract, or store a price. Auto-enrich new bottles on save, and give me a one-time backfill over the existing collection. If a name or brand is ambiguous and the search returns junk, retry leading with the category.

5. The public showcase.

Build a separate, read-only process that registers only the public routes (the admin and all editing paths physically absent from it, not just auth-gated): image-forward cards, faceted browse generated from what I own, and a per-bottle detail page leading with the photo, then specs, then the producer's prose, then the cited sources as plain links. Omit owner-only fields (what I paid, where it's stored).

6. Cocktails, real recipes, a mixologist view, and off-menu invention.

Add cocktails in three layers. First, per-bottle recipe enrichment on the same search/rank/fetch/extract/cite pipeline, but tuned for recipe pages: keep a much larger text window (recipes sit below the marketing chrome), guard that a scraped drink actually uses the bottle's base-spirit family (drop cross-product leaks; exempt liqueurs and mixers), strip my brand and distillery tokens before de-duplicating titles, and seed the crawl from the bottle's own product/distillery URL before searching. Store real recipes in two labeled tiers: Signature (names this bottle) and Classic (canonical for the type, as a fallback). Validate that any record-supplied URL resolves to a public address before fetching it. Second, a mixologist view that browses all my recipes as one pool, filterable by spirit type, cocktail flavor, tasting note, and tier, with a sensible default order and tier headings. Third (optional), off-menu invention: one local-LLM call that invents a few original cocktails built only from bottles I own, each attached to every bottle it uses, badged "Invented" and never mixed in with the real recipes. If that endpoint is public, rate-limit it on an unforgeable client IP, and don't let saving one invention reload away the others.

A caution worth pasting when relevant:

The label on the bottle is ground truth; anything from the web is a draft that defers to it. Never let enrichment overwrite a value I read off the label or typed myself: fill blanks only, and propose everything else as an advisory I approve. And if there's no good source for a bottle, leave it untouched rather than writing a guess.

This brief documents the project through photo intake, the expression-vs-bottle model, web enrichment with cited sources, local image knockout, the public showcase, the private wishlist, and cocktails (real-recipe enrichment, a mixologist browse, and fenced off-menu invention). Pricing is out of scope by design, this is a conversation piece, not an investment tracker.