AI Models & Tools for Collection Management

This was a project that, by all accounts in this stepping stone era toward whatever AI is becoming, was pretty straightforward. It reminded me of the first time I really created a project in HTML, or the first time I animated something with interactivity in Flash, or that first time I connected ColdFusion to MySQL and ran a query against a database. These weren't monumental accomplishments, yet each one was a moment of clarity.

That clarity for me is that SaaS, in the way I suspect many people are pursuing it, is going to require a lot more than someone who can few-shot their way through a handful of disparate technologies and weave them into something coherent, because that's not all that special anymore. And I'm not here for that. I'm here to learn, share what I can, and push myself toward the next thing on my list of things I'm interested in learning.

With that, if you skip to the end, you'll get the sharing part: the PRD to build your own collection management tool using computer vision, OCR, and web scraping. I hope you do, and I hope it helps declutter some of your shelves, too.

After a trip that included a visit to our friends at Hanalei Spirits, I came home with six bottles of rum and dropped them into a collection that had been comfortably bourbon-and-whiskey-heavy until that exact moment. Or so I thought. It turns out I hadn't done a great job keeping track of what I actually own, which is dangerous territory. The shelf had tipped from "collection" into "wait, do I already own that?" It was time to organize and catalog what I had on hand.

That collection has grown a bottle or two at a time, picked up on trips and at tastings, each one a little more about the memory than the market. I don't collect as an investment; every bottle connects to a place or a moment along the way. It's the experience, really, and that's the part I wanted to memorialize and do a better job of keeping track of.

When I was looking at the app landscape, I found the usual stuff that makes me cringe: someone else owns the data, there's a pay gate with a monthly or annual fee, and there's no good way to easily show and share the collection, or just parts of it. I felt like what I wanted was a fairly simple record of what I own, without pouring time and energy into maintenance, and without adding the cost of a micro-subscription that gets used sporadically.

Like so many things lately, from coffee profiles to deep research tooling to automated heuristic analysis to custom workout programs, it was pretty obvious I already had the tools at home to build exactly what I wanted, without someone else's everything in the way.

So I built my own.

And so can you.

Read along if you're interested in the build, or skip to the end for a link to the PRD. The product requirements document is available so you can hand it to your own AI tool to build whatever you want or need, without relying on someone else's system, someone else's inference, and someone else's subscription fees.

Threading the Pieces Together

The infrastructure was already in place for me. This was less a quick win and more a matter of connecting things I'd already built. I'd used computer vision models before in a heuristics analysis tool, and I'd built web search and retrieval into both a deep research tool and a UX sentiment research tool. Reading a bottle label from a photo is essentially OCR, and pulling supporting detail off the web was a solved problem. All of the parts existed, the inference is local, and the work was wiring it all into a single connected flow.

Capturing a label was the logical starting point, however, labels on bottles pose a bit of a challenge: curved glass makes for imperfect reads. Factor in the occasional handwritten batch number, labels that wrap around, or worse, an angled label and it's pretty easy to trip up even a good vision model, so OCR alone was never going to be enough. To fill the gaps, I added web enrichment: search for the bottle, fetch the maker's own page, and pull in only the facts the sources actually state, with a list of those sources stored on every entry. That turned thin label reads into rich detail pages with almost no manual effort.

The refinement was less about hard problems and more about noticing gaps and subtle improvements that I could make as I went. Some of the default, and easy/most sensible sources are aggressively anti-bot and block scraping, so I had to find ones that welcomed the traffic, had the same or more information, and let me bring their information in to the system. A few rounds of testing the capture, making adjustments, and locking it down got data enrichment to a fairly robust place.

Then there were the photos. Bottles shot on a checkerboard mat in my basement, couch and wall art in the frame, look like exactly what they are: an amateur phone photographer doing his best to match size and angle with a messy room in the background. I used a local background-removal process running on CPU that knocks out the background and leaves a (mostly) clean cutout of the bottle, which the catalog floats on a styled backdrop that matches the page design. While it's not professional-grade, these are product-looking shots in seconds, again with next to no effort on my end, and just for a personal vanity project, which makes it all pretty amazing.

And that's kind of amusing to me, and says a lot about me, to me. I started this out saying "my needs are simple" and then I went and kept creeping on my own scope. Probably because I'm a designer. Most likely because I'm rotten with perfection.

The Fun Part (And It Was All Fun!)

Once all the pieces were more or less in place, the workflow was pretty simple:

• I take a photo of the bottle.
• The vision model reads the details.
• I make any quick edits, and save.
• Then I take additional photos of the back, the sides, or the neck (or anything else like a label, etc.), and save again.
• I tell the model to read all the labels, and review the updated results for edits, and save.
• If I'd like (and I'd always like), I run web enrichment, review the proposed edits, and/or elevate web enrichment metadata, and save.

Beyond adding any cursory notes, like where I purchased the bottle, if it's open or sealed, or any other notes I'd like, I've got a complete entry: high-quality details, cited references, a cleaned-up product shot, tags, and a published card.

And this all takes under a minute per bottle.

I'm a bit of a local inference / self-hosting nerd, so every model in that flow runs on my own hardware. The vision model that reads the label, the language model that extracts facts from the web, and the background removal that cleans the photos. I don't make calls to external AI APIs, and I don't pay a subscription to someone else's services, and I definitely am not paying per-token billing. The only cost is the hardware running quietly in a little server closet that doubles as a sump pump closet.

And one thing it never does, on purpose: it never captures pricing or collection value. That's the one thing the subscription apps sell that is likely of value to some folks, and it's the one thing I didn't want.

About the only thing I'm quite literally adding as I write this: a wishlist functionality so I can take a photo of a bottle in the wild, enrich it, and have it sitting in a tab reminding my future self that past me thought this was a bottle worth sleeping on. This works on the same foundation that the platform works on; I take a photo, read the label, enrich the content, and the information is now saved to my Wishlist. If future Russ gets financial approval to make the investment in the collection, he can promote that bottle to the live list, update the details (better photos, most likely), and then it'll be part of the collection.

You Can Make Your Own; I'll Help! (Or: Why I'm Not Selling This)

I'm increasingly convinced that SaaS has to be genuinely special to be worth building, and a personal spirits catalog isn't that. I don't want to maintain a database for other people, I don't want to keep scrapers alive as websites change, I don't want to referee whether someone prefers a different model, different fields, tasting notes, a sip log, or whatever else they'd want their version to do. And none of those are negatives at all - they're maintenance and operations labor that require me to be at someone else's disposal for their happiness when it comes to how they want to maintain their collection.

So instead of creating a SaaS app with only one user (me), or signing up to spend unlimited time maintaining a passion project on other people's behalf, I'm giving away the recipe.

I wrote a full PRD for Spirit Spy: what it does, how the pieces fit, and the specifics about reading labels and enriching from the web that will save you the fiddly parts. There's no code or repository to wrangle, review, or wonder whether it's quietly tracking you. You hand the brief to your own AI tool, in your own environment, and build your own version, on your own hardware, owning your own data. And if you want to log every sip along the way and turn it into a full-blown diary per bottle, I think that's amazing. For you!

Please review it first to make sure it's clean and fits your needs, or that it gives you enough of a shape to modify. Talk it through with your AI, change whatever you like, and republish with all the cool things you've added that make it better.

Let me know what those are, because I'm sure I'm forgetting things that could make my own collection cooler, and I am, after all, rotten with perfection.

None of this is wholly about spirits, either. This is about a collection, and the same framing works for just about any collection you care about. Swap the metadata and it does what you need. My comics are next.

Read the build-it-yourself PRD ↗ Download the Markdown ↓

Enjoy!