The Entity Resolution Saga

Voice interfaces have a problem that text interfaces mostly avoid: ambiguity. When you type a site name into a search box, you can see a dropdown of matches and pick the right one. When you say a site name out loud to Siri, there's no dropdown. The system has to decide, in real time, which real object in your Mist organization you meant, and it has to be right almost all of the time, or the feature stops feeling reliable and starts feeling like a toy.

The naive first approach, and why it wasn't enough

The first version of entity resolution in MistPulse did the obvious thing: take the spoken text, normalize it (lowercase, trim whitespace), and look for an exact string match against the list of site names in the organization. This works perfectly in a demo, because you control exactly what you say. It falls apart quickly in real use, for reasons that only show up once you're not the one carefully enunciating a name into a microphone.

Speech-to-text transcription introduces its own variation: Siri might transcribe a spoken site name slightly differently depending on background noise, accent, or how the name is phonetically structured. A site named with an underscore or a specific capitalization pattern in Mist's configuration doesn't map cleanly onto how a person naturally says that name out loud. And multi-word names introduce word-order and spacing questions that an exact match simply can't tolerate.

An exact match against a spoken name works perfectly in a demo, and falls apart the moment a real person, not a script, is talking to it.

Moving to fuzzy matching with a confidence threshold

The fix was to stop requiring exact matches and instead score how close the spoken text is to each known site or device name, using string-similarity techniques that tolerate small differences in spelling, spacing, and word order. Instead of a binary match/no-match, every candidate gets a similarity score, and the system picks the best match, but only if that best match clears a confidence threshold high enough to be trustworthy.

That threshold turned out to be the hardest number in the entire project to tune. Set it too loose, and the app confidently resolves a spoken name to the wrong site, which is worse than not resolving at all, because a wrong answer delivered with confidence is exactly the kind of failure that destroys trust in a tool fast. Set it too strict, and legitimate matches get rejected for trivial transcription differences, and the feature feels broken even when it's technically working as designed.

• Exact string matching failed on real speech-to-text variation almost immediately
• Fuzzy similarity scoring against all known entity names solved the variation problem
• A confidence threshold prevents low-quality matches from being presented as if they were certain
• Below the threshold, the honest answer is "I couldn't find a clear match" rather than a guess

Designing for graceful ambiguity, not silent failure

The most important design principle that came out of this saga wasn't a specific algorithm; it was a philosophy: when the system isn't confident, it should say so, rather than pick its best guess and present it as fact. If a spoken name is genuinely ambiguous between two sites, or doesn't clear the confidence bar for any of them, the response needs to communicate that uncertainty honestly rather than silently returning data for the wrong site.

This mirrors something I care about deeply on the security and reliability side of my day job: a system that fails loudly and honestly is far safer to operate than one that fails silently and confidently. A network monitoring tool that occasionally reports the wrong device's health, without any indication that it might be wrong, is more dangerous than one that simply says "I'm not sure", because the first one erodes the operator's ability to trust any of its output, while the second one preserves that trust by being honest about its limits.

What this taught me about testing voice features

Testing this properly meant something different from the curl-based API validation covered in Part 3. It meant collecting a real, varied set of transcription outputs, actual phrasings, actual near-misses, and running the matching logic against that set repeatedly as the threshold and scoring approach were tuned, rather than testing against a handful of clean, idealized inputs. It's a reminder that "testing" has to be adapted to what a feature actually depends on: for an API integration, that's real API responses; for a voice interface, that's real, messy, human speech patterns.

With entity resolution solid enough to trust, the app's feature set kept growing, and growing features safely, without duplicating logic across similar-but-not-identical use cases, became its own discipline. That's the subject of the next post: the story of generalizing a single-purpose tool into something that could serve multiple device types without doubling the code or the risk.

Coming up next: Generalizing Features: the bulk rename story, and what it taught me about building for reuse without overbuilding