Vector databases search by meaning, not keywords — that's what makes RAG useful. But in enterprise production, the harder problem isn't semantic quality. It's scoping: finding the right documents in the right business context, without a schema change turning into a reindexing project.
The Scoping Problem
An engineer searches: "Users cannot log in after resetting their password."
Semantic search correctly surfaces related incidents — different wording, same underlying issue. But the engineer usually only wants incidents scoped to Customer: Acme Corp, Product: Identity Platform, Environment: Production. Without that scoping, similarity search happily returns a topically-relevant incident from someone else's production environment — relevant in the abstract, useless for the ticket at hand.
Where Metadata Filtering Breaks Down
The standard fix is metadata filtering: tag documents at index time (customer, product, environment), then AND a filter clause onto the similarity search. This works — until the schema needs a new field.
Add compliance_classification or support_tier six months in, and in most architectures that means: extend the schema, backfill it onto existing documents, and reindex the affected vectors. For a small corpus, that's an afternoon. At millions of vectors with ongoing ingestion, it's recurring infrastructure cost — every time filtering requirements grow.
Two Search Modes, and How Results Get Ranked
WaveflowDB runs hybrid search (semantic + filter) in one of two modes:
`flat` — Full Corpus Fusion Semantic search and the keyword/content filter each run independently across all documents. Results are then combined by a proprietary algorithm into a single ranked list, split into three tiers:
- •Tier 1: matched both filter and semantic search — highest confidence
- •Tier 2: filter match only — structured match, lower semantic relevance
- •Tier 3: semantic match only — meaning-based fallback, no filter alignment
Use this for exploratory search, where you want every document to have a chance to surface even if it only partially matches.
`flat_filter` — Filtered Semantic Search The filter runs first as a hard gate — only documents that pass are eligible for semantic ranking at all. No tiers here: it's a single-pass, narrower, higher-precision result set. Use this when filter criteria are strong and you want tight, targeted retrieval rather than broad recall.
Filtering Against Content Instead of a Schema
WaveflowDB's approach: stop treating filterable fields as something that must be pre-indexed. Instead of checking a customer metadata field, the query engine checks the document's content directly for Acme Corp — no metadata JSON filter step, no predefined schema to extend. If the information is in the document, it's filterable immediately.
The CONTAINS constraint is pre-filter and exact-match — applied before/alongside the semantic ranking, not fuzzy or embedding-based. That split is deliberate: semantic ranking answers "what does this mean," CONTAINS answers "is this true," and a document has to pass every CONTAINS clause to be eligible before similarity ranking even applies to it.
That's the core of deterministic retrieval: semantic ranking (probabilistic) and content constraints (deterministic — pass or fail, no partial credit) evaluated together in one query, instead of semantic search first and a brittle metadata filter bolted on after.
Expressing It in VQL
SELECT TOP 5
WHERE QUERY IS
"Users cannot log in after resetting their password"
CONTAINS
{customer:"Acme Corp"}
AND {product:"Identity Platform"}
AND {version:"4.2"}
AND {environment:"Production"}QUERY IS drives semantic ranking; CONTAINS drives exact-match content constraints. This is a minimal example — see WaveflowDB's documentation for the full VQL syntax and more query examples.
Keeping the Index Current
Content changes constantly, and reprocessing the whole corpus on every edit doesn't scale. WaveflowDB detects what actually changed using a combination of hashing, embedding diffing and a proprietary reconciliation algorithm that combines these signals to re-embed only the changed spans and update only the affected index entries — not the whole document.
Trade-offs
- •Exact-match can miss variation. "Acme Corporation" won't satisfy {customer:"Acme Corp"}. Without a predefined schema, there's also no forced normalization step — inconsistent source content produces inconsistent filtering.
- •Content-evaluated constraints trade indexing-time cost for query-time cost, not eliminate cost. Performance profile differs from a metadata index hit.
- •Not a universal replacement. For high-QPS lookups on a small set of stable, well-known fields, a conventional indexed metadata filter can still win. This approach targets the case where filter requirements evolve faster than you want to maintain a schema for.
- •"Proprietary algorithm" is doing real work in the change-detection story above — fair for technical readers to want more transparency than one line offers; worth raising directly with WaveflowDB if auditability matters for your use case.
Why It Matters
Semantic understanding is largely solved. The harder problem is keeping retrieval accurate and maintainable as data and filtering requirements keep changing underneath you. Deterministic retrieval decouples "what can I filter on" from "what did I predefine at index time" — it doesn't remove every trade-off, but it removes the one that hurts most at scale: rebuilding the index every time the business asks a new kind of question.