RAG Semantic Completion and Recommendation

An empty search result does not always mean that the platform has no relevant product. A customer may use a rare term, make a spelling mistake, use non-standard wording, or search for a new concept with little behavioural data. Keyword rules cannot always connect those expressions to the catalogue, creating a zero-hit query even when plausible products exist.

This prototype added a semantic fallback to traditional search. A product-text knowledge index first interpreted the query and expanded candidate terms; suitable cases could then receive recommendation content. The system supplemented the existing search path instead of replacing it.

Role and key responsibilities

I independently designed, developed, and deployed the prototype. The work included:

• building a searchable knowledge index from product titles, categories, attributes, marketing copy, and reviews;
• using vector representations to find related products and expressions that keyword rules missed;
• designing tiered responses based on confidence, from semantic candidates to conservative suggestions or the unchanged original result;
• caching common completions in Redis to reduce repeated computation and external model calls;
• generating and refreshing recommendation copy asynchronously so an LLM did not block the primary search request;
• moving thresholds, policies, and fallback conditions into configuration for limited rollout and rapid disablement.

Keeping the LLM outside the critical path

Letting a language model decide every search response in real time would introduce variable latency, cost, and content risk. The design therefore separated online retrieval from content generation. Live requests used the prepared semantic index and cached outputs, while the LLM generated reusable recommendation copy in the background under explicit policy control.

The primary search service remained independent. If the model service slowed down or failed, the platform could fall back to semantic candidates or conventional search instead of turning a generation failure into a search outage.

Establishing confidence

Semantic similarity alone was not enough to rewrite a customer query. The completion policy also considered candidate volume, category consistency, and feedback. With insufficient confidence, the interface could offer a suggestion without replacing the original query; with no credible evidence, it preserved the existing behaviour.

Online measures included completion latency, zero-hit recall, interaction with recommendations, incorrect-expansion rate, and cache behaviour. The prototype could therefore be evaluated as a system rather than through a small set of successful demonstrations.

Outcomes

• Completion latency remained below 250 ms.
• Recall on zero-hit queries improved by approximately 35%.
• Caching, asynchronous execution, and configurable policy isolated the generative capability from the primary search path and provided an explicit rollback mechanism.

The project demonstrated that RAG and LLMs can usefully supplement production search when they are placed inside an architecture with latency boundaries, evidence-based decisions, and fault isolation.