GuideAdvanced

LLM Search and Recommendation

A store's search box and its "you may also like" section are product: they decide what each shopper sees and, therefore, what sells. This guide teaches you to build them and improve them with LLMs without losing sight of what actually matters: measuring. You walk the whole funnel, from candidate retrieval (lexical, vector and hybrid) to understanding the query, ranking with a trained ranker, reranking with an LLM and applying business rules; and on the recommendation side, from popularity and collaborative filtering to semantic IDs, generative retrieval and hybrids where an LLM interprets and a classical recommender ranks. You learn it at **Kestrova**, a fictional outdoor gear store with 560 products, 1,500 shoppers and 92 queries with relevance judgments. You see why a lexical search returns gaiters when someone asks for waterproof boots, how much you gain by understanding the query with an LLM versus reranking with one (and what each costs per thousand searches), how a reranker's position bias changes the result depending on the order you hand it the candidates, why an LLM judge can flip the comparison between systems when the pool is built wrong, how a semantic ID lets you recommend a product nobody has bought yet, and why interleaving detects with far less traffic a difference that an A/B test cannot see. Everything runs on your laptop without an API key: a deterministic synthetic catalog, BM25 written from scratch, vectors computed with numpy (no model downloads), a simulated model with the same response shape as the Claude Messages API, and real metrics (recall@k, MRR, NDCG, bootstrap intervals) computed on that data. You see exactly the same numbers as the guide. An optional `--live` mode uses the real API with structured outputs.