LLM Semantic Cache: Medium Coding Interview Problem

LLM Semantic Cache

LLM generation is slow and expensive. Since users often ask similar questions, production architectures run a Semantic Cache. When a query is asked, you compute its embedding and check if a highly similar query has already been answered.

Tasks

Implement the function lookup_semantic_cache(query_embedding, cache, threshold) to:

Compute the cosine similarity between the 1D query embedding $q$ and each cached query embedding $c$.

$\text{CosineSimilarity}(q, c) = \frac{q \cdot c}{|q| |c|}$
Return the text response of the entry in the cache that yields the highest cosine similarity, provided that similarity is $\ge threshold$.
Return None if the cache is empty, or if no entry meets the threshold, or if vectors are zero vectors.

Constraints:

Use numpy for vector operations. Do not use loops where numpy vectors can be calculated.