Lesson 5 · 10 min
Embedding drift and production monitoring
Embedding models change. Corpora drift. When your retrieval quality degrades silently in production, embedding drift is often the cause — and it requires different monitoring than traditional software metrics.
Why embeddings drift
Model updates: Your embedding model provider updates the model. The new model produces different vector representations for the same text. Documents embedded with the old model and queries embedded with the new model are now in incompatible spaces — similarity scores become meaningless.
Corpus drift: Your knowledge base gains new content over time. If new documents introduce vocabulary or concepts not represented in the existing index, retrieval for those topics degrades without any model change.
Query distribution shift: Users' queries change over time. If your initial retrieval eval set was collected at launch but users' language has evolved (new product features, new terminology), eval metrics may look stable while production quality degrades.
All three types of drift are real and have caused production incidents.