Context Adherence

The metric is mainly intended for RAG workflows.

Definition: Measures whether your model's response was purely based on the context provided. A high Context Adherence score means your response is supported by the context provided.

Context Adherence is a measurement of closed-domain hallucinations: cases where your model said things that were not provided in the context.

If a response is adherent to the context (i.e. it has a value of 1 or close to 1), it only contains information given in the context. If a response is not adherent (i.e. it has a value of 0 or close to 0), it's likely to contain facts not included in the context provided to the model.

Basic vs Plus

We offer two ways of calculating Context Adherence: Basic and Plus.

Context Adherence Basic is computed using Galileo in-house small language models. They're free of cost, but lack 'explanations'. Context Adherence Basic is a cost effective way to scale up you RAG evaluation workflows.

Context Adherence Plus is computed using the Chainpoll technique. It relies on OpenAI models so it incurs an additional cost. Context Adherence Plus has shown better results in internal benchmarks. Additionally, Plus offers explanations for its ratings (i.e. why something was or was not adherent).