Context Adherence

Understand Galileo's Context Adherence Metric

Definition: Context Adherence is a measurement of closed-domain hallucinations: cases where your model said things that were not provided in the context.

If a response is adherent to the context (i.e. it has a value of 1 or close to 1), it only contains information given in the context. If a response is not adherent (i.e. it has a value of 0 or close to 0), it's likely to contain facts not included in the context provided to the model.

Luna vs Plus

We offer two ways of calculating Context Adherence: Luna and Plus.

Context Adherence Luna is computed using Galileo in-house small language models (Luna). They're free of cost, but lack 'explanations'. Context Adherence Luna is a cost effective way to scale up you RAG evaluation workflows.

Context Adherence Plus is computed using the Chainpoll technique. It relies on OpenAI models so it incurs an additional cost. Context Adherence Plus has shown better results in internal benchmarks. Additionally, Plus offers explanations for its ratings (i.e. why something was or was not adherent).