Context Adherence Basic

Understand Galileo's Context Adherence Basic Metric

The metric is intended for RAG workflows.

Definition: Measures whether your model's response was purely based on the context provided. A high Context Adherence score means your response is supported by the context provided.

Context Adherence is a measurement of closed-domain hallucinations: cases where your model said things that were not provided in the context.

If a response is adherent to the context (i.e. it has a value of 1 or close to 1), it only contains information given in the context. If a response is not adherent (i.e. it has a value of 0 or close to 0), it's likely to contain facts not included in the context provided to the model.

Calculation: Context Adherence Basic is computed using a fine-tuned in-house Galileo evaluation model. The model is a transformer-based encoder that predicts the probability of Context Adherence for an input response and context. The model is trained on carefully curated RAG datasets and optimized to mimic the Context Adherence Plus metric.

The same model is used to compute Chunk Adherence, Chunk Completeness, Chunk Attribution and Utilization, and a single inference call is used to compute all the Basic metrics at once.

At the moment, we don't offer explanations for Context Adherence Basic.