Setting Up Alerts

How to set up Alerts and automatically be alerted when things go wrong
Galileo enables you to get alerted whenever unexpected things happen (e.g. your cost is higher than expected, your model is hallucinating more than you want, users are entering foul language into your app).
Before setting up alerting, make sure you:


To set up your alerts, you need to define:
  1. 1.
    Who should be alerted (and how)
  2. 2.
    What they should be alerted on
Your Alerting Settings will be under your Project Settings page (i.e. the
icon near the top-right of your Monitoring Dashboard).
To define who should be alerted, scroll to the Alerting section, and add your recipients' email addresses:
Then configure your alerts:
Each alert is composed of a Metric (e.g. Correctness, Cost, Toxicity), an Aggregation Function (e.g. Min, Max, Average, Total), a threshold (e.g. greater than 0.5), and a time window (e.g. hourly).
Example Alerts:
  • Exceeding costs: If you want to get alerted with an uptick in cost (above $10/day, you select Sum Value of Cost is more than or equal to your 10 in the last day.
  • Hallucinations: If you want to get alerted any time you have a hallucination, select Min Value of Correctness or Context Adherence is equal to 0.
  • Hallucination Rate: If you're comfortable with a certain level of hallucinations (e.g. 5%), you can select Average value of Correctness or Context Adherence is less than or equal to 0.05.
Triggering Alerts
Once your Alerts are configured, we periodically run jobs to check whether your Alerts have been triggered. Once they do, you should receive an email letting you know which alert has triggered and what the value of the alert is.
From your email, you can click on the "Open Project" link to open your dashboard and find the problematic requests.