Quickstart

How to monitor your apps with Galileo Observe

Getting started with Galileo Observe is really easy. It involves 3 steps:

1. Create a project

2. Integrate Galileo in your code

3. Choose your Guardrail metrics

Create a Project

The first step is creating a project from the Galileo UI.

1. Go to Galileo Console

2. Click on the big + icon on the top left

3. Follow the steps to create your Observe project.

Integrate into your code

Galileo Observe can integrate via Langchain callbacks, our Python Logger, or via RESTFul APIs.

Authentication

To authenticate Galileo, perform the following steps

1. Add your Galileo Console URL (GALILEO_CONSOLE_URL) to your environment variables.

import os

os.environ['GALILEO_CONSOLE_URL']="https://console.galileo.XYZ.com"
# Ask your Galileo ADMIN for the URL if you don't know

2. Set your user name and password OR your API key (recommended) to your environment variables

import os

os.environ["GALILEO_USERNAME"]="Your Galileo registered email e.g. [e-mail]"
os.environ["GALILEO_PASSWORD"]="Your Galileo password"

# OR

os.environ["GALILEO_API_KEY"]="Your Galileo API key"

We recommend using an API key to authenticate. To create one, go to "API Keys" under your profile bubble.

Getting an API Key

To create an API key:

1. Go to your Galileo Console settings

2. Go to API Keys

3. Select Create a new key

Integrating with Langchain

We support integrating into both Python-based and Typescript-based Langchain systems:

Python

Integrating into your Python-based Langchain application is the easiest and recommended route. You can just add GalileoObserveCallback(project_name="YOUR_PROJECT_NAME") to the callbacks of your chain invocation.

from galileo_observe import GalileoObserveCallback
from langchain.chat_models import ChatOpenAI

prompt = ChatPromptTemplate.from_template("tell me a joke about {foo}")
model = ChatOpenAI()
chain = prompt | model

monitor_handler = GalileoObserveCallback(project_name="YOUR_PROJECT_NAME")
chain.invoke({'foo':'bears'}, 
             config(dict(callbacks=[monitor_handler])))

The GalileoObserveCallback logs your input, output, and relevant statistics back to Galileo, where additional evaluation metrics are computed.

Typescript

Integrating into your Typescript-based Langchain application is a very simple process. You can just add aGalileoObserveCallback object to the callbacks of your chain invocation.

import { GalileoObserveCallback } from "@rungalileo/observe";
const observe_callback = new GalileoObserveCallback("observe_example", "app_v1")
await observe_callback.init();

Add the callback {callbacks: [observe_callback]} in the invoke step of your application:

const result = await chain.invoke(
    { question: "What is the powerhouse of the cell?"}, 
    {callbacks: [observe_callback]});

The GalileoObserveCallback callback logs your input, output, and relevant statistics back to Galileo, where additional evaluation metrics are computed.

Non Langchain: Custom Logging via Python Logger

If you're not using LangChain, you can use our Python Logger to log your data to Galileo. The Logger automatically batches and uploads data in the background asynchronously, ensuring the performance and reliability of your application is not compromised.

Use the log_prompt() and log_completion() to log your prompt, model, hyperparameters and the response respectively, as shown below:

from galileo_observe import GalileoObserveCallback

monitor = GalileoObserveCallback(project_name="YOUR_PROJECT_NAME")

# Example Prompt, Model and Temperature
prompt = "Tell me a joke about Large Language Models"
model = "gpt-3.5-turbo"
temperature = 0.3

# Call log_prompt before response generation.
trace_id = monitor.log_prompt(
    prompt=prompt,
    model=model,
    temperature=temperature,
)

# Generic Chat Completion 
chat_completion = openai.ChatCompletion.create(
    model=model,
    messages=[{"role": "user", "content": prompt}],
    temperature=temperature,
)

# Call log_completion after response generation.
monitor.log_completion(
    node_id=trace_id,
    output_text=chat_completion["choices"][0]["message"]["content"],
    num_input_tokens=chat_completion["usage"]["prompt_tokens"],
    num_output_tokens=chat_completion["usage"]["completion_tokens"],
    num_total_tokens=chat_completion["usage"]["total_tokens"],
    finish_reason=chat_completion["choices"][0]["finish_reason"],
    user_metadata={"env": "production"},
    tags=["prod", "chat"]
)

Logging through our REST APIs

If you are looking to log directly from the client (e.g. using Javascript), you can log directly through our APIs. The Monitor API endpoints can be found on the swagger docs for your environment. More instructions can be found here.

Once you've integrated Galileo into your production app code, you can choose your Guardrail metrics.