Galileo
Search
⌃K
🧠

Galileo & LLMs (Coming Soon!)

Galileo has in built support for 2 key LLM workflows -
  1. 1.
    Prompt Engineering Tooling
  2. 2.
    Sequence to Sequence Fine Tuning Models

LLM Inference - Prompt Engineering Tooling

There are 5 Key ingredients of Galileo's Prompt Evaluation tool
  1. 1.
    The Prompt Evaluation Method (the core algorithm)
    1. 1.
      F1 (between ground truth & model output)
    2. 2.
      Prompt Error Potential (PEP)
    3. 3.
      Hallucination Score (likelihood of the model hallucinating in the output)
    4. 4.
      Edit Distance or Semantic Distance between output and GT
    5. 5.
      BLEU / BERT / ROGUE scores
    ​
  2. 2.
    Automating RAG (Retrieval Augmentation with Context)
    1. 1.
      With Generic OOTB Embeddings (V1)
    2. 2.
      With Customized Embeddings (V2 + more suitable for Enterprises)
    ​
  3. 3.
    ​
    Galileo Prompting Architecture
  4. 4.
    Template Management
    1. 1.
      CRUD operations on Templates: Add, delete and edit templates
    2. 2.
      Template Registry: Have a registry of the user's templates
    3. 3.
      Template Versioning: Prompt templates can be edited by the user, hence tracking version history of prompts becomes important
  5. 5.
    Evaluating 2 types of interactions
    1. 1.
      Zero shot prompts
    2. 2.
      Multi-shot / Chain of Thought Prompting
Previous
Tabular Data classification
Next - How-To and FAQ
Python SDK
Last modified 1mo ago