Galileo
Search
⌃K

Types

Subpackages

Submodules

promptquality.types.config module

pydantic settings Config

Bases: BaseSettings
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "Config", "type": "object", "properties": { "console_url": { "format": "uri", "maxLength": 2083, "minLength": 1, "title": "Console Url", "type": "string" }, "username": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Username" }, "password": { "anyOf": [ { "format": "password", "type": "string", "writeOnly": true }, { "type": "null" } ], "default": null, "title": "Password" }, "token": { "anyOf": [ { "format": "password", "type": "string", "writeOnly": true }, { "type": "null" } ], "default": null, "title": "Token" }, "current_user": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Current User" }, "current_project_id": { "anyOf": [ { "format": "uuid4", "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Project Id" }, "current_project_name": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Project Name" }, "current_run_id": { "anyOf": [ { "format": "uuid4", "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Run Id" }, "current_run_name": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Run Name" }, "current_run_task_type": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Current Run Task Type" }, "current_template_id": { "anyOf": [ { "format": "uuid4", "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Template Id" }, "current_template_name": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Template Name" }, "current_template_version_id": { "anyOf": [ { "format": "uuid4", "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Template Version Id" }, "current_template_version": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Current Template Version" }, "current_template": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Template" }, "current_dataset_id": { "anyOf": [ { "format": "uuid4", "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Dataset Id" }, "current_job_id": { "anyOf": [ { "format": "uuid4", "type": "string" }, { "type": "null" } ], "default": null, "title": "Current Job Id" } }, "required": [ "console_url" ] } ```
* **Config:** - **extra**: *str = ignore* - **env_prefix**: *str = GALILEO_* - **validate_assignment**: *bool = True* * **Fields:** - [`console_url (pydantic_core._pydantic_core.Url)`](#promptquality.types.config.Config.console_url) - [`current_dataset_id (uuid.UUID | None)`](#promptquality.types.config.Config.current_dataset_id) - [`current_job_id (uuid.UUID | None)`](#promptquality.types.config.Config.current_job_id) - [`current_project_id (uuid.UUID | None)`](#promptquality.types.config.Config.current_project_id) - [`current_project_name (str | None)`](#promptquality.types.config.Config.current_project_name) - [`current_run_id (uuid.UUID | None)`](#promptquality.types.config.Config.current_run_id) - [`current_run_name (str | None)`](#promptquality.types.config.Config.current_run_name) - [`current_run_task_type (int | None)`](#promptquality.types.config.Config.current_run_task_type) - [`current_template (str | None)`](#promptquality.types.config.Config.current_template) - [`current_template_id (uuid.UUID | None)`](#promptquality.types.config.Config.current_template_id) - [`current_template_name (str | None)`](#promptquality.types.config.Config.current_template_name) - [`current_template_version (int | None)`](#promptquality.types.config.Config.current_template_version) - [`current_template_version_id (uuid.UUID | None)`](#promptquality.types.config.Config.current_template_version_id) - [`current_user (str | None)`](#promptquality.types.config.Config.current_user) - [`password (pydantic.types.SecretStr | None)`](#promptquality.types.config.Config.password) - [`token (pydantic.types.SecretStr | None)`](#promptquality.types.config.Config.token) - [`username (str | None)`](#promptquality.types.config.Config.username) * **Validators:** - [`http_url`](#promptquality.types.config.Config.http_url) » [`console_url`](#promptquality.types.config.Config.console_url) - [`validate_api_url`](#promptquality.types.config.Config.validate_api_url) » `all fields`

field console_url*: Annotated[Url]* [Required]

  • Constraints:
    • max_length = 2083
    • allowed_schemes = [‘http’, ‘https’]
  • Validated by:

field current_dataset_id*: Optional[Annotated[UUID]]* = None

field current_job_id*: Optional[Annotated[UUID]]* = None

field current_project_id*: Optional[Annotated[UUID]]* = None

field current_project_name*: Optional[str]* = None

field current_run_id*: Optional[Annotated[UUID]]* = None

field current_run_name*: Optional[str]* = None

field current_run_task_type*: Optional[int]* = None

field current_template*: Optional[str]* = None

field current_template_id*: Optional[Annotated[UUID]]* = None

field current_template_name*: Optional[str]* = None

field current_template_version*: Optional[int]* = None

field current_template_version_id*: Optional[Annotated[UUID]]* = None

field current_user*: Optional[str]* = None

field password*: Optional[SecretStr]* = None

field token*: Optional[SecretStr]* = None

field username*: Optional[str]* = None

validator http_url » console_url​

  • Return type: str

login()

  • Return type: None

logout()

  • Return type: None

merge_dataset(dataset)

  • Return type: None

merge_job(job)

  • Return type: None

merge_project(project)

  • Return type: None

merge_run(run)

  • Return type: None

merge_template(template)

  • Return type: None

merge_template_version(template_version)

  • Return type: None

classmethod read()

serialize_token(value)

  • Return type: str

token_login()

  • Return type: str

validator validate_api_url » all fields

write()

  • Return type: None

property api_client*: ApiClient*

property api_url*: Url*

property config_file*: Path*

property project_url*: str*

property run_url*: str*

promptquality.types.custom_scorer module

pydantic model CustomScorer

Bases: BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "CustomScorer", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "executor": { "default": null, "title": "Executor" }, "aggregator": { "default": null, "title": "Aggregator" } }, "required": [ "name" ] } ```
* **Fields:** - [`aggregator (Callable[[List[float | int | bool | str | None], List[int]], Dict[str, float | int | bool | str | None]] | None)`](#promptquality.types.custom_scorer.CustomScorer.aggregator) - [`executor (Callable[[promptquality.types.rows.PromptRow], float | int | bool | str | None])`](#promptquality.types.custom_scorer.CustomScorer.executor) - [`name (str)`](#promptquality.types.custom_scorer.CustomScorer.name)

field aggregator*: Optional[Callable[[List[Union[float, int, bool, str, None]], List[int]], Dict[str, Union[float, int, bool, str, None]]]]* = None

field executor*: Callable[[PromptRow], Union[float, int, bool, str, None]]* [Required]

field name*: str* [Required]

promptquality.types.pagination module

pydantic model PaginationRequestMixin

Bases: BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "PaginationRequestMixin", "type": "object", "properties": { "starting_token": { "default": 0, "title": "Starting Token", "type": "integer" }, "limit": { "default": 25, "title": "Limit", "type": "integer" } } } ```
* **Fields:** - [`limit (int)`](#promptquality.types.pagination.PaginationRequestMixin.limit) - [`starting_token (int)`](#promptquality.types.pagination.PaginationRequestMixin.starting_token)

field limit*: int* = 25

field starting_token*: int* = 0

pydantic model PaginationResponseMixin

Bases: PaginationRequestMixin, BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "PaginationResponseMixin", "type": "object", "properties": { "starting_token": { "default": 0, "title": "Starting Token", "type": "integer" }, "limit": { "default": 25, "title": "Limit", "type": "integer" }, "paginated": { "default": false, "title": "Paginated", "type": "boolean" }, "next_starting_token": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Next Starting Token" } } } ```
* **Fields:** - [`next_starting_token (int | None)`](#promptquality.types.pagination.PaginationResponseMixin.next_starting_token) - [`paginated (bool)`](#promptquality.types.pagination.PaginationResponseMixin.paginated)

field next_starting_token*: Optional[int]* = None

field paginated*: bool* = False

promptquality.types.registered_scorers module

pydantic model RegisteredScorer

Bases: BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "RegisteredScorer", "type": "object", "properties": { "registered_scorer_id": { "format": "uuid4", "title": "Registered Scorer Id", "type": "string" }, "metric_name": { "title": "Metric Name", "type": "string" } }, "required": [ "registered_scorer_id", "metric_name" ] } ```
* **Config:** - **populate_by_name**: *bool = True* * **Fields:** - [`id (uuid.UUID)`](#promptquality.types.registered_scorers.RegisteredScorer.id) - [`name (str)`](#promptquality.types.registered_scorers.RegisteredScorer.name)

field id*: Annotated[UUID]* [Required] (alias 'registered_scorer_id')

  • Constraints:
    • uuid_version = 4

field name*: str* [Required] (alias 'metric_name')

pydantic model ListRegisteredScorers

Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "ListRegisteredScorers", "type": "object", "properties": { "starting_token": { "default": 0, "title": "Starting Token", "type": "integer" }, "limit": { "default": 25, "title": "Limit", "type": "integer" }, "paginated": { "default": false, "title": "Paginated", "type": "boolean" }, "next_starting_token": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Next Starting Token" }, "scorers": { "items": { "$ref": "#/$defs/RegisteredScorer" }, "title": "Scorers", "type": "array" } }, "$defs": { "RegisteredScorer": { "properties": { "registered_scorer_id": { "format": "uuid4", "title": "Registered Scorer Id", "type": "string" }, "metric_name": { "title": "Metric Name", "type": "string" } }, "required": [ "registered_scorer_id", "metric_name" ], "title": "RegisteredScorer", "type": "object" } }, "required": [ "scorers" ] } ```
* **Fields:** - [`scorers (List[promptquality.types.registered_scorers.RegisteredScorer])`](#promptquality.types.registered_scorers.ListRegisteredScorers.scorers)

field scorers*: List[RegisteredScorer]* [Required]

promptquality.types.rows module

pydantic model GetRowsRequest

Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "GetRowsRequest", "type": "object", "properties": { "starting_token": { "default": 0, "title": "Starting Token", "type": "integer" }, "limit": { "default": 25, "title": "Limit", "type": "integer" }, "project_id": { "format": "uuid4", "title": "Project Id", "type": "string" }, "run_id": { "format": "uuid4", "title": "Run Id", "type": "string" }, "task_type": { "title": "Task Type", "type": "integer" } }, "required": [ "project_id", "run_id", "task_type" ] } ```
* **Fields:** - [`project_id (uuid.UUID)`](#promptquality.types.rows.GetRowsRequest.project_id) - [`run_id (uuid.UUID)`](#promptquality.types.rows.GetRowsRequest.run_id) - [`task_type (int)`](#promptquality.types.rows.GetRowsRequest.task_type)

field project_id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4

field run_id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4

field task_type*: int* [Required]

params()

Params to be passed to the API request.
These are primarily the pagination parameters and task type.
  • Returns: Params to be passed to the API request.
  • Return type: Dict[str, Any]

pydantic model Metrics

Bases: BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "Metrics", "type": "object", "properties": { "hallucination": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Hallucination" }, "bleu": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Bleu" }, "rouge": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Rouge" }, "pii": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "title": "Pii" }, "like_dislike": { "anyOf": [ { "type": "boolean" }, { "type": "null" } ], "default": null, "title": "Like Dislike" }, "toxicity": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Toxicity" }, "factuality": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Factuality" }, "factuality_explanation": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Factuality Explanation" }, "groundedness": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Groundedness" }, "groundedness_explanation": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Groundedness Explanation" }, "latency": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Latency" }, "context_relevance": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Context Relevance" } }, "additionalProperties": true } ```
* **Config:** - **extra**: *str = allow* * **Fields:** - [`bleu (float | None)`](#promptquality.types.rows.Metrics.bleu) - [`context_adherence (float | None)`](#promptquality.types.rows.Metrics.context_adherence) - [`context_adherence_explanation (str | None)`](#promptquality.types.rows.Metrics.context_adherence_explanation) - [`context_relevance (float | None)`](#promptquality.types.rows.Metrics.context_relevance) - [`correctness (float | None)`](#promptquality.types.rows.Metrics.correctness) - [`correctness_explanation (str | None)`](#promptquality.types.rows.Metrics.correctness_explanation) - [`latency (float | None)`](#promptquality.types.rows.Metrics.latency) - [`like_dislike (bool | None)`](#promptquality.types.rows.Metrics.like_dislike) - [`pii (List[str] | None)`](#promptquality.types.rows.Metrics.pii) - [`rouge (float | None)`](#promptquality.types.rows.Metrics.rouge) - [`toxicity (float | None)`](#promptquality.types.rows.Metrics.toxicity) - [`uncertainty (float | None)`](#promptquality.types.rows.Metrics.uncertainty)

field bleu*: Optional[float]* = None

field context_adherence*: Optional[float]* = None

field context_adherence_explanation*: Optional[str]* = None

field context_relevance*: Optional[float]* = None

field correctness*: Optional[float]* = None

field correctness_explanation*: Optional[str]* = None

field latency*: Optional[float]* = None

field like_dislike*: Optional[bool]* = None

field pii*: Optional[List[str]]* [Optional]

field rouge*: Optional[float]* = None

field toxicity*: Optional[float]* = None

field uncertainty*: Optional[float]* = None

pydantic model PromptRow

Bases: BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "PromptRow", "type": "object", "properties": { "index": { "title": "Index", "type": "integer" }, "prompt": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Prompt" }, "response": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Response" }, "target": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Target" }, "inputs": { "additionalProperties": { "anyOf": [ {}, { "type": "null" } ] }, "title": "Inputs", "type": "object" }, "hallucination": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Hallucination" }, "bleu": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Bleu" }, "rouge": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Rouge" }, "cost": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Cost" }, "like_dislike": { "anyOf": [ { "type": "boolean" }, { "type": "null" } ], "default": null, "title": "Like Dislike" }, "metrics": { "$ref": "#/$defs/Metrics" } }, "$defs": { "Metrics": { "additionalProperties": true, "properties": { "hallucination": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Hallucination" }, "bleu": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Bleu" }, "rouge": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Rouge" }, "pii": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "title": "Pii" }, "like_dislike": { "anyOf": [ { "type": "boolean" }, { "type": "null" } ], "default": null, "title": "Like Dislike" }, "toxicity": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Toxicity" }, "factuality": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Factuality" }, "factuality_explanation": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Factuality Explanation" }, "groundedness": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Groundedness" }, "groundedness_explanation": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Groundedness Explanation" }, "latency": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Latency" }, "context_relevance": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Context Relevance" } }, "title": "Metrics", "type": "object" } }, "additionalProperties": true, "required": [ "index" ] } ```
* **Config:** - **extra**: *str = allow* * **Fields:** - [`bleu (float | None)`](#promptquality.types.rows.PromptRow.bleu) - [`cost (float | None)`](#promptquality.types.rows.PromptRow.cost) - [`hallucination (float | None)`](#promptquality.types.rows.PromptRow.hallucination) - [`index (int)`](#promptquality.types.rows.PromptRow.index) - [`inputs (Dict[str, Any | None])`](#promptquality.types.rows.PromptRow.inputs) - [`like_dislike (bool | None)`](#promptquality.types.rows.PromptRow.like_dislike) - [`metrics (promptquality.types.rows.Metrics)`](#promptquality.types.rows.PromptRow.metrics) - [`prompt (str | None)`](#promptquality.types.rows.PromptRow.prompt) - [`response (str | None)`](#promptquality.types.rows.PromptRow.response) - [`rouge (float | None)`](#promptquality.types.rows.PromptRow.rouge) - [`target (str | None)`](#promptquality.types.rows.PromptRow.target)

field bleu*: Optional[float]* = None

field cost*: Optional[float]* = None

field hallucination*: Optional[float]* = None

field index*: int* [Required]

field inputs*: Dict[str, Optional[Any]]* [Optional]

field like_dislike*: Optional[bool]* = None

field metrics*: Metrics* [Optional]

field prompt*: Optional[str]* = None

field response*: Optional[str]* = None

field rouge*: Optional[float]* = None

field target*: Optional[str]* = None

pydantic model PromptRows

Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "PromptRows", "type": "object", "properties": { "starting_token": { "default": 0, "title": "Starting Token", "type": "integer" }, "limit": { "default": 25, "title": "Limit", "type": "integer" }, "paginated": { "default": false, "title": "Paginated", "type": "boolean" }, "next_starting_token": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Next Starting Token" }, "rows": { "items": { "$ref": "#/$defs/PromptRow" }, "title": "Rows", "type": "array" } }, "$defs": { "Metrics": { "additionalProperties": true, "properties": { "hallucination": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Hallucination" }, "bleu": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Bleu" }, "rouge": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Rouge" }, "pii": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "title": "Pii" }, "like_dislike": { "anyOf": [ { "type": "boolean" }, { "type": "null" } ], "default": null, "title": "Like Dislike" }, "toxicity": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Toxicity" }, "factuality": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Factuality" }, "factuality_explanation": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Factuality Explanation" }, "groundedness": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Groundedness" }, "groundedness_explanation": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Groundedness Explanation" }, "latency": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Latency" }, "context_relevance": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Context Relevance" } }, "title": "Metrics", "type": "object" }, "PromptRow": { "additionalProperties": true, "properties": { "index": { "title": "Index", "type": "integer" }, "prompt": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Prompt" }, "response": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Response" }, "target": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Target" }, "inputs": { "additionalProperties": { "anyOf": [ {}, { "type": "null" } ] }, "title": "Inputs", "type": "object" }, "hallucination": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Hallucination" }, "bleu": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Bleu" }, "rouge": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Rouge" }, "cost": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "title": "Cost" }, "like_dislike": { "anyOf": [ { "type": "boolean" }, { "type": "null" } ], "default": null, "title": "Like Dislike" }, "metrics": { "$ref": "#/$defs/Metrics" } }, "required": [ "index" ], "title": "PromptRow", "type": "object" } } } ```
* **Fields:** - [`rows (List[promptquality.types.rows.PromptRow])`](#promptquality.types.rows.PromptRows.rows)

field rows*: List[PromptRow]* [Optional]

promptquality.types.run module

pydantic model RandomName

Bases: BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "RandomName", "type": "object", "properties": { "name": { "title": "Name", "type": "string" } } } ```
* **Fields:** - [`name (str)`](#promptquality.types.run.RandomName.name) * **Validators:** - [`set_name`](#promptquality.types.run.RandomName.set_name) » [`name`](#promptquality.types.run.RandomName.name)

field name*: str* [Optional]

validator set_name » name​

  • Return type: str

pydantic model CreateProjectRequest

Bases: RandomName​
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "CreateProjectRequest", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "type": { "default": "prompt_evaluation", "title": "Type", "type": "string" } } } ```
* **Fields:** - [`type (str)`](#promptquality.types.run.CreateProjectRequest.type) * **Validators:** - [`set_name`](#promptquality.types.run.RandomName.set_name) » `name`

field type*: str* = 'prompt_evaluation'

pydantic model CreateProjectResponse

Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "CreateProjectResponse", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "type": { "default": "prompt_evaluation", "title": "Type", "type": "string" }, "id": { "format": "uuid4", "title": "Id", "type": "string" } }, "required": [ "id" ] } ```
* **Fields:** - [`id (uuid.UUID)`](#promptquality.types.run.CreateProjectResponse.id) * **Validators:** - [`set_name`](#promptquality.types.run.RandomName.set_name) » `name`

field id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4

pydantic model BaseTemplateVersionRequest

Bases: BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "BaseTemplateVersionRequest", "type": "object", "properties": { "template": { "title": "Template", "type": "string" }, "version": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Version" } }, "required": [ "template" ] } ```
* **Fields:** - [`template (str)`](#promptquality.types.run.BaseTemplateVersionRequest.template) - [`version (int | None)`](#promptquality.types.run.BaseTemplateVersionRequest.version)

field template*: str* [Required]

field version*: Optional[int]* = None

pydantic model CreateTemplateRequest

Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "CreateTemplateRequest", "type": "object", "properties": { "template": { "title": "Template", "type": "string" }, "version": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Version" }, "name": { "title": "Name", "type": "string" }, "project_id": { "format": "uuid4", "title": "Project Id", "type": "string" } }, "required": [ "template", "project_id" ] } ```
* **Fields:** - [`project_id (uuid.UUID)`](#promptquality.types.run.CreateTemplateRequest.project_id) * **Validators:** - [`set_name`](#promptquality.types.run.RandomName.set_name) » `name`

field project_id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4

pydantic model CreateTemplateVersionRequest

Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "CreateTemplateVersionRequest", "type": "object", "properties": { "template": { "title": "Template", "type": "string" }, "version": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Version" }, "template_id": { "format": "uuid4", "title": "Template Id", "type": "string" }, "project_id": { "format": "uuid4", "title": "Project Id", "type": "string" } }, "required": [ "template", "template_id", "project_id" ] } ```
* **Fields:** - [`project_id (uuid.UUID)`](#promptquality.types.run.CreateTemplateVersionRequest.project_id) - [`template_id (uuid.UUID)`](#promptquality.types.run.CreateTemplateVersionRequest.template_id)

field project_id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4

field template_id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4

pydantic model BaseTemplateVersionResponse

Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "BaseTemplateVersionResponse", "type": "object", "properties": { "template": { "title": "Template", "type": "string" }, "version": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Version" }, "id": { "format": "uuid4", "title": "Id", "type": "string" } }, "required": [ "template", "id" ] } ```
* **Fields:** - [`id (uuid.UUID)`](#promptquality.types.run.BaseTemplateVersionResponse.id)

field id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4

pydantic model CreateTemplateVersionResponse

Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "CreateTemplateVersionResponse", "type": "object", "properties": { "template": { "title": "Template", "type": "string" }, "version": { "title": "Version", "type": "integer" }, "id": { "format": "uuid4", "title": "Id", "type": "string" } }, "required": [ "template", "version", "id" ] } ```
* **Fields:** - [`version (int)`](#promptquality.types.run.CreateTemplateVersionResponse.version)

field version*: int* [Required]

pydantic model BaseTemplateResponse

Bases: RandomName​
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "BaseTemplateResponse", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "id": { "format": "uuid4", "title": "Id", "type": "string" }, "template": { "title": "Template", "type": "string" }, "selected_version": { "$ref": "#/$defs/CreateTemplateVersionResponse" }, "selected_version_id": { "format": "uuid4", "title": "Selected Version Id", "type": "string" }, "all_versions": { "items": { "$ref": "#/$defs/CreateTemplateVersionResponse" }, "title": "All Versions", "type": "array" } }, "$defs": { "CreateTemplateVersionResponse": { "properties": { "template": { "title": "Template", "type": "string" }, "version": { "title": "Version", "type": "integer" }, "id": { "format": "uuid4", "title": "Id", "type": "string" } }, "required": [ "template", "version", "id" ], "title": "CreateTemplateVersionResponse", "type": "object" } }, "required": [ "id", "template", "selected_version", "selected_version_id" ] } ```
* **Fields:** - [`all_versions (List[promptquality.types.run.CreateTemplateVersionResponse])`](#promptquality.types.run.BaseTemplateResponse.all_versions) - [`id (uuid.UUID)`](#promptquality.types.run.BaseTemplateResponse.id) - [`selected_version (promptquality.types.run.CreateTemplateVersionResponse)`](#promptquality.types.run.BaseTemplateResponse.selected_version) - [`selected_version_id (uuid.UUID)`](#promptquality.types.run.BaseTemplateResponse.selected_version_id) - [`template (str)`](#promptquality.types.run.BaseTemplateResponse.template) * **Validators:** - [`set_name`](#promptquality.types.run.RandomName.set_name) » `name`

field all_versions*: List[CreateTemplateVersionResponse]* [Optional]

field id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4

field selected_version*: CreateTemplateVersionResponse* [Required]

field selected_version_id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4

field template*: str* [Required]

pydantic model BaseDatasetRequest

Bases: BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "BaseDatasetRequest", "type": "object", "properties": { "format": { "allOf": [ { "$ref": "#/$defs/DatasetFormat" } ], "default": "csv" }, "file_path": { "format": "path", "title": "File Path", "type": "string" } }, "$defs": { "DatasetFormat": { "enum": [ "csv", "feather" ], "title": "DatasetFormat", "type": "string" } }, "required": [ "file_path" ] } ```
* **Fields:** - [`file_path (pathlib.Path)`](#promptquality.types.run.BaseDatasetRequest.file_path) - [`format (promptquality.constants.dataset_format.DatasetFormat)`](#promptquality.types.run.BaseDatasetRequest.format) * **Validators:** - [`dataset_to_path`](#promptquality.types.run.BaseDatasetRequest.dataset_to_path) » `all fields`

field file_path*: Path* [Required]

field format*: DatasetFormat* = DatasetFormat.csv

validator dataset_to_path » all fields

  • Return type: Dict

property files*: Dict[str, BufferedReader]*

pydantic model UploadDatasetRequest

Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.
__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.
Show JSON schema
```json { "title": "UploadDatasetRequest", "type": "object", "properties": { "format": { "allOf": [ { "$ref": "#/$defs/DatasetFormat" } ], "default": "csv" }, "file_path": { "format": "path", "title": "File Path", "type": "string" }, "project_id": { "format": "uuid4", "title": "Project Id", "type": "string" }, "prompt_template_version_id": { "format": "uuid4", "title": "Prompt Template Version Id", "type": "string" } }, "$defs": { "DatasetFormat": { "enum": [ "csv", "feather" ], "title": "DatasetFormat", "type": "string" } }, "required": [ "file_path", "project_id", "prompt_template_version_id" ] } ```
* **Fields:** - [`file_path (pathlib.Path)`](#promptquality.types.run.UploadDatasetRequest.file_path) - [`project_id (uuid.UUID)`](#promptquality.types.run.UploadDatasetRequest.project_id) - [`prompt_template_version_id (uuid.UUID)`](#promptquality.types.run.UploadDatasetRequest.prompt_template_version_id) * **Validators:** - [`dataset_to_path`](#promptquality.types.run.BaseDatasetRequest.dataset_to_path) » `all fields`

field file_path*: Path* [Required]

field project_id*: Annotated[UUID]* [Required]

field prompt_template_version_id*: Annotated[UUID]* [Required]

  • Constraints:
    • uuid_version = 4
  • Validated by: