Keras
Log a human-readable version of your dataset. Galileo will join these samples with the model's outputs and present them in the Console.
Keras
import dataquality as dq
dq.init(task_type="text_classification", # Change this based on your task type
project_name="example_keras_project",
run_name="example_keras_run")
# 🔭🌕 Log the class labels in the order they are outputted by the model
labels_list = ["positive review", "negative review", "very positive review", "very negative review"]
dq.set_labels_for_run(labels_list)
# 🔭🌕 Log your pandas/huggingface/tf datasets to Galileo
dq.log_dataset(train_dataset, split="train")
dq.log_dataset(test_dataset, split="test")
Add our logging layers to your Keras model's definition. This works with the functional or sequential syntax for defining models in Keras.
Keras
from dataquality.integrations.keras import DataQualityLoggingLayer
model = keras.Sequential(
[
DataQualityLoggingLayer("ids"), # 🌕🔭
...
DataQualityLoggingLayer("embs"), # 🌕🔭
...
DataQualityLoggingLayer("probs"), # 🌕🔭
]
)
model.summary()
Make sure to compile your model to run eagerly if it's not the default; add ids to your model's inputs; and add the Galileo callback to auto-log the epochs and splits.
Keras
from dataquality.integrations.keras import add_ids_to_numpy_arr, DataQualityCallback
x_train = add_ids_to_numpy_arr(x_train, train_ids) # 🌕🔭 ids from dataset logging
model.compile(..., run_eagerly=True)
model.fit(x_train, y_train, ...,
callbacks=[ DataQualityCallback() ]) # 🌕🔭
dq.finish() # 🔭🌕 This will wait until the run is processed by Galileo
Last modified 8mo ago