Trace and Customize Training Runs

Capture additional metrics, parameters, and tags against run_id under a particular experiment before saving the model.

Note

  • Following utlities are subject to local storage which facts client uses as an intermediary step. In case when using Watson Studio notebook environment, reloading notebooks sessions might cause exceptions to local storage thus these utilities will not function as expected but it does not impact results saved to factsheet and users would be able to see results through factsheet UI.

Experiments

class Experiments(root_directory=None)

Bases: object

Utility to explore current experiments.

list_experiments(max_results: int = 100) → DataFrame

List all active experiments.

Returns:

DataFrame object.

Return type:

Pandas.DataFrame

A way you might use me is:

>>> client.experiments.list_experiments()
get_current_experiment_id()

Shows current experiment id.

Returns:

str

A way you might use me is:

>>> client.experiments.get_current_experiment_id()

Runs

class Runs(root_directory=None)

Bases: object

Utilities to explore runs within any experiment.

list_runs_by_experiment(experiment_id: str, order_by: List[str] | None = None) → DataFrame

List all runs under any experiment

Parameters:

  • experiment_id (str) – ID of the experiment.

  • order_by – List of order_by clauses. Currently supported values are metric.key, parameter.key, tag.key.For example, order_by=["tag.release ASC", "metric.training_score DESC"]

Returns:

DataFrame object that satisfy the search expressions.

Return type:

Pandas.DataFrame

A way you might use me is:

>>> client.runs.list_runs_by_experiment("1")

>>> client.runs.list_runs_by_experiment("1", order_by=["metric.training_score DESC"]))
get_current_run_id()

Shows current active run id.

Returns:

str

A way you might use me is:

>>> client.runs.get_current_run_id()
log_metric(run_id: str, key: str, value: float, step: int | None = None) → None

Log a metric against the run ID.

Parameters:

  • run_id (str) – The unique id for run.

  • key (str) – Metric name.

  • value (float) – Metric value (float).

  • step (int) – Integer training step (iteration) at which was the metric calculated. Defaults to 0.

Returns:

None

A way you might use me is:

>>> client.runs.log_metric(run_id, "mae", .77)
log_metrics(run_id: str, metrics: Dict[str, float], step: int | None = None) → None

Log multiple metrics for the given run.

Parameters:

  • run_id (str) – The unique id for run.

  • metrics (dict) – Dictionary of metric_name: String -> value: Float.

  • step (int) – Integer training step (iteration) at which was the metric calculated. Defaults to 0.

Returns:

None

A way you might use me is:

>>> client.runs.log_metrics(run_id, {"mse": 2000.00, "rmse": 50.00})
log_param(run_id: str, key: str, value: Any) → None

Log a param against the run ID.

Parameters:

  • run_id (str) – The unique id for run.

  • key (str) – Param name.

  • value – Param value.Value is converted to a string.

Returns:

None

A way you might use me is:

>>> client.runs.log_param(run_id, "c", 1)
log_params(run_id, params: Dict[str, Any]) → None

Log multiple params for the given run.

Parameters:

  • run_id (str) – The unique id for run.

  • params (dict) – Dictionary of String -> value: (String, but will be string-ified if not)

Returns:

None

A way you might use me is:

>>> client.runs.log_params(run_id, {"n_estimators": 3, "random_state": 42})
set_tags(run_id: str, tags: Dict[str, Any]) → None

Log multiple tags for the given run.

Parameters:

  • run_id (str) – The unique id for run.

  • tags (dict) – Dictionary of tags names: String -> value: (String, but will be string-ified if not)

Returns:

None

A way you might use me is:

>>> client.runs.set_tags(run_id, {"engineering": "ML Platform",
"release.candidate": "RC1"})