ibm_aigov_facts_client.factsheet.asset_utils_runs module

class NotebookExperimentRunsUtilities(assets_client: Assets, cur_exp_runs=None, run_id: str = None, run_idx: str = None, run_info=None, facts_type: str = 'notebook_experiment')

Bases: object

Model notebook experiment runs utilities. Running client.assets.model() makes all methods in NotebookExperimentUtilities object available to use.

to_dict() → Dict: Return a json dictionary representing this model.

debug_mode_print()

refresh_state_before_execution()

get_info()

Get run info. Supported for CPD version >=4.6.4

A way to use me is:

>>> run=exp.get_run() # returns latest if run_id is not given
>>> run.get_info()

set_custom_date(date: float = None) → None

set_custom_metric(metric_id: str, value: int | float) → None

Set model training metric against run_id

Parameters:

metric_id (str) – Metric key name
value (float) – Metric value

Raises:

ClientError – Raises client error for exceptions

Returns:

None

A way to use me is:

>>> run.set_custom_metric(metric_key=<key>,value=<value>)

set_custom_param(param_id: str, value: str) → None

Set model training param against the run_id

Parameters:

param_id (str) – Param key name
value (str) – Param value

Raises:

ClientError – Raises client error for exceptions

Returns:

None

A way to use me is:

>>> run.set_custom_param(param_id=<key>,value=<value>)

set_custom_tag(tag_id: str, value: str) → None

Set model training tag against the run_id

Parameters:

tag_id (str) – Tag key name
value (str) – Tag value

Raises:

ClientError – Raises client error for exceptions

Returns:

None

A way to use me is:

>>> run.set_custom_tag(tag_id=<key>,value=<value>)

get_custom_metric(metric_id: str) → List

Get custom metric value by id against the run_id

Parameters:: metric_id (str) – Custom metric id to retrieve.
Return type:: list

A way you might use me is:

>>> run.get_custom_metric_by_id(metric_id="<metric_id>")

get_custom_metrics(metric_ids: List[str] = None) → List

Get all logged custom metrics against the run_id

Parameters:: metrics_ids (list) – (Optional) Metrics ids to get. If not provided, returns all metrics available for the latest run
Return type:: list[dict]

A way you might use me is:

>>> run.get_custom_metrics() # uses last logged run
>>> run.get_custom_metrics(metric_ids=["id1","id2"]) # uses last logged run

get_custom_param(param_id: str) → Dict

Get custom param value by id against the run_id

Parameters:: param_id (str) – Custom param id to retrieve.
Return type:: list

A way you might use me is:

>>> run.get_custom_param(param_id="<param_id>")

get_custom_params(param_ids: List[str] = None) → List

Get all logged params against the run_id

Parameters:: param_ids (list) – (Optional) Params ids to get. If not provided, returns all params available for the latest run
Return type:: list[dict]

A way you might use me is:

>>> run.get_custom_params() # uses last logged run
>>> run.get_custom_params(param_ids=["id1","id2"]) # uses last logged run

get_custom_tag(tag_id: str) → Dict

Get custom tag value by id against the run_id

Parameters:: tag_id (str) – Custom tag id to retrieve.
Return type:: dict

A way you might use me is:

>>> run.get_custom_tag(tag_id="<tag_id>")

get_custom_tags(tag_ids: List[str] = None) → List

Get all logged tags against the run_id

Parameters:: tag_ids (list) – (Optional) Tags ids to get. If not provided, returns all tags available for the latest run
Return type:: list[dict]

A way you might use me is:

>>> run.get_custom_tags() # uses last logged run
>>> run.get_custom_tags(tag_ids=["id1","id2"]) # uses last logged run

remove_custom_metric(metric_id: str) → None

Remove metric by id against the run_id

Parameters:: metric_id (str) – Metric id to remove.

A way you might use me is:

>>> run.remove_custom_metric(metric_id=<metric_id>)

remove_custom_metrics(metric_ids: List[str]) → None

Remove multiple metrics.

Parameters:: metric_ids (list of str) – List of metric ids to remove from run.

A way you might use me is:

>>> model.remove_custom_metrics(metric_ids=["id1", "id2"])

remove_custom_param(param_id: str) → None

Remove param by id against the run_id

Parameters:: param_id (str) – Param id to remove.

A way you might use me is:

>>> run.remove_custom_param(param_id=<param_id>)

remove_custom_tag(tag_id: str) → None

Remove tag by id against the run_id

Parameters:: tag_id (str) – Tag id to remove.

A way you might use me is:

>>> run.remove_custom_tag(tag_id=<tag_id>)

set_custom_run_facts(metrics: Dict[str, Any] = None, params: Dict[str, Any] = None, tags: Dict[str, Any] = None, overwrite: bool = False, debug: bool = False)

Set custom run facts for an experiment.

Parameters:

metrics (dict, optional) – A dictionary containing metrics.
params (dict, optional) – A dictionary containing parameters.
tags (dict, optional) – A dictionary containing tags.
overwrite (bool, optional) – Whether to overwrite existing facts for a selected run. Default is False.
debug (bool, optional) – Whether to display additional details for debugging purposes. Default is False.

A way to use me is:

>>> metrics = {"training_score": 0.955, "training_mse": 0.911}
    params = {"batch_size": 32, "learning_rate": 0.001}
    tags = {"experiment_type": "CNN", "dataset": "MNIST"}
>>> run.set_custom_run_facts(metrics_dict=metrics, params_dict=params, tags_dict=tags, overwrite=False, debug=True)

set_custom_metrics(metrics_dict: Dict[str, Any], debug: bool = False) → None

Set model training metrics against the run_id.

Parameters:

metrics_dict (dict) – Metric key, value pairs.
debug (bool, optional) – Whether to display additional details for debugging purposes. Default is False.

Raises:

ClientError – Raises client error for exceptions.

Returns:

None

A way to use me is:

>>> run.set_custom_metrics(metrics_dict={"training_score": 0.955, "training_mse": 0.911})

set_custom_tags(tags_dict: Dict[str, Any], debug: bool = False) → None

Set model training tags against the run_id.

Parameters:

tags_dict (dict) – Tags key, value pairs.
debug (bool, optional) – Whether to display additional details for debugging purposes. Default is False.

Raises:

ClientError – Raises client error for exceptions.

Returns:

None

A way to use me is:

>>> run.set_custom_tags(tags_dict={"tag1": <value>, "tag2": <value>})

set_custom_params(params_dict: Dict[str, Any], debug: bool = False) → None

Set model training params against the run_id.

Parameters:

params_dict (dict) – Params key, value pairs.
debug (bool, optional) – Whether to display additional details for debugging purposes. Default is False.

Raises:

ClientError – Raises client error for exceptions.

Returns:

None

A way to use me is:

>>> run.set_custom_params(params_dict={"num_class": 3, "early_stopping_rounds": 10})

remove_custom_tags(tag_ids: List[str], debug: bool = False) → None

Remove multiple tags against the run_id.

Parameters:

tag_ids (list of str) – List of tag ids to remove from run.
debug (bool, optional) – Whether to display additional details for debugging purposes. Default is False.

A way you might use me is:

>>> run.remove_custom_tags(tag_ids=["id1", "id2"])

remove_custom_params(param_ids: List[str], debug: bool = False) → None

Remove multiple params against the run_id.

Parameters:

param_ids (list of str) – List of param ids to remove from run.
debug (bool, optional) – Whether to display additional details for debugging purposes. Default is False.

A way you might use me is:

>>> run.remove_custom_params(param_ids=["id1", "id2"])

remove_custom_run_facts(metric_ids: List[str] = None, param_ids: List[str] = None, tag_ids: List[str] = None, debug: bool = False) → None

Remove multiple custom run facts (metrics,params or tags) from the run.

Parameters:

tag_ids – List of tag IDs to remove.
param_ids – List of parameter IDs to remove.
metric_ids – List of metric IDs to remove.
optional) (debug (bool,) – Used to display additional details for debugging purpose. Default is False.

At least one of tag_ids, param_ids, or metric_ids must be provided.

Example usage:

>>> run.remove_custom_run_facts(metric_ids=["metric1"], param_ids=["param1"],tag_ids=["tag1", "tag2"])