matrice.model_store module#

class matrice.model_store.BYOM(session, family_name=None, family_id=None, family_config=None)[source]#

Bases: object

A class to interact with the BYOM (Bring Your Own Model) API for managing model families, model information, and model action configurations.

Attributes:#

sessionSession: A session object containing account information and RPC (Remote Procedure Call) details.
account_numberstr: The account number associated with the session.
rpcRPC: The RPC object used to make API calls.

Methods:#

delete_model_family(model_family_id): Deletes a model family using its ID.
delete_model_info(model_info_id): Deletes model information using its ID.
delete_model_action_config(model_action_config_id): Deletes a model action configuration using its ID.
add_model_family(…): Adds a new model family.
add_model_info(…): Adds new model information.
add_model_action_config(…): Adds a new model action configuration.
update_model_family(…): Updates a model family.
update_model_info(…): Updates model information.
update_model_action_config(…): Updates a model action configuration.
add_model_family_action_config(…): Adds an action configuration to a model family.

__init__(session, family_name=None, family_id=None, family_config=None)[source]#

Initializes the BYOM class with a session object.

Parameters:#

sessionSession: A session object containing account information and RPC details.

add_action_config(action_type, action_config, model_checkpoint, export_format)[source]#

Adds a new action configuration for a specific model in the model store.

This function sends a POST request to add a new action configuration for a model with the provided parameters.

Parameters:#

action_typestr: The type of action (e.g., ‘train_model’, ‘export_model’).
action_configdict: Configuration details for the action.
model_checkpointstr: Path or identifier for the model checkpoint.
export_formatstr: Format for exporting the model.

Returns:#

: tuple

A tuple containing three elements: - API response (dict): The raw response from the API. - error_message (str or None): Error message if an error occurred, None otherwise. - status_message (str): A status message indicating success or failure.

Raises:#

May raise exceptions related to network issues or API errors.

Notes:#

This function uses the self.rpc.post method to send the request and handle_response to process the response.

add_codebase(code_path, project_type, matrice_version, training_framework, framework_version, license_content)[source]#

Add model family code for a given project.

Parameters:

cloud_path (str) – The cloud path where the model family code is stored.
project_type (str) – The type of the project (e.g., “classification”, “detection”, “instance_segmentation”).
python_sdk_version (str) – The version of the Python SDK.
pytorch_version (str) – The version of PyTorch.
license_info (str) – The license information for the model family code.

Returns:

A tuple containing the response data, error (if any), and message.

Return type:

tuple

Example

>>> resp, error, message = model_store.add_model_family_code(
>>>     "cloud/path/to/code", "classification", "1.0.0", "1.7.1", "MIT"
>>> )
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Model family code added: {resp}")

add_or_update_model_info(config)[source]#

Adds or updates information for a specific model in the model store.

This function sends a POST request to add or a PUT request to update information about a model with the provided parameters.

Parameters:#

configstr or dict: The path to the local JSON file containing the model config or the model config dictionary.

Returns:#

: tuple

A tuple containing three elements: - API response (dict): The raw response from the API. - error_message (str or None): Error message if an error occurred, None otherwise. - status_message (str): A status message indicating success or failure.

Raises:#

ValueError: If the config is neither a valid file path nor a dictionary.

delete_action_config(model_action_config_id, export_format=None)[source]#

Deletes a model action configuration using its ID.

Parameters:#

model_action_config_idstr: The ID of the model action configuration to delete.

Returns:#

: tuple

A tuple containing the API response, error message (or None if successful), and a status message.

delete_model_family()[source]#

Deletes a model family using its ID.

Parameters:#

model_family_idstr: The ID of the model family to delete.

Returns:#

: tuple

A tuple containing the API response, error message (or None if successful), and a status message.

delete_model_info(model_info_id, model_key)[source]#

Deletes model information using its ID.

Parameters:#

model_info_idstr: The ID of the model information to delete.

Returns:#

: tuple

A tuple containing the API response, error message (or None if successful), and a status message.

download_codebase(local_path)[source]#

Fetch the download path for the user code base of a model family.

Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> resp, error, message = model_store.get_user_code_base_download_path()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Model code path: {resp}")

get_byom_status()[source]#

get_codebase_details()[source]#

Fetch user code base details for the specified model family.

Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> resp, error, message = model_store.get_user_code_base_details()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"User code details: {resp}")

get_test_case_logs()[source]#

get_validation_logs()[source]#

Get the validation logs for a given model family.

Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> resp, error, message = model_store.get_validation_logs()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Validation logs: {resp}")

update_action_config(model_action_config_id, model_info_id, action_type, action_config, export_format, model_checkpoint)[source]#

Updates the action configuration for a specific model in the model store.

This function sends a PUT request to update model action configuration with the provided parameters.

Parameters:

model_action_config_id (str) – The unique identifier of the model action config to update.
model_info_id (str) – The identifier of the model info this action config belongs to.
action_type (str) – The updated type of action (e.g., ‘train_model’, ‘export_model’).
action_config (dict) – Updated configuration details for the action.
export_format (str) – Updated format for exporting the model.

Returns:

A tuple containing three elements: - API response (dict): The raw response from the API. - error_message (str or None): Error message if an error occurred, None otherwise. - status_message (str): A status message indicating success or failure.

Return type:

tuple

Raises:

May raise exceptions related to network issues or API errors. –

Notes

This function uses the self.rpc.put method to send the request and handle_response to process the response.

Example

>>> resp, error, message = model_store.update_action_config(
>>>     model_action_config_id="12345",
>>>     model_info_id="67890",
>>>     action_type="train_model",
>>>     action_config={"param1": "value1", "param2": "value2"},
>>>     export_format="ONNX"
>>> )
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Action config updated: {resp}")

update_model_family(config)[source]#

Updates an existing model family in the model store.

This function sends a PUT request to update a model family with the provided parameters.

Parameters:#

model_family_idstr: The unique identifier of the model family to update.
configstr or dict: The path to the local JSON file containing the model config or the model config dictionary.

Returns:#

: tuple

A tuple containing three elements: - API response (dict): The raw response from the API. - error_message (str or None): Error message if an error occurred, None otherwise. - status_message (str): A status message indicating success or failure.

Raises:#

ValueError: If the config is neither a valid file path nor a dictionary.

validate_all()[source]#

Validate all configurations and logs for the model family.

Returns:: A dictionary containing the results of all validations.
Return type:: dict

Example

>>> results = model_store.validate_all()
>>> for key, (resp, error, message) in results.items():
>>>     if error:
>>>         print(f"{key} validation failed: {error}")
>>>     else:
>>>         print(f"{key} validation succeeded: {message}")

validate_export_config()[source]#

Validate the export configuration for a given project.

Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> resp, error, message = model_store.validate_export_config()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Validation result: {resp}")

validate_local_test_logs(file_path)[source]#

Validate the local test logs for a given project.

Parameters:: file_path (str) – The path to the test log file.
Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> resp, error, message = model_store.validate_test_logs("path/to/log/file")
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Validation result: {resp}")

validate_model_info()[source]#

Validate the model information for a given project.

Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> resp, error, message = model_store.validate_model_info()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Validation result: {resp}")

validate_train_config()[source]#

Validate the training configuration for a given project.

Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> resp, error, message = model_store.validate_train_config()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Validation result: {resp}")

class matrice.model_store.ModelArch(session, model_family_name, model_key)[source]#

Bases: object

A class to interact with model architectures through the model architecture API.

This class handles fetching and storing model architecture information, including configuration parameters, export formats, and other model metadata.

Parameters:

session (Session) – Active session object for making API calls
model_family_name (str) – Name of the model family this architecture belongs to
model_key (str) – Unique identifier key for the model architecture

account_number#

Account number from the session

Type:: str

project_id#

Project identifier from the session

Type:: str

model_family_name#

Name of the model family

Type:: str

model_key#

Model’s unique identifier key

Type:: str

last_refresh_time#

Timestamp of last data refresh

Type:: datetime

rpc#

RPC client object from session for API calls

Type:: RPCClient

model_info_id#

Model information unique identifier

Type:: str or None

model_name#

Human readable name of the model

Type:: str or None

model_family_id#

Unique identifier of the model family

Type:: str or None

params_millions#

Number of parameters in millions

Type:: float or None

export_formats#

List of supported export formats

Type:: list or None

model_config#

Default configuration parameters for model training

Type:: dict or None

Notes

Upon initialization, the class automatically fetches: - Model information using _get_model_info() - Training configuration using get_model_train_config() - Export formats using get_export_formats()

If model_key is not provided, these fetches are skipped and the class initializes with minimal information.

Example

>>> session = Session()
>>> model = ModelArch(
...     session=session,
...     model_family_name="resnet",
...     model_key="resnet50"
... )
>>> print(f"Model: {model.model_name}")
>>> print(f"Parameters: {model.params_millions}M")
>>> print(f"Export formats: {model.export_formats}")

Raises:: AssertionError – If model_key or model_family_name is None

__init__(session, model_family_name, model_key)[source]#

get_default_export_config(export_format)[source]#

Retrieves the default configuration for exporting a model in a specified format.

This method fetches the export configuration for the given model_info_id and export format, returning a dictionary of default export settings.

Parameters:

model_info_id (str) – The unique identifier of the model whose export configuration is to be retrieved.
export_format (str) – The format in which the model is to be exported (e.g., ‘ONNX’, ‘TF SavedModel’).

Returns:

A dictionary containing default export configuration settings, where keys are parameter names and values are default values.

Return type:

dict

Example

>>> export_format = "ONNX"
>>> default_export_config = model_arch.get_default_model_export_config(export_format)
>>> print(default_export_config)
{
    optimize: True,
    int8: False,
    ...
}

get_export_action_configs(export_format)[source]#

Fetch action configuration for model export.

Parameters:

model_info_id (str) – The ID of the model info.
export_format (str) – The export format.

Returns:

A tuple containing the response data, error (if any), and message.

Return type:

tuple

Example

>>> resp, error, message = model_arch.get_action_config_for_model_export("ONNX")
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Action config for model export: {resp}")

get_export_formats()[source]#

Fetch export formats for a given model.

If model_info_id is provided, it fetches the export formats using the ID. Otherwise, it fetches the export formats using the model key and family name.

Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> session = Session(account="your_account_number", access_key="your_access_key", secret_key="your_secret_key")
>>> model_arch = ModelArch(session, model_family_name="resnet", model_key="resnet50")
>>> resp, error, message = model_arch.get_export_formats()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Export formats: {resp}")

get_model_action_configs()[source]#

Fetch model action configuration by its ID.

Parameters:: model_action_config_id (str) – The ID of the model action config to fetch.
Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> resp, error, message = model_arch.get_model_action_config()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Model action config: {resp}")

get_model_train_config()[source]#

Fetch model training configuration by its ID.

Parameters:: model_info_id (str) – The ID of the model info.
Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> resp, error, message = model_arch.get_model_train_config()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Model train config: {resp}")

get_train_config(experiment_id, tuning_type='default', model_checkpoint='auto')[source]#

Fetch the training configuration for a given model.

This function retrieves the training configuration for a specified model and experiment. It constructs the payload with the provided parameters and sends a POST request to fetch the model parameters. The response is then processed to extract the action configuration and model configuration, which are used to construct the final training configuration.

Parameters:

experiment_id (str) – The ID of the experiment for which the training configuration is to be fetched.
tuning_type (str, optional) – The type of parameter search to be used for tuning (default is “default”).
model_checkpoint (str, optional) – The model checkpoint to be used (default is “auto”).

Returns:

A tuple containing the ModelArch instance and the training configuration dictionary.

Return type:

tuple

Example

>>> session = Session(account="your_account_number", access_key="your_access_key", secret_key="your_secret_key")
>>> model_arch = ModelArch(session, model_family_name="resnet", model_key="resnet50")
>>> experiment_id = "your_experiment_id"
>>> model_arch_instance, train_config = model_arch.get_train_config(experiment_id)
>>> print("ModelArch Instance:", model_arch_instance)
>>> print("Training Configuration:", train_config)

refresh()[source]#: Refresh the instance by reinstantiating it with the previous values.

class matrice.model_store.ModelFamily(session, model_family_id=None, model_family_name=None)[source]#

Bases: object

Class to interact with the model family API to get model configuration info and model-related info.

This class handles fetching and storing model family information, including model inputs, outputs, supported runtimes, metrics, and other metadata.

Parameters:

session (Session) – The session object containing authentication information.
model_family_id (str, optional) – The ID of the model family to fetch.
model_family_name (str, optional) – The name of the model family to fetch.

session#

The session object containing authentication information.

Type:: Session

account_number#

The account number from the session.

Type:: str

project_id#

The project identifier from the session.

Type:: str

rpc#

The RPC client object from the session for API calls.

Type:: RPCClient

model_family_id#

The ID of the model family.

Type:: str

model_family_name#

The name of the model family.

Type:: str

family_data#

The data of the model family fetched from the API.

Type:: dict

model_inputs#

List of model inputs.

Type:: list

model_outputs#

List of model outputs.

Type:: list

model_keys#

Dictionary mapping model keys to model names.

Type:: dict

description#

Description of the model family.

Type:: str

training_framework#

Training framework used for the model family.

Type:: str

supported_runtimes#

List of supported runtimes.

Type:: list

benchmark_datasets#

List of benchmark datasets.

Type:: list

supported_metrics#

List of supported metrics.

Type:: list

input_format#

Input format for the model family.

Type:: str

__get_model_family()#: Fetch a model family by its ID or name.

get_model_archs(model_name=None, model_key=None)[source]#: Fetch model information by model family or by name and key.

Example

>>> session = Session(account_number="your_account_number", access_key="your_access_key", secret_key="your_secret_key")
>>> model_family = ModelFamily(session, model_family_name="resnet")
>>> print(f"Model Family: {model_family.model_family_name}")
>>> print(f"Model Inputs: {model_family.model_inputs}")
>>> print(f"Model Outputs: {model_family.model_outputs}")
>>> print(f"Supported Runtimes: {model_family.supported_runtimes}")
>>> print(f"Supported Metrics: {model_family.supported_metrics}")

Raises:: AssertionError – If neither model_family_id nor model_family_name is provided.

__init__(session, model_family_id=None, model_family_name=None)[source]#

get_model_archs(model_name=None, model_key=None)[source]#

Fetch a model family by its ID or name.

Returns:: A tuple containing the response data, error (if any), and message.
Return type:: tuple

Example

>>> session = Session(account_number="your_account_number", access_key="your_access_key", secret_key="your_secret_key")
>>> model_family = ModelFamily(session, model_family_name="resnet")
>>> resp, error, message = model_family.__get_model_family()
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Model family: {resp}")

matrice.model_store.byom_status_summary(session, project_id, project_name)[source]#

Fetch the BYOM (Bring Your Own Model) status summary for a given project.

Parameters:

project_id (str) – The ID of the project.
project_name (str) – The name of the project.

Returns:

A tuple containing the response data, error (if any), and message.

Return type:

tuple

Example

>>> resp, error, message = byom_status_summary(session,"66912342583678074789d")
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"BYOM status summary: {resp}")

matrice.model_store.check_family_exists_by_name(session, family_name)[source]#

Check if a model family exists by its name.

Parameters:

session (Session) – The session object containing authentication information.
family_name (str) – The name of the model family to check.

Returns:

True if the model family exists, False otherwise.

Return type:

bool

Example

>>> session = Session(account="your_account_number", access_key="your_access_key", secret_key="your_secret_key")
>>> family_name = "ResNet"
>>> exists = check_family_exists_by_name(session, family_name)
>>> if exists:
>>>     print(f"The model family '{family_name}' exists.")
>>> else:
>>>     print(f"The model family '{family_name}' does not exist.")

matrice.model_store.fetch_supported_runtimes_metrics(session, project_id, model_inputs, model_outputs)[source]#

Fetch supported runtimes and metrics for a given project.

Parameters:

model_inputs (list) – List of model inputs.
model_outputs (list) – List of model outputs.

Returns:

A tuple containing the response data, error (if any), and message.

Return type:

tuple

Example

>>> resp, error, message = fetch_supported_runtimes_metrics(session,["image"], ["classification"])
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Supported runtimes and metrics: {resp}")

matrice.model_store.get_automl_config(session, project_id, experiment_id, model_count, recommended_runtime, performance_tradeoff, tuning_type='auto')[source]#

Generate AutoML configurations for model training based on specified parameters.

This static method fetches recommended model configurations from the backend and processes them into a format suitable for model training. It calculates the number of model variants based on hyperparameter combinations.

Parameters:

session (Session) – Active session object for making API calls
project_id (str) – Identifier for the project
experiment_id (str) – Identifier for the experiment
model_count (int) – Number of models to request configurations for
recommended_runtime (bool) – Flag to indicate whether to only include models within recommended runtime
performance_tradeoff (float) – Value indicating the trade-off between performance and resource usage
tuning_type (str, optional) – Type of hyperparameter tuning strategy (default: “auto”)

Returns:

A tuple containing three elements: - model_archs (list): List of ModelArch instances for recommended models - configs (list): List of configuration dictionaries for each model

Each config contains:

is_autoML (bool): Set to True for AutoML

tuning_type (str): Type of tuning strategy

model_checkpoint (str): Checkpoint configuration

checkpoint_type (str): Type of checkpoint

action_config (dict): Raw configuration parameters

model_config (dict): Processed configuration values

model_counts (list): List of integers representing the number of model variants for each model based on hyperparameter combinations

Return type:

tuple

Example

>>> session = Session()
>>> model_archs, configs, counts = get_automl_config(
...     session=session,
...     project_id="project123",
...     experiment_id="exp456",
...     model_count=5,
...     recommended_runtime=True,
...     performance_tradeoff=0.7
... )
>>> for arch, config, count in zip(model_archs, configs, counts):
...     print(f"Model: {arch.model_key}, Variants: {count}")
...     print(f"Config: {config}")

Notes

The number of model variants (model_counts) is calculated by multiplying the number of unique values for batch size, epochs, and learning rate for each model. This represents the total number of training configurations that will be generated for each model architecture.

matrice.model_store.list_private_model_archs(session, project_id=None, project_name=None, page_size=10, page_num=0)[source]#

Fetch private model architectures for a given project.

Parameters:

project_id (str) – The ID of the project.
project_name (str) – The name of the project.
page_size (int, optional) – The number of model architectures to fetch per page (default is 10).
page_num (int, optional) – The page number to fetch (default is 0).

Returns:

A tuple containing the response data, error (if any), and message.

Return type:

tuple

Example

>>> resp, error, message = list_private_model_archs(session,"66912342583678074789d")
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Private model architectures: {resp}")

matrice.model_store.list_private_model_families(session, project_id=None, project_name=None, page_size=10, page_num=0)[source]#

Fetch private model families for a given project.

Parameters:

project_id (str) – The ID of the project.
project_name (str) – The name of the project.
page_size (int, optional) – The number of model families to fetch per page (default is 10).
page_num (int, optional) – The page number to fetch (default is 0).

Returns:

A tuple containing the response data, error (if any), and message.

Return type:

tuple

Example

>>> resp, error, message = list_private_model_families(session,"66912342583678074789d")
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Private model families: {resp}")

matrice.model_store.list_public_model_archs(session, project_type='classification', page_size=10, page_num=0)[source]#

Fetch public model architectures for a given project.

Parameters:

project_type (str, optional) – The type of the project (default is “classification”)(Available types are “detection” and “instance_segmentation”).
page_size (int, optional) – The number of model architectures to fetch per page (default is 10).
page_num (int, optional) – The page number to fetch (default is 0).

Returns:

A tuple containing the response data, error (if any), and message.

Return type:

tuple

Example

>>> resp, error, message = list_public_model_archs(session,"classification")
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Public model architectures: {resp}")

matrice.model_store.list_public_model_families(session, project_type='classification', page_size=10, page_num=0)[source]#

Fetch public model families for a given project.

Parameters:

project_type (str, optional) – The type of the project (default is “classification”)(Available types are “detection” and “instance_segmentation”).
page_size (int, optional) – The number of model families to fetch per page (default is 10).
page_num (int, optional) – The page number to fetch (default is 0).

Returns:

A tuple containing the response data, error (if any), and message.

Return type:

tuple

Example

>>> resp, error, message = list_public_model_families(session,"classification")
>>> if error:
>>>     print(f"Error: {error}")
>>> else:
>>>     print(f"Public model families: {resp}")