matrice.testing module

class matrice.testing.ModelDownloadMock

Bases: object

Mock class for downloading models in the testing pipeline.

__init__()

Initializes the ModelDownloadMock class and sets up the testing logs folder path.

download_model(model_path, model_type='trained', runtime_framework='')

Mock method to download a model file and copy it to the specified path.

Parameters:

• model_path (str) – Path where the model should be downloaded.

• model_type (str, optional) – Type of model to download (‘trained’ or ‘exported’). Default is ‘trained’.

• runtime_framework (str, optional) – Runtime framework used for the model (default is ‘’).

Returns:

Returns True after successfully copying the model file.

Return type:

bool

class matrice.testing.SplitMetricStruct(**data)

Bases: BaseModel

This is a private class used internally to store split metrics.

splitType

Type of the dataset split (e.g., ‘train’, ‘val’, ‘test’).

Type:

str

metricName

Name of the evaluation metric (e.g., ‘accuracy’, ‘precision’).

Type:

str

metricValue

Value of the metric for the given split.

Type:

float

metricName: str

metricValue: float

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'metricName': FieldInfo(annotation=str, required=True), 'metricValue': FieldInfo(annotation=float, required=True), 'splitType': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

splitType: str

class matrice.testing.TestingActionTracker(model_family_info_path, model_info_path, config_path)

Bases: object

Handles logging, dataset preparation, and configuration management for model testing actions.

Parameters:

• model_family_info_path (str) – Path to the model family information file.

• model_info_path (str) – Path to the model information file.

• config_path (str) – Path to the action configuration file.

__init__(model_family_info_path, model_info_path, config_path)

Initializes the TestingActionTracker class, loading model family info, model info, and configurations.

Parameters:

• model_family_info_path (str) – Path to the model family information JSON file.

• model_info_path (str) – Path to the model information JSON file.

• config_path (str) – Path to the action configuration file.

add_index_to_category(*args, **kwargs)

add_logs(step, status, description)

Adds a log entry for a specific step, including status and description.

Parameters:

• step (str) – The step or action being logged (e.g., ‘load_model’).

• status (str) – The status of the step (e.g., ‘SUCCESS’, ‘ERROR’).

• description (str) – A description or error message related to the step.

cast_value(value_type, value)

Casts a value to its specified type (int, float, string, bool).

Parameters:

• value_type (str) – The type to cast the value to (e.g., ‘int32’, ‘float32’).

• value (any) – The value to be cast.

Returns:

The casted value.

Return type:

any

convert_bbox_to_yolo(size, box)

Converts bounding box coordinates to YOLO format.

Parameters:

• size (tuple) – The width and height of the image.

• box (list) – Bounding box coordinates in the format [x, y, width, height].

Returns:

Converted bounding box in YOLO format.

Return type:

tuple

create_data_yaml(dataset_dir, class_names)

Creates a data.yaml file for the YOLO model from the dataset.

Parameters:

• dataset_dir (str) – The directory where the dataset is located.

• class_names (list) – List of class names in the dataset.

create_yolo_labels_from_mscoco_ann(dataset_dir, dst_images_dir, dst_annotations_dir, annotation_file)

Creates YOLO labels from MSCOCO annotations.

Parameters:

• dataset_dir (str) – Directory where the dataset is stored.

• dst_images_dir (str) – Directory where images are stored.

• dst_annotations_dir (str) – Directory where annotations are stored.

• annotation_file (str) – Path to the MSCOCO annotation file.

Returns:

List of class names from the annotations.

Return type:

list

download_and_extract_dataset(dataset_url, dataset_dir)

Downloads and extracts a dataset from a given URL.

Parameters:

• dataset_url (str) – The URL from which to download the dataset.

• dataset_dir (str) – The directory where the dataset should be extracted.

download_model(*args, **kwargs)

get_checkpoint_path(*args, **kwargs)

get_file_extension(content_type)

Returns the appropriate file extension based on content type.

Parameters:

content_type (str) – The content type of the file.

Returns:

The file extension (e.g., ‘.zip’, ‘.tar’).

Return type:

str

get_index_to_category(*args, **kwargs)

get_job_params(*args, **kwargs)

get_main_action_logs_path()

Determines the appropriate log file path based on the action type (train, export, eval).

Returns:

Path to the main log file for the current action.

Return type:

str

get_model_train(*args, **kwargs)

load_action_config(*args, **kwargs)

load_model_family_info(*args, **kwargs)

load_model_info(*args, **kwargs)

log_decorator()

log_epoch_results(*args, **kwargs)

log_to_json(file_path, payload)

Logs data to a JSON file, appending the payload if the file exists.

Parameters:

• file_path (str) – Path to the JSON log file.

• payload (dict) – The data to log in the JSON file.

mock_action_doc(*args, **kwargs)

prepare_classification_dataset(dataset_dir)

Prepares a dataset for classification tasks.

Parameters:

dataset_dir (str) – The directory where the dataset is located.

prepare_dataset(*args, **kwargs)

prepare_detection_dataset(dataset_dir)

Prepares a dataset for object detection tasks.

Parameters:

dataset_dir (str) – The directory where the dataset is located.

prepare_yolo_dataset(dataset_dir)

Prepares the dataset for YOLO model training.

Parameters:

dataset_dir (str) – The directory where the dataset is located.

round_metrics(epoch_result_list)

Rounds the metric values to four decimal places, replacing NaN or inf with 0.

Parameters:

epoch_result_list (List[dict]) – List of metrics with values to be rounded.

Returns:

List of metrics with rounded values.

Return type:

List[dict]

save_evaluation_results(*args, **kwargs)

update_status(stepCode, status, status_description)

Mocks the status update for a given step, adding it to logs.

Parameters:

• stepCode (str) – The code for the current step.

• status (str) – The current status (e.g., ‘SUCCESS’, ‘ERROR’).

• status_description (str) – Description or details about the step status.

upload_checkpoint(*args, **kwargs)

validate_metrics_structure(metrics_list)

Validates the structure of a list of metrics.

Parameters:

metrics_list (List[SplitMetricStruct]) – List of metrics to be validated.

Returns:

The validated metrics.

Return type:

List[SplitMetricStruct]

class matrice.testing.TestingMatriceDeploy(load_model, predict)

Bases: object

Class to handle deployment and inference of models for testing purposes.

This class handles model downloading, logging, and running inference with a provided model.

Parameters:

• load_model (function) – Function to load a model during testing.

• predict (function) – Function to make predictions using the loaded model.

__init__(load_model, predict)

Initializes the TestingMatriceDeploy class, setting up logs and triggering inference.

Parameters:

• load_model (function) – Function that loads a model for inference.

• predict (function) – Function to perform prediction with the loaded model.

add_logs(step, status, description)

Adds a log entry for a specific step, including status and description.

Parameters:

• step (str) – The step or action being logged (e.g., ‘inference’).

• status (str) – The status of the step (e.g., ‘SUCCESS’, ‘ERROR’).

• description (str) – A description or error message related to the step.

create_image_bytes()

Creates a simple test image in memory as a byte stream.

Returns:

Image data in JPEG format.

Return type:

bytes

inference(*args, **kwargs)

load_predictor_model(*args, **kwargs)

log_decorator()

A decorator to log the execution status of a function.

log_to_json(file_path, payload)

Logs data to a JSON file, appending the payload if the file exists.

Parameters:

• file_path (str) – Path to the JSON log file.

• payload (dict) – The data to log in the JSON file.

class matrice.testing.dotdict

Bases: dict

A dictionary subclass that provides dot notation access to attributes.

__getattr__

Allows accessing dictionary keys as object attributes.

Type:

function

__setattr__

Allows setting dictionary keys as object attributes.

Type:

function

__delattr__

Allows deleting dictionary keys as object attributes.

Type:

function