Command Line Interface¶
mlcube¶
MLCube® is a tool for packaging, distributing and running Machine Learning (ML) projects and models.
- GitHub: https://github.com/mlcommons/mlcube
- Documentation: https://mlcommons.github.io/mlcube/
- Example MLCubes: https://github.com/mlcommons/mlcube_examples
Usage:
mlcube [OPTIONS] COMMAND [ARGS]...
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--log-level, --log_level |
choice (critical | error | warning | info | debug) |
Logging level is a lower-case string value for Python's logging library (see Logging Levels for more details). Only messages with this logging level or higher are logged. | warning |
--help, -h |
boolean | Show help message and exit. | False |
Subcommands
- config: Display or change MLCube system settings.
- configure: Configure MLCube.
- create: Create a new Python project from the MLCube cookiecutter template.
- describe: Describe this MLCube.
- inspect: Return low-level information on MLCube objects.
- run: Run MLCube task(s).
- show_config: Show effective MLCube configuration.
mlcube config¶
Display or change MLCube system settings.
MLCube system settings define global configuration common for all MLCube runners and platforms. When this command runs without arguments, a path to system settings file is printed out. This is useful to automate certain operations with system settings. system settings file is printed out. This is useful to automate certain operations with system settings.
Alternatively, it may be easier to manipulate system settings file directly (it is a yaml file).
Usage:
mlcube config [OPTIONS]
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--list |
boolean | Print out the content of system settings file. | False |
--get |
text | Return value of the key (use OmegaConf notation, e.g. mlcube config --get runners.docker). |
None |
--create_platform, --create-platform |
tuple | Create a new platform instance for this runner. Default runner parameters are used to initialize this new platform. | None |
--remove_platform, --remove-platform |
text | Remove this platform. If this is one of the default platforms (e.g., docker), it will be recreated (with default values) next time mlcube runs. |
None |
--rename_platform, --rename-platform |
tuple | Rename existing platform. If default platform is to be renamed (e.g., docker), it will be recreated (with default values) next time mlcube runs. |
None |
--copy_platform, --copy-platform |
tuple | Copy existing platform. This can be useful for creating new platforms off existing platforms, for instance,creating a new SSH runner configuration that runs MLCubes on a new remote server. | None |
--rename_runner, --rename-runner |
tuple | Rename existing MLCube runner. If platforms exist that reference this runner, users must explicitly provide --update-platforms flag to confirm they want to update platforms' description too. |
None |
--remove_runner, --remove-runner |
text | Remove existing runner. If platforms exist that reference this runner, users must explicitly provide --remove-platforms flag to confirm they want to remove platforms too. |
None |
--help, -h |
boolean | Show help message and exit. | False |
mlcube configure¶
Configure MLCube.
Some MLCube projects need to be configured first. For instance, docker-based MLCubes distributed via GitHub with
source code most likely will provide a Dockerfile to build a docker image. In this case, the process of building
a docker image before MLCube runner can run it, is called a configuration phase. In general, users do not need to
run this command manually - MLCube runners should be able to figure out when they need to run it, and will run it
as part of mlcube run command.
Usage:
mlcube configure [OPTIONS]
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--mlcube |
text | Path to an MLCube project. It can be a directory path, or a path to an MLCube configuration file. When it is a directory path, MLCube runtime assumes this directory is the MLCube root directory containing mlcube.yaml file. When it is a file path, this file is assumed to be the MLCube configuration file (mlcube.yaml), and a parent directory of this file is considered to be the MLCube root directory. Default value is current directory. |
None |
--platform |
text | Platform name to run MLCube on (a platform is a configured instance of an MLCube runner). Multiple platforms are supported, including docker (Docker and Podman), singularity (Singularity). Other runners are in experimental stage: gcp (Google Cloud Platform), k8s (Kubernetes), kubeflow (KubeFlow), ssh (SSH runner). Default is docker. Platforms are defined and configured in MLCube system settings file. |
docker |
-P, -p |
text | MLCube configuration parameter is a key-value pair. Must start with -P or '-p'. The dot (.) is used to refer to nested parameters, for instance, -Pdocker.build_strategy=always. These parameters have the highest priority and override any other parameters in system settings and MLCube configuration. |
None |
--help, -h |
boolean | Show help message and exit. | False |
mlcube create¶
Create a new Python project from the MLCube cookiecutter template.
MLCube uses the cookiecutter library with the
mlcube_cookiecutter template. The library is not installed
automatically: install it with pip install cookiecutter.
Usage:
mlcube create [OPTIONS]
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--help, -h |
boolean | Show help message and exit. | False |
mlcube describe¶
Describe this MLCube.
Usage:
mlcube describe [OPTIONS]
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--mlcube |
text | Path to an MLCube project. It can be a directory path, or a path to an MLCube configuration file. When it is a directory path, MLCube runtime assumes this directory is the MLCube root directory containing mlcube.yaml file. When it is a file path, this file is assumed to be the MLCube configuration file (mlcube.yaml), and a parent directory of this file is considered to be the MLCube root directory. Default value is current directory. |
None |
--help, -h |
boolean | Show help message and exit. | False |
mlcube inspect¶
Return low-level information on MLCube objects.
Usage:
mlcube inspect [OPTIONS]
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--mlcube |
text | Path to an MLCube project. It can be a directory path, or a path to an MLCube configuration file. When it is a directory path, MLCube runtime assumes this directory is the MLCube root directory containing mlcube.yaml file. When it is a file path, this file is assumed to be the MLCube configuration file (mlcube.yaml), and a parent directory of this file is considered to be the MLCube root directory. Default value is current directory. |
None |
--platform |
text | Platform name to run MLCube on (a platform is a configured instance of an MLCube runner). Multiple platforms are supported, including docker (Docker and Podman), singularity (Singularity). Other runners are in experimental stage: gcp (Google Cloud Platform), k8s (Kubernetes), kubeflow (KubeFlow), ssh (SSH runner). Default is docker. Platforms are defined and configured in MLCube system settings file. |
docker |
--force |
boolean | Force inspecting the MLCube object. For instance, if MLCube has not been pulled or built it, then pull or build it. | False |
--format |
choice (json | yaml) |
Format for reporting results. | json |
--output-file, --output_file |
text | File path to store the MLCube information. Defaults to print to STDOUT | None |
--help, -h |
boolean | Show help message and exit. | False |
mlcube run¶
Run MLCube task(s).
Usage:
mlcube run [OPTIONS]
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--mlcube |
text | Path to an MLCube project. It can be a directory path, or a path to an MLCube configuration file. When it is a directory path, MLCube runtime assumes this directory is the MLCube root directory containing mlcube.yaml file. When it is a file path, this file is assumed to be the MLCube configuration file (mlcube.yaml), and a parent directory of this file is considered to be the MLCube root directory. Default value is current directory. |
None |
--platform |
text | Platform name to run MLCube on (a platform is a configured instance of an MLCube runner). Multiple platforms are supported, including docker (Docker and Podman), singularity (Singularity). Other runners are in experimental stage: gcp (Google Cloud Platform), k8s (Kubernetes), kubeflow (KubeFlow), ssh (SSH runner). Default is docker. Platforms are defined and configured in MLCube system settings file. |
docker |
--task |
text | MLCube task name(s) to run, default is main. This parameter can take a list of values, in which case task names are separated with comma (,). |
None |
--workspace |
text | Location of a workspace to store input and output artifacts of MLCube tasks. If not specified (None), ${MLCUBE_ROOT}/workspace/ is used. |
None |
--network |
text | Networking options defined during MLCube container execution. | None |
--security |
text | Security options defined during MLCube container execution. | None |
--gpus |
text | GPU usage options defined during MLCube container execution. | None |
--memory |
text | Memory RAM options defined during MLCube container execution. | None |
--cpu |
text | CPU options defined during MLCube container execution. | None |
--mount |
choice (rw | ro) |
Mount options for all input parameters. These mount options override any other mount options defined for each input parameters. A typical use case is to ensure that inputs are mounted in read-only (ro) mode. | None |
-P, -p |
text | MLCube configuration parameter is a key-value pair. Must start with -P or '-p'. The dot (.) is used to refer to nested parameters, for instance, -Pdocker.build_strategy=always. These parameters have the highest priority and override any other parameters in system settings and MLCube configuration. |
None |
--help, -h |
boolean | Show help message and exit. | False |
mlcube show_config¶
Show effective MLCube configuration.
Effective MLCube configuration is the one used by one of MLCube runners to run this MLCube. This configuration is built by merging (1) default runner configuration retrieved from system settings, (2) MLCube project configuration and (3) configuration parameters passed by a user on a command line (CONFIG_PARAM).
Usage:
mlcube show_config [OPTIONS]
Options:
| Name | Type | Description | Default |
|---|---|---|---|
--mlcube |
text | Path to an MLCube project. It can be a directory path, or a path to an MLCube configuration file. When it is a directory path, MLCube runtime assumes this directory is the MLCube root directory containing mlcube.yaml file. When it is a file path, this file is assumed to be the MLCube configuration file (mlcube.yaml), and a parent directory of this file is considered to be the MLCube root directory. Default value is current directory. |
None |
--platform |
text | Platform name to run MLCube on (a platform is a configured instance of an MLCube runner). Multiple platforms are supported, including docker (Docker and Podman), singularity (Singularity). Other runners are in experimental stage: gcp (Google Cloud Platform), k8s (Kubernetes), kubeflow (KubeFlow), ssh (SSH runner). Default is docker. Platforms are defined and configured in MLCube system settings file. |
docker |
--workspace |
text | Location of a workspace to store input and output artifacts of MLCube tasks. If not specified (None), ${MLCUBE_ROOT}/workspace/ is used. |
None |
--resolve |
boolean | Resolve MLCube parameters. The mlcube uses OmegaConf library to manage its configuration, including configuration files, system settings files and configuration parameters provided by users on command lines. OmegaConf supports variable interpolation (when one variables depend on other variables, e.g., {'docker.image': 'mlcommons/{name}:${version}'}). When this flag is set to true, the mlcube computes actual values of all variables. |
False |
-P, -p |
text | MLCube configuration parameter is a key-value pair. Must start with -P or '-p'. The dot (.) is used to refer to nested parameters, for instance, -Pdocker.build_strategy=always. These parameters have the highest priority and override any other parameters in system settings and MLCube configuration. |
None |
--help, -h |
boolean | Show help message and exit. | False |