Cometx
Open source contributions for integrations with comet_ml
Install / Use
/learn @comet-ml/CometxREADME
cometx
_________ ____ ___ ___ / /__ __
/ ___/ __ \/ __ `__ \/ _ \/ __/ |/_/
/ /__/ /_/ / / / / / / __/ /__> <
\___/\____/_/ /_/ /_/\___/\__/_/|_|
Open source extensions for the Comet SDK.
These extensions are created and supported by the community and are not an official project of Comet ML. We welcome contributions!
Installation
pip install cometx --upgrade
To use these command-line functions, you can set your Comet API key and URL override using command-line flags:
cometx --api-key="YOUR-COMET-API-KEY" COMMAND
If you are a Comet on-prem user, and your installation does not use smart-keys, you'll also need to set the URL override:
cometx --api-key="YOUR-COMET-API-KEY" --url-override="https://your-companys-comet.com/clientlib/" COMMAND
Usage
cometx is composed of a series of commands that are useful
independently, and can be used together to create sophisticated tools
for ML management.
Commands
- cometx admin
- cometx config
- cometx copy
- cometx count
- cometx delete-assets
- cometx download
- cometx list
- cometx log
- cometx migrate-users
- cometx rename-duplicates
- cometx reproduce
- cometx smoke-test
- cometx update
For all commands, use the --help flag to get additional information.
Global Options
These flags are availble before a command:
--api-key API_KEY- Set the COMET_API_KEY--url-override URL_OVERRIDE- Set the COMET_URL_OVERRIDE
This command shows the cometx version:
--version- Display comet_ml version
This command can be used globally, or for individual commands:
-h, --help- Show help message
cometx list
This command is used to:
- get a list of all workspaces that you are a member of
- get a list of all projects in a workspace
- get a list of all experiments (by name or key) in a project
cometx list examples:
cometx list WORKSPACE/PROJECT/EXPERIMENT-KEY-OR-NAME
cometx list WORKSPACE/PROJECT
cometx list WORKSPACE
cometx list
Flags
-u, --use-name- Use experiment names for experiment folders and listings--query QUERY- Only list experiments that match this Comet query string--debug- Provide debug info
For more information, cometx list --help
cometx count
This command is used to:
- count the number of workspaces you are a member of
- count the number of projects in workspaces
- count the number of experiments in projects
- count the number of artifacts in workspaces
cometx count examples:
cometx count
cometx count --workspaces-only
cometx count --with-projects
cometx count --with-experiments
cometx count --count-all
cometx count --limit 10
Flags
--workspaces-only- Count only workspaces (fastest)--with-projects- Count workspaces and projects (default)--with-experiments- Count workspaces, projects, and experiments (slowest, most detailed)--count-all- Count everything: workspaces, projects, artifacts, and experiments (most comprehensive)--limit LIMIT- Process only the first N workspaces (useful for testing)--debug- Provide debug info
For more information, cometx count --help
cometx download
This command is used to:
- download all workspaces, projects, and experiments of workspaces that you are a member of
- download all projects, and experiments of a given workspace
- download all experiments of a given workspace/project
- download artifacts and models from the registry
- download panels
Note: For detailed information on copying experiments from one Comet installation to another, see MIGRATIONS.md.
cometx download examples:
cometx download WORKSPACE/PROJECT/EXPERIMENT-KEY-OR-NAME [RESOURCE ...] [FLAGS ...]
cometx download WORKSPACE/PROJECT [RESOURCE ...] [FLAGS ...]
cometx download WORKSPACE [RESOURCE ...] [FLAGS ...]
cometx download [RESOURCE ...] [FLAGS ...]
Where [RESOURCE ...] is zero or more of the following names:
- run - alias for: code, git, output, graph, and requirements
- system
- others
- parameters
- metadata
- metrics
- assets
- html
- project - alias for: project_notes, project_metadata
If no RESOURCE is given it will download all of them.
Flags
--from from- Source of data to copy. Should be: comet, wandb, or neptune. When using--from neptune, you must set theNEPTUNE_API_TOKENenvironment variable.-i IGNORE [IGNORE ...], --ignore IGNORE [IGNORE ...]- Resource(s) (or 'experiments') to ignore-j PARALLEL, --parallel PARALLEL- The number of threads to use for parallel downloading; default (None) is based on CPUs-o OUTPUT, --output OUTPUT- Output directory for downloads-u, --use-name- Use experiment names for experiment folders and listings-l, --list- List the items at this level (workspace, project, experiment, artifacts, or model-registry) rather than download--flat- Download the files without subfolders-f, --ask- Queries the user; if flag not included system will answeryesfor all queries--filename FILENAME- Only get resources ending with this--query QUERY- Only download experiments that match this Comet query string--asset-type ASSET_TYPE- Only get assets with this type--sync SYNC- What level to sync at: all, experiment, project, or workspace--debug- Provide debug info
To download artifacts:
cometx download WORKSPACE/artifacts/NAME [FLAGS ...]
cometx download WORKSPACE/artifacts/NAME/VERSION-OR-ALIAS [FLAGS ...]
To download models from the model registry:
cometx download WORKSPACE/model-registry/NAME [FLAGS ...]
cometx download WORKSPACE/model-registry/NAME/VERSION-OR-STAGE [FLAGS ...]
To download panels:
cometx download WORKSPACE/panels/NAME-OR-ID [FLAGS ...]
cometx download WORKSPACE/panels [FLAGS ...]
For more information, cometx download --help
cometx copy
This command is used to:
- copy downloaded data to a new experiment
- create a symlink from one project to existing experiments
- copy panels
Note: For detailed information on copying experiments from one Comet installation to another, see MIGRATIONS.md.
cometx copy examples:
cometx copy SOURCE DESTINATION
cometx copy --symlink SOURCE DESTINATION
cometx copy --path /base/path SOURCE DESTINATION
cometx copy --path ~/Downloads SOURCE DESTINATION
where SOURCE is:
- if not
--symlink, "WORKSPACE/PROJECT/EXPERIMENT", "WORKSPACE/PROJECT", or "WORKSPACE" folder - if
--symlink, then it is a Comet path to workspace or workspace/project - "WORKSPACE/panels" or "WORKSPACE/panels/PANEL-ZIP-FILENAME" to copy panels
where DESTINATION is:
- WORKSPACE
- WORKSPACE/PROJECT
Not all combinations are possible:
| Destination → <br/>Source ↓ | WORKSPACE | WORKSPACE/PROJECT |
|--------------------|----------------------|------------------------|
| WORKSPACE/*/* | Copies all projects | N/A |
| WORKSPACE/PROJ/* | N/A | Copies all experiments |
| WORKSPACE/PROJ/EXP | N/A | Copies experiment |
Asset Types
- 3d-image
- 3d-points - deprecated
- audio
- confusion-matrix - may contain assets
- curve
- dataframe
- dataframe-profile
- datagrid
- embeddings - may reference image asset
- histogram2d - not used
- histogram3d - internal only, single histogram, partial logging
- histogram_combined_3d
- image
- llm_data
- model-element
- notebook
- source_code
- tensorflow-model-graph-text - not used
- text-sample
- video
Flags
-i IGNORE [IGNORE ...], --ignore IGNORE [IGNORE ...]- Resource(s) (or 'experiments') to ignore-j PARALLEL, --parallel PARALLEL- The number of threads to use for parallel uploading; default (None) is based on CPUs--debug- If given, allow debugging--quiet- If given, don't display update info--symlink- Instead of copying, create a link to an experiment in a project--sync- Check to see if experiment name has been created first; if so, skip--path PATH- Path to prepend to workspace_src when accessing files (supports ~ for home directory)--create-workspaces- Attempt to create the destination workspace if it does not exist (requires appropriate API permissions)
Using --path
The --path option allows you to specify a base directory where your workspace folders are located. This is useful when your downloaded experiments are stored in a specific directory structure.
Examples:
# Copy from experiments in /data/experiments/workspace
cometx copy --path /data/experiments workspace dest-workspace
# Copy from experiments in your home directory
cometx copy --path ~ workspace dest-workspace
# Copy from experiments in Downloads folder
cometx copy --path ~/Downloads workspace dest-workspace
For more information, cometx copy --help
cometx log
This command is used to log a resource (metrics, parameters, asset, etc) file to a specific experiment or experiments.
cometx log examples:
cometx log WORKSPACE/PROJECT/EXPERIMENT-KEY FILENAME ... --type=TYPE
cometx log WORKSPACE PANEL-ZIP-FILENAME ... --type=panel
cometx log WORKSPACE PANEL.py ... --type=panel
cometx log WORKSPACE PANEL-URL ... --type=panel
cometx log WORKSPACE/PROJECT --type=other --set "key:value"
cometx log WORKSPACE --type=other --set "key:value"
Where TYPE is one of the following names:
- all
- asset
- audio
- code
- image
- metrics
- notebook
- panel
- tensorflow-file
- text-sample
- video
- other
- tensorboard-folder-assets
Flags
--type TYPE
