Science Data Challenge 2 scoring service CLI¶
This package contains two simple command line interface (CLI) applications for interaction with the SDC2 online scoring service:
sdc2-score: A tool for upload and check the status of submissions.
sdc2-score-admin: A tool managing SDC2 users and groups.
This package can be installed with:
pip install ska-sdc2-scoring-utils
Create a submission and generate its SUBMISSION_ID using
sdc2-score create CATALOGUE_FILE
Retrieve information about the submission using
sdc2-score info SUBMISSION_ID
All submissions belonging to a team can be printed using
By default, all commands are run against the ‘full’ dataset. The -c flag can be used to run commands against the ‘dev’ dataset.
Credentials can either be specified using the command line options as follows:
sdc2-score create -u USER -p PASSWORD CATALOGUE_FILE
or are loaded from the following environment variables:
If both are supplied, the command line option flags are used.
Usage: sdc2-score leaderboard [--sub-info] [--score-info] [--raw] [-c VERSION] sdc2-score create CATALOGUE_FILE [-u USERNAME] [-p PASSWORD] [-c VERSION] [--hidden] sdc2-score info SUBMISSION_ID [-c VERSION] [--raw] sdc2-score ls [-u USERNAME] [-p PASSWORD] [-n LIMIT] [--score-info] [--raw] [-c VERSION] sdc2-score -h | --help | --version Options: --sub-info Display additional submission information. --score-info Display additional score information. --raw Display the raw response from the SDC scoring service. --hidden If passed, submission will be hidden from leaderboards -c VERSION Set the catalogue version (1.dev, 1.ldev, 1.full) [default: 1.full]. -n LIMIT Maximum number of submissions to list [default: 20]. -u USER Scoring service username. -p PASSWORD Scoring service user password. -h, --help Show this screen. --version Show the version. Arguments CATALOGUE_FILE Path to catalogue file to submit. SUBMISSION_ID Submission ID, minimally matched.
The admin client is intended for SDC2 challenge administrators and requires that the following environment variables are set:
Once these are set, the basic usage settings are as follows:
Usage: sdc2-score-admin group ls [-u] [--raw] sdc2-score-admin group add GROUP_NAME... sdc2-score-admin group rm GROUP_ID... sdc2-score-admin user ls [--raw] sdc2-score-admin user add FIRST_NAME LAST_NAME USERNAME EMAIL GROUP [PASSWORD] sdc2-score-admin user rm USER_ID... sdc2-score-admin submission ls [-n LIMIT] [-c VERSION] [--raw] sdc2-score-admin submission rm SUBMISSION_ID [-c VERSION] sdc2-score-admin (-h | --help | --version) Options: -u Show users in the group. --raw Display unformatted API response JSON object. -n LIMIT Maximum number of submissions to show [default: 20]. -c VERSION Catalogue version (1.full, 1.dev) [default: 1.full]. -h, --help Show this screen. --version Show the version. Arguments: EMAIL Email address of the user. FIRST_NAME User's first name. GROUP_ID Group name or ID, minimally matched. GROUP_NAME Name of the group. LAST_NAME User's last name. PASSWORD Password for the user. Optional, autogenerated if not set. SUBMISSION_ID UUID for a submission, minimally matched USER_ID User name or ID, minimally matched. USERNAME User's user name
For local development, commands can also be run and tested using:
python -m ska_sdc2_scoring_utils.sdc2_score python -m ska_sdc2_scoring_utils.sdc2_score_admin