Command Line Interfaces
This module provides click-based Command-Line Interface (CLI) scripts that allow accessing all user-facing features from this library through the terminal.
sl-crc
Calculates the CRC32-XFER checksum for the input string.
sl-crc [OPTIONS]
Options
- -i, --input_string <input_string>
The string to calculate the CRC checksum for.
sl-devices
Displays information about all Zaber devices available through USB ports of the host-system.
sl-devices [OPTIONS]
Options
- -e, --errors
Determines whether to display errors encountered when connecting to evaluated serial ports.
- Default:
False
sl-create-system-config
Generates a precursor system configuration file for the target acquisition system and configures all local Sun lab libraries to use that file to load the acquisition system configuration data.
This command is typically used when setting up a new data acquisition system in the lab. The system configuration only needs to be specified on the machine (PC) that runs the sl-experiment library and manages the acquisition runtime if the system uses multiple machines (PCs). Once the system configuration .yaml file is created via this command, editing the configuration parameters in the file will automatically take effect during all following runtimes.
sl-create-system-config [OPTIONS]
Options
- -od, --output_directory <output_directory>
Required The absolute path to the directory where to store the generated system configuration file.
- -as, --acquisition_system <acquisition_system>
Required The type (name) of the data acquisition system for which to generate the configuration file. Note, currently, only the following types are supported: mesoscope-vr.
- Default:
'mesoscope-vr'
sl-create-project
Generates a new project directory hierarchy on the local machine.
This command creates new Sun lab projects. Until a project is created in this fashion, all data-acquisition and data-processing commands from sl-experiment library targeting the project will not work. This command is intended to be called on the main computer of the data-acquisition system(s) used by the project. Note, this command assumes that the local machine (PC) is the main PC of the data acquisition system and has a valid acquisition system configuration .yaml file.
sl-create-project [OPTIONS]
Options
- -p, --project <project>
Required The name of the project to be created.
sl-create-experiment
Generates a precursor experiment configuration .yaml file for the target experiment inside the project’s configuration folder.
This command assists users in creating new experiment configurations by statically resolving the structure (layout) of the appropriate experiment configuration file for the acquisition system of the local machine (PC). Specifically, the generated precursor will contain the correct number of experiment state entries initialized to nonsensical default value. The user needs to manually edit the configuration file to properly specify their experiment runtime parameters and state transitions before running the experiment. In a sense, this command acts as an ‘experiment template’ generator.
sl-create-experiment [OPTIONS]
Options
- -p, --project <project>
Required The name of the project for which to generate the new experiment configuration file.
- -e, --experiment <experiment>
Required The name of the experiment. Note, the generated experiment configuration file will also use this name.
- -sc, --state_count <state_count>
Required The total number of experiment and acquisition system state combinations in the experiment.
- -tc, --trial_count <trial_count>
Required The total number of unique trial types used in the experiment.
sl-maintain
Exposes a terminal interface to interact with the water delivery solenoid valve and the running-wheel break.
This CLI command is primarily designed to fill, empty, check, and, if necessary, recalibrate the solenoid valve used to deliver water to animals during training and experiment runtimes. Also, it is capable of locking or unlocking the wheel breaks, which is helpful when cleaning the wheel (after each session) and maintaining the wrap around the wheel surface (weekly to monthly).
sl-maintain [OPTIONS]
sl-lick-train
Runs the lick training session for the specified animal and project combination.
Lick training is the first phase of preparing the animal to run experiment runtimes in the lab, and is usually carried out over the first two days of head-fixed training. Primarily, this training is designed to teach the animal to operate the lick-port and associate licking at the port with water delivery.
sl-lick-train [OPTIONS]
Options
- -u, --user <user>
Required The ID of the user supervising the training session.
- -p, --project <project>
Required The name of the project to which the trained animal belongs.
- -a, --animal <animal>
Required The ID of the animal undergoing the lick training session.
- -w, --animal_weight <animal_weight>
Required The weight of the animal, in grams, at the beginning of the training session.
- -min, --minimum_delay <minimum_delay>
The minimum number of seconds that has to pass between two consecutive reward deliveries during training.
- Default:
6
- -max, --maximum_delay <maximum_delay>
The maximum number of seconds that can pass between two consecutive reward deliveries during training.
- Default:
18
- -v, --maximum_volume <maximum_volume>
The maximum volume of water, in milliliters, that can be delivered during training.
- Default:
1.0
- -t, --maximum_time <maximum_time>
The maximum time to run the training, in minutes.
- Default:
20
- -ur, --unconsumed_rewards <unconsumed_rewards>
The maximum number of rewards that can be delivered without the animal consuming them, before reward delivery is paused. Set to 0 to disable enforcing reward consumption.
- Default:
1
- -r, --restore_parameters
Determines whether to load and use the same training parameters as used during the previous lick training session of the target animal. Note, this only overrides the maximum and minimum reward delays, all other parameters are not affected by this flag.
- Default:
False
sl-run-train
Runs the run training session for the specified animal and project combination.
Run training is the second phase of preparing the animal to run experiment runtimes in the lab, and is usually carried out over the five days following the lick training sessions. Primarily, this training is designed to teach the animal how to run the wheel treadmill while being head-fixed and associate getting water rewards with running on the treadmill. Over the course of training, the task requirements are adjusted to ensure the animal performs as many laps as possible during experiment sessions lasting ~60 minutes.
sl-run-train [OPTIONS]
Options
- -u, --user <user>
Required The ID of the user supervising the training session.
- -p, --project <project>
Required The name of the project to which the trained animal belongs.
- -a, --animal <animal>
Required The name of the animal undergoing the run training session.
- -w, --animal_weight <animal_weight>
Required The weight of the animal, in grams, at the beginning of the training session.
- -is, --initial_speed <initial_speed>
The initial speed, in centimeters per second, the animal must maintain to obtain water rewards.
- Default:
0.4
- -id, --initial_duration <initial_duration>
The initial duration, in seconds, the animal must maintain above-threshold running speed to obtain water rewards.
- Default:
0.4
- -it, --increase_threshold <increase_threshold>
The volume of water delivered to the animal, in milliliters, after which the speed and duration thresholds are increased by the specified step-sizes. This is used to make the training progressively harder for the animal over the course of the training session.
- Default:
0.1
- -ss, --speed_step <speed_step>
The amount, in centimeters per second, to increase the speed threshold each time the animal receives the volume of water specified by the ‘increase-threshold’ parameter.
- Default:
0.05
- -ds, --duration_step <duration_step>
The amount, in seconds, to increase the duration threshold each time the animal receives the volume of water specified by the ‘increase-threshold’ parameter.
- Default:
0.05
- -v, --maximum_volume <maximum_volume>
The maximum volume of water, in milliliters, that can be delivered during training.
- Default:
1.0
- -t, --maximum_time <maximum_time>
The maximum time to run the training, in minutes.
- Default:
40
- -ur, --unconsumed_rewards <unconsumed_rewards>
The maximum number of rewards that can be delivered without the animal consuming them, before reward delivery is paused. Set to 0 to disable enforcing reward consumption.
- Default:
1
- -mit, --maximum_idle_time <maximum_idle_time>
The maximum time, in seconds, the animal is allowed to maintain speed that is below the speed threshold, tostill be rewarded. Set to 0 to disable allowing the animal to temporarily dip below running speed threshold.
- Default:
0.3
- -r, --restore_parameters
Determines whether to load and use the same training parameters as used during the previous lick training session of the target animal. Note, this only overrides the initial speed and duration thresholds, all other parameters are not affected by this flag.
- Default:
False
sl-experiment
Runs the requested experiment session for the specified animal and project combination.
Experiment runtimes are carried out after the lick and run training sessions Unlike training session commands, this command can be used to run different experiments. Each experiment runtime is configured via the user-defined configuration .yaml file, which should be stored inside the ‘configuration’ folder of the target project. The experiments are discovered by name, allowing a single project to have multiple different experiments. To create a new experiment configuration, use the ‘sl-create-experiment’ CLI command.
sl-experiment [OPTIONS]
Options
- -u, --user <user>
Required The ID of the user supervising the experiment session.
- -p, --project <project>
Required The name of the project to which the trained animal belongs.
- -e, --experiment <experiment>
Required The name of the experiment to carry out during runtime.
- -a, --animal <animal>
Required The name of the animal undergoing the experiment session.
- -w, --animal_weight <animal_weight>
Required The weight of the animal, in grams, at the beginning of the experiment session.
- -ur, --unconsumed_rewards <unconsumed_rewards>
The maximum number of rewards that can be delivered without the animal consuming them, before reward delivery is paused. Set to 0 to disable enforcing reward consumption.
- Default:
1
sl-check-window
Runs the cranial window and surgery quality checking session for the specified animal and project combination.
Before the animals are fully inducted (included) into a project, the quality of the surgical intervention (craniotomy and window implantation) is checked to ensure the animal will produce high-quality scientific data. As part of this process, various parameters of the Mesoscope-VR data acquisition system are also calibrated to best suit the animal. This command aggregates all steps necessary to verify and record the quality of the animal’s window and to generate customized Mesoscope-VR parameters for the animal.
sl-check-window [OPTIONS]
Options
- -u, --user <user>
Required The ID of the user supervising the experiment session.
- -p, --project <project>
Required The name of the project to which the trained animal belongs.
- -a, --animal <animal>
Required The name of the animal undergoing the experiment session.
sl-preprocess
Preprocesses the target session’s data.
This command aggregates all session data on the VRPC, compresses the data to optimize it for network transmission and storage, and transfers the data to the NAS and the BioHPC cluster. It automatically skips already completed processing stages as necessary to optimize runtime performance.
Primarily, this command is intended to retry or resume failed or interrupted preprocessing runtimes. Preprocessing should be carried out immediately after data acquisition to optimize the acquired data for long-term storage and distribute it to the NAS and the BioHPC cluster for further processing and storage.
sl-preprocess [OPTIONS]
Options
- -sp, --session-path <session_path>
Required The path to the session directory to preprocess.
sl-purge
Removes all redundant data directories for ALL projects from the ScanImagePC and the VRPC.
Redundant data purging is now executed automatically as part of data preprocessing. This command is primarily maintained as a fall-back option if automated data purging fails for any reason. Data purging should be carried out at least weekly to remove no longer necessary data from the PCs used during data acquisition.
sl-purge [OPTIONS]
sl-delete-session-data
Removes ALL data of the target session from ALL data acquisition and long-term storage machines accessible to the host-machine.
This is an EXTREMELY dangerous command that can potentially delete valuable data if not used well. This command is intended exclusively for removing failed and test sessions from all computers used in the Sun lab data acquisition process. Never call this command unless you know what you are doing.
sl-delete-session-data [OPTIONS]
Options
- -sp, --session-path <session_path>
Required The path to the local session directory of the session to be removed.
Mesoscope-VR Acquisition System
This package stores the data acquisition and preprocessing assets used by the Mesoscope-VR data acquisition system. It exposes the APIs that allow interfacing with the Mesoscope-VR system to acquire new experiment data, train animals, and preprocess already acquired data.
- class sl_experiment.mesoscope_vr.CRCCalculator
Bases:
objectA CRC32-XFER checksum calculator that works with raw bytes or pythonic strings.
This utility class exposes methods that generate CRC checksum labels and bytes objects, which are primarily used by Zaber binding classes to verify that Zaber devices have been configured to work with the binding interface exposed by this library.
- _calculator
Stores the configured Calculator class object used to calculate the checksums.
- bytes_checksum(data)
Calculates the CRC32-XFER checksum for the input bytes.
While the class is primarily designed for generating CRC-checksums for strings, this method can be used to checksum any bytes-converted object.
- Parameters:
data (
bytes) – The bytes-converted data to calculate the CRC checksum for.- Return type:
int- Returns:
The integer CRC32-XFER checksum.
- string_checksum(string)
Calculates the CRC32-XFER checksum for the input string.
The input strings are first converted to bytes using ASCII protocol. The checksum is then calculated on the resultant bytes-object.
- Parameters:
string (
str) – The string for which to calculate the CRC checksum.- Return type:
int- Returns:
The integer CRC32-XFER checksum.
- sl_experiment.mesoscope_vr.discover_zaber_devices(silence_errors=True)
Scans all available serial ports and displays information about connected Zaber devices.
- Parameters:
silence_errors (
bool, default:True) – Determines whether to display encountered errors. By default, when the discovery process runs into an error, it labels the error port as having no devices and suppresses the error. Enabling this flag will also print encountered error messages, which may be desirable for debugging purposes.- Return type:
None
- sl_experiment.mesoscope_vr.experiment_logic(experimenter, project_name, experiment_name, animal_id, animal_weight, maximum_unconsumed_rewards=1)
Encapsulates the logic used to run experiments via the Mesoscope-VR system.
This function can be used to execute any valid experiment using the Mesoscope-VR system. Each experiment should be broken into one or more experiment states (phases), such as ‘baseline’, ‘task’ and ‘cooldown’. Furthermore, each experiment state can use one or more Mesoscope-VR system states. Currently, the system has two experiment states: rest (1) and run (2). The states are used to broadly configure the Mesoscope-VR system, and they determine which components (modules) are active and what data is collected (see library ReadMe for more details on system states).
Primarily, this function is concerned with iterating over the states stored inside the experiment configuration file loaded using the ‘experiment_name’ argument value. Each experiment and Mesoscope-VR system state combination is maintained for the requested duration of seconds. Once all states have been executed, the experiment runtime ends. Under this design pattern, each experiment is conceptualized as a sequence of states.
Notes
During experiment runtimes, the task logic and the Virtual Reality world are resolved via the Unity game engine. This function itself does not resolve the task logic, it is only concerned with iterating over experiment states and controlling the Mesoscope-VR system.
- Parameters:
experimenter (
str) – The id of the experimenter conducting the experiment.project_name (
str) – The name of the project for which the experiment is conducted.experiment_name (
str) – The name or ID of the experiment to be conducted. Note, must match the name of the experiment configuration file stored under the ‘configuration’ project-specific directory.animal_id (
str) – The numeric ID of the animal participating in the experiment.animal_weight (
float) – The weight of the animal, in grams, at the beginning of the experiment session.maximum_unconsumed_rewards (
int, default:1) – The maximum number of rewards that can be delivered without the animal consuming them, before reward delivery (but not the experiment!) pauses until the animal consumes available rewards. If this is set to a value below 1, the unconsumed reward limit will not be enforced. A value of 1 means the animal has to consume each reward before getting the next reward.
- Return type:
None
- sl_experiment.mesoscope_vr.lick_training_logic(experimenter, project_name, animal_id, animal_weight, minimum_reward_delay=6, maximum_reward_delay=18, maximum_water_volume=1.0, maximum_training_time=20, maximum_unconsumed_rewards=1, load_previous_parameters=False)
Encapsulates the logic used to train animals to operate the lick port.
The lick training consists of delivering randomly spaced 5 uL water rewards via the solenoid valve to teach the animal that water comes out of the lick port. Each reward is delivered after a pseudorandom delay. Reward delay sequence is generated before training runtime by sampling a uniform distribution that ranges from ‘minimum_reward_delay’ to ‘maximum_reward_delay’. The training continues either until the valve delivers the ‘maximum_water_volume’ in milliliters or until the ‘maximum_training_time’ in minutes is reached, whichever comes first.
- Parameters:
experimenter (
str) – The ID (net-ID) of the experimenter conducting the training.project_name (
str) – The name of the project to which the trained animal belongs.animal_id (
str) – The numeric ID of the animal being trained.animal_weight (
float) – The weight of the animal, in grams, at the beginning of the training session.minimum_reward_delay (
int, default:6) – The minimum time, in seconds, that has to pass between delivering two consecutive rewards.maximum_reward_delay (
int, default:18) – The maximum time, in seconds, that can pass between delivering two consecutive rewards.maximum_water_volume (
float, default:1.0) – The maximum volume of water, in milliliters, that can be delivered during this runtime.maximum_training_time (
int, default:20) – The maximum time, in minutes, to run the training.maximum_unconsumed_rewards (
int, default:1) – The maximum number of rewards that can be delivered without the animal consuming them, before reward delivery (but not the training!) pauses until the animal consumes available rewards. If this is set to a value below 1, the unconsumed reward limit will not be enforced. A value of 1 means the animal has to consume each reward before getting the next reward.load_previous_parameters (
bool, default:False) – Determines whether to override all input runtime-defining parameters with the parameters used during the previous session. If this is set to True, the function will ignore most input parameters and will instead load them from the cached session descriptor of the previous session. If the descriptor is not available, the function will fall back to using input parameters.
- Return type:
None
- sl_experiment.mesoscope_vr.maintenance_logic()
Encapsulates the logic used to maintain various components of the Mesoscope-VR system.
This runtime is primarily used to verify and, if necessary, recalibrate the water valve between training or experiment days and to maintain the surface material of the running wheel.
- Return type:
None
- sl_experiment.mesoscope_vr.preprocess_session_data(session_data)
Aggregates all data on VRPC, compresses it for efficient network transmission, safely transfers the data to the BioHPC server and the Synology NAS for long-term storage, and removes all local data copies.
This method should be called at the end of each training and experiment runtime to preprocess the data. Primarily, it prepares the data for further processing, moves it to appropriate long-term storage destinations, and keeps the VRPC and ScanImagePC filesystem free from clutter by removing redundant local data copies.
- Parameters:
session_data (
SessionData) – The SessionData instance for the processed session.- Return type:
None
- sl_experiment.mesoscope_vr.purge_failed_session(session_data)
Removes all data and directories associated with the input session.
This function is extremely dangerous and should be used with caution. It is designed to remove all data from failed or no longer necessary sessions. Never use this function on sessions that contain valid scientific data.
- Parameters:
session_data (
SessionData) – The SessionData instance for the session whose data needs to be removed.- Return type:
None
- sl_experiment.mesoscope_vr.purge_redundant_data()
Loops over ScanImagePC and VRPC directories that store training and experiment data and removes no longer necessary data caches.
This function searches the ScanImagePC and VRPC for no longer necessary directories and removes them from the respective systems. ScanImagePC directories are marked for deletion once they are safely copied to the VRPC. VRPC directories are marked for deletion once the data is safely copied to the BioHPC server. Both copying steps include verifying the integrity of the transferred data using xxHash-128 checksums.
- Return type:
None
Notes
This is a service function intended to maintain the ScanImagePC and VRPC disk space. Once the data is moved to the BioHPC server and the NAS, it is generally safe to remove the copies stored on the ScanImagePC and VRPC.
Currently, this function does not discriminate between projects or animals. It will remove all data marked for deletion via the ubiquitin.bin markers.
- sl_experiment.mesoscope_vr.run_training_logic(experimenter, project_name, animal_id, animal_weight, initial_speed_threshold=0.5, initial_duration_threshold=0.5, speed_increase_step=0.05, duration_increase_step=0.05, increase_threshold=0.1, maximum_water_volume=1.0, maximum_training_time=40, maximum_idle_time=0.5, maximum_unconsumed_rewards=1, load_previous_parameters=False)
Encapsulates the logic used to train animals to run on the wheel treadmill while being head-fixed.
The run training consists of making the animal run on the wheel with a desired speed, in centimeters per second, maintained for the desired duration of time, in seconds. Each time the animal satisfies the speed and duration thresholds, it receives 5 uL of water reward, and the speed and duration trackers reset for the next training ‘epoch’. Each time the animal receives ‘increase_threshold’ of water, the speed and duration thresholds increase to make the task progressively more challenging. The training continues either until the training time exceeds the ‘maximum_training_time’, or the animal receives the ‘maximum_water_volume’ of water, whichever happens earlier.
- Parameters:
experimenter (
str) – The id of the experimenter conducting the training.project_name (
str) – The name of the project to which the trained animal belongs.animal_id (
str) – The numeric ID of the animal being trained.animal_weight (
float) – The weight of the animal, in grams, at the beginning of the training session.initial_speed_threshold (
float, default:0.5) – The initial running speed threshold, in centimeters per second, that the animal must maintain to receive water rewards.initial_duration_threshold (
float, default:0.5) – The initial duration threshold, in seconds, that the animal must maintain above-threshold running speed to receive water rewards.speed_increase_step (
float, default:0.05) – The step size, in centimeters per second, by which to increase the speed threshold each time the animal receives ‘increase_threshold’ milliliters of water.duration_increase_step (
float, default:0.05) – The step size, in seconds, by which to increase the duration threshold each time the animal receives ‘increase_threshold’ milliliters of water.increase_threshold (
float, default:0.1) – The volume of water received by the animal, in milliliters, after which the speed and duration thresholds are increased by one step. Note, the animal will at most get ‘maximum_water_volume’ of water, so this parameter effectively controls how many increases will be made during runtime, assuming the maximum training time is not reached.maximum_water_volume (
float, default:1.0) – The maximum volume of water, in milliliters, that can be delivered during this runtime.maximum_training_time (
int, default:40) – The maximum time, in minutes, to run the training.maximum_idle_time (
float, default:0.5) – The maximum time, in seconds, the animal’s speed can be below the speed threshold to still receive water rewards. This parameter is designed to help animals with a distinct ‘step’ pattern to not lose water rewards due to taking many large steps, rather than continuously running at a stable speed. This parameter allows the speed to dip below the threshold for at most this number of seconds, for the ‘running epoch’ to not be interrupted.maximum_unconsumed_rewards (
int, default:1) – The maximum number of rewards that can be delivered without the animal consuming them, before reward delivery (but not the training!) pauses until the animal consumes available rewards. If this is set to a value below 1, the unconsumed reward limit will not be enforced. A value of 1 means the animal has to consume all rewards before getting the next reward.load_previous_parameters (
bool, default:False) – Determines whether to override all input runtime-defining parameters with the parameters used during the previous session. If this is set to True, the function will ignore most input parameters and will instead load them from the cached session descriptor of the previous session. If the descriptor is not available, the function will fall back to using input parameters.
- Return type:
None
- sl_experiment.mesoscope_vr.window_checking_logic(experimenter, project_name, animal_id)
Encapsulates the logic used to verify the surgery quality (cranial window) and generate the initial snapshot of the Mesoscope-VR system configuration for a newly added animal of the target project.
This function is used when new animals are added to the project, before any other training or experiment runtime. Primarily, it is used to verify that the surgery went as expected and the animal is fit for providing high-quality scientific data. As part of this process, the function also generates the snapshot of zaber motor positions, the mesoscope objective position, and the red-dot alignment screenshot to be reused by future sessions.
Notes
This function largely behaves similar to all other training and experiment session runtimes. However, it does not use most of the Mesoscope-VR components and does not make most of the runtime data files typically generated by other sessions. All window checking sessions are automatically marked as ‘incomplete’ and excluded from automated data processing.
- Parameters:
experimenter (
str) – The id of the experimenter conducting the window checking session.project_name (
str) – The name of the project to which the checked animal belongs.animal_id (
str) – The numeric ID of the animal whose cranial window is being checked.
- Return type:
None