moseq2_extract package

CLI Module

moseq2-extract

moseq2-extract [OPTIONS] COMMAND [ARGS]...

Options

--version

Show the version and exit.

Default:

False

agg-to-index

Generate an index file from aggregated results with default as group names

moseq2-extract agg-to-index [OPTIONS]

Options

-i, --input-dir <input_dir>

Directory for aggregated results folder

Default:

/home/wingillis/dev/moseq/moseq2-extract/docs/aggregate_results

aggregate-results

Copies all extracted results (h5, yaml, mp4) files from all extracted sessions to a new directory for modeling and analysis

moseq2-extract aggregate-results [OPTIONS]

Options

-i, --input-dir <input_dir>

Directory to find h5 files

Default:

/home/wingillis/dev/moseq/moseq2-extract/docs

-f, --format <format>

New file name formats from resepective metadata

Default:

{start_time}_{session_name}_{subject_name}

-o, --output-dir <output_dir>

Location for storing all results together

Default:

/home/wingillis/dev/moseq/moseq2-extract/docs/aggregate_results/

--mouse-threshold <mouse_threshold>

Threshold value for mean depth to include frames in aggregated results

Default:

0

batch-extract

Batch processes all the raw depth recordings located in the input folder.

moseq2-extract batch-extract [OPTIONS] INPUT_FOLDER

Options

-p, --progress-bar

Show verbose progress bars.

Default:

False

--config-file <config_file>
--recompute-bg <recompute_bg>

Overwrite previously computed background image

Default:

False

--use-plane-bground

Use a plane fit for the background. Useful when mice don’t move much

Default:

False

--output-dir <output_dir>

Output directory to save the results h5 file

Default:

proc

--noise-tolerance <noise_tolerance>

Extent of noise to accept during RANSAC Plane ROI computation.

Default:

30

--erode-iterations <erode_iterations>

Number of erosion iterations to decrease bucket floor size.

Default:

0

--bg-roi-erode <bg_roi_erode>

Size of cv2 Structure Element to erode roi.

Default:

1, 1

--dilate-iterations <dilate_iterations>

Number of dilation iterations to increase bucket floor size.

Default:

1

--bg-sort-roi-by-position-max-rois <bg_sort_roi_by_position_max_rois>

The number of maximum ROIs sorted by area

Default:

2

--bg-sort-roi-by-position <bg_sort_roi_by_position>

Sort ROIs by position

Default:

False

--bg-roi-fill-holes <bg_roi_fill_holes>

Fill holes in ROI

Default:

True

--bg-roi-gradient-kernel <bg_roi_gradient_kernel>

Kernel size for Sobel gradient filtering

Default:

7

--bg-roi-gradient-threshold <bg_roi_gradient_threshold>

Gradient must be less than threshold to include points

Default:

3000

--bg-roi-gradient-filter <bg_roi_gradient_filter>

Use graident filter to exclude walls for detected ROI

Default:

False

--bg-roi-depth-range <bg_roi_depth_range>

Range to search for floor of arena (in mm)

Default:

650, 750

--manual-set-depth-range

Flag to deactivate auto depth range setting.

Default:

False

--camera-type <camera_type>

Camera type used for recording for auto-sets bg-roi-weights to precomputed values for different camera types. Possible types: [“kinect”, “azure”, “realsense”]

Default:

auto

Options:

auto | kinect | azure | realsense

--bg-roi-weights <bg_roi_weights>

ROI feature weighting (area, extent, dist to center)

Default:

1, 0.1, 1

--bg-roi-index <bg_roi_index>

Index of which detected ROI mask to use

Default:

0

--bg-roi-shape <bg_roi_shape>

Shape to use to detect roi (ellipse or rect)

Default:

ellipse

--bg-roi-dilate <bg_roi_dilate>

Size of StructuringElement to dilate roi

Default:

10, 10

-m, --mapping <mapping>

Ffprobe stream selection variable. Default: DEPTH

Default:

DEPTH

-t, --threads <threads>

Number of threads for encoding

Default:

8

--delete

Delete raw file if encoding is sucessful

Default:

False

--fps <fps>

Video FPS

Default:

30

-b, --chunk-size <chunk_size>

Chunk size

Default:

3000

-o, --output-file <output_file>

Path to output file

--skip-completed

Will skip the extraction if it is already completed.

Default:

False

--compress-threads <compress_threads>

Number of threads for encoding

Default:

3

--compress-chunk-size <compress_chunk_size>

Chunk size for .avi compression

Default:

3000

--compress <compress>

Convert .dat to .avi after successful extraction

Default:

False

--frame-trim <frame_trim>

Frames to trim from beginning and end of data

Default:

0, 0

--model-smoothing-clips <model_smoothing_clips>

Model smoothing clips

Default:

0, 0

--angle-hampel-sig <angle_hampel_sig>

Angle filter sig

Default:

3

--angle-hampel-span <angle_hampel_span>

Angle filter span

Default:

0

--centroid-hampel-sig <centroid_hampel_sig>

Hampel filter sig

Default:

3

--centroid-hampel-span <centroid_hampel_span>

Hampel filter span

Default:

0

--pixel-format <pixel_format>

Pixel format for reading in .avi and .mkv videos

Default:

gray16le

--movie-dtype <movie_dtype>

Data type for raw frames read in for extraction

Default:

<i2

--frame-dtype <frame_dtype>

Data type for processed frames

Default:

uint8

Options:

uint8 | uint16

--write-movie <write_movie>

Write a results output movie including an extracted mouse

Default:

True

--chunk-overlap <chunk_overlap>

Frames overlapped in each chunk. Useful for cable tracking

Default:

0

-t, --temporal-filter-size <temporal_filter_size>

Time prefilter kernel (median filter, must be odd)

Default:

0

-s, --spatial-filter-size <spatial_filter_size>

Space prefilter kernel (median filter, must be odd)

Default:

3

--tail-filter-shape <tail_filter_shape>

Tail filter shape

Default:

ellipse

--tail-filter-size <tail_filter_size>

Tail filter size

Default:

9, 9

--tail-filter-iters <tail_filter_iters>

Number of tail filter iterations

Default:

1

--cable-filter-size <cable_filter_size>

Cable filter size (in pixels)

Default:

5, 5

--cable-filter-shape <cable_filter_shape>

Cable filter shape (rectangle or ellipse)

Default:

rectangle

--cable-filter-iters <cable_filter_iters>

Number of cable filter iterations

Default:

0

--tracking-model-init <tracking_model_init>

Method for tracking model initialization

Default:

raw

--tracking-model-segment <tracking_model_segment>

Segment likelihood mask from tracking model

Default:

True

--tracking-model-ll-clip <tracking_model_ll_clip>

Clip log-likelihoods below this value

Default:

-100

--tracking-model-mask-threshold <tracking_model_mask_threshold>

Threshold on log-likelihood to include pixels for centroid and angle calculation

Default:

-16

--tracking-model-ll-threshold <tracking_model_ll_threshold>

Threshold on log-likelihood for pixels to use for update during tracking

Default:

-100

--use-tracking-model <use_tracking_model>

Use an expectation-maximization style model to aid mouse tracking. Useful for data with cables

Default:

False

--use-cc <use_cc>

Extract features using largest connected components.

Default:

True

--widen-radius <widen_radius>

Number of pixels to increase/decrease radius by when graduating bucket walls.

Default:

0

--graduate-walls <graduate_walls>

Graduates and dilates the background image to compensate for slanted bucket walls. _/

Default:

False

--flip-classifier-smoothing <flip_classifier_smoothing>

Number of frames to smooth flip classifier probabilities

Default:

51

--flip-classifier <flip_classifier>

path to the flip classifier used to properly orient the mouse (.pkl file)

--compute-raw-scalars

Compute scalar values from raw cropped frames.

Default:

False

--detected-true-depth <detected_true_depth>

Option to override automatic depth estimation during extraction. This is only a debugging parameter, for cases where dilate_iterations > 1, otherwise has no effect. Either “auto” or an int value.

Default:

auto

--max-height <max_height>

Max mouse height threshold from floor (mm)

Default:

120

--min-height <min_height>

Min mouse height threshold from floor (mm)

Default:

10

-n, --num-frames <num_frames>

Number of frames to extract. Will extract full session if set to None.

-c, --crop-size <crop_size>

Width and height of cropped mouse image

Default:

80, 80

--extensions <extensions>

File extension of raw data

Default:

.dat

--skip-checks

Flag: skip checks for the existance of a metadata file

Default:

False

--extract-out-script <extract_out_script>

Name of bash script file to save extract commands.

Default:

extract_out.sh

--cluster-type <cluster_type>

Platform to train models on

Default:

local

Options:

local | slurm

--prefix <prefix>

Batch command string to prefix model training command (slurm only).

Default:

-c, --ncpus <ncpus>

Number of cores to use in extraction

Default:

1

--memory <memory>

RAM (slurm only)

Default:

5GB

--wall-time <wall_time>

Wall time (slurm only)

Default:

3:00:00

--partition <partition>

Partition name (slurm only)

Default:

short

--get-cmd

Print scan command strings.

Default:

True

--run-cmd

Run scan command strings.

Default:

False

Arguments

INPUT_FOLDER

Required argument

convert-raw-to-avi

Loss less compresses a raw depth file (dat) into an avi file that is 8x smaller.

moseq2-extract convert-raw-to-avi [OPTIONS] INPUT_FILE

Options

-m, --mapping <mapping>

Ffprobe stream selection variable. Default: DEPTH

Default:

DEPTH

-t, --threads <threads>

Number of threads for encoding

Default:

8

--delete

Delete raw file if encoding is sucessful

Default:

False

--fps <fps>

Video FPS

Default:

30

-b, --chunk-size <chunk_size>

Chunk size

Default:

3000

-o, --output-file <output_file>

Path to output file

Arguments

INPUT_FILE

Required argument

copy-slice

Copies a segment of an input depth recording into a new video file.

moseq2-extract copy-slice [OPTIONS] INPUT_FILE

Options

-m, --mapping <mapping>

Ffprobe stream selection variable. Default: DEPTH

Default:

DEPTH

-t, --threads <threads>

Number of threads for encoding

Default:

8

--delete

Delete raw file if encoding is sucessful

Default:

False

--fps <fps>

Video FPS

Default:

30

-b, --chunk-size <chunk_size>

Chunk size

Default:

3000

-o, --output-file <output_file>

Path to output file

-c, --copy-slice <copy_slice>

Slice indices used for copy

Default:

0, 1000

Arguments

INPUT_FILE

Required argument

download-flip-file

Downloads Flip-correction model that helps with orienting the mouse during extraction.

moseq2-extract download-flip-file [OPTIONS] [CONFIG_FILE]

Options

--output-dir <output_dir>

Output directory for downloaded flip flie

Default:

/home/wingillis/dev/moseq/moseq2-extract/docs

Arguments

CONFIG_FILE

Optional argument

extract

Processes raw input depth recordings to output a cropped and orientedvideo of the mouse and saves the output+metadata to h5 files in the given output directory.

moseq2-extract extract [OPTIONS] INPUT_FILE

Options

--cluster-type <cluster_type>

Platform to train models on

Default:

local

Options:

local | slurm

-p, --progress-bar

Show verbose progress bars.

Default:

False

--config-file <config_file>
--recompute-bg <recompute_bg>

Overwrite previously computed background image

Default:

False

--use-plane-bground

Use a plane fit for the background. Useful when mice don’t move much

Default:

False

--output-dir <output_dir>

Output directory to save the results h5 file

Default:

proc

--noise-tolerance <noise_tolerance>

Extent of noise to accept during RANSAC Plane ROI computation.

Default:

30

--erode-iterations <erode_iterations>

Number of erosion iterations to decrease bucket floor size.

Default:

0

--bg-roi-erode <bg_roi_erode>

Size of cv2 Structure Element to erode roi.

Default:

1, 1

--dilate-iterations <dilate_iterations>

Number of dilation iterations to increase bucket floor size.

Default:

1

--bg-sort-roi-by-position-max-rois <bg_sort_roi_by_position_max_rois>

The number of maximum ROIs sorted by area

Default:

2

--bg-sort-roi-by-position <bg_sort_roi_by_position>

Sort ROIs by position

Default:

False

--bg-roi-fill-holes <bg_roi_fill_holes>

Fill holes in ROI

Default:

True

--bg-roi-gradient-kernel <bg_roi_gradient_kernel>

Kernel size for Sobel gradient filtering

Default:

7

--bg-roi-gradient-threshold <bg_roi_gradient_threshold>

Gradient must be less than threshold to include points

Default:

3000

--bg-roi-gradient-filter <bg_roi_gradient_filter>

Use graident filter to exclude walls for detected ROI

Default:

False

--bg-roi-depth-range <bg_roi_depth_range>

Range to search for floor of arena (in mm)

Default:

650, 750

--manual-set-depth-range

Flag to deactivate auto depth range setting.

Default:

False

--camera-type <camera_type>

Camera type used for recording for auto-sets bg-roi-weights to precomputed values for different camera types. Possible types: [“kinect”, “azure”, “realsense”]

Default:

auto

Options:

auto | kinect | azure | realsense

--bg-roi-weights <bg_roi_weights>

ROI feature weighting (area, extent, dist to center)

Default:

1, 0.1, 1

--bg-roi-index <bg_roi_index>

Index of which detected ROI mask to use

Default:

0

--bg-roi-shape <bg_roi_shape>

Shape to use to detect roi (ellipse or rect)

Default:

ellipse

--bg-roi-dilate <bg_roi_dilate>

Size of StructuringElement to dilate roi

Default:

10, 10

-m, --mapping <mapping>

Ffprobe stream selection variable. Default: DEPTH

Default:

DEPTH

-t, --threads <threads>

Number of threads for encoding

Default:

8

--delete

Delete raw file if encoding is sucessful

Default:

False

--fps <fps>

Video FPS

Default:

30

-b, --chunk-size <chunk_size>

Chunk size

Default:

3000

-o, --output-file <output_file>

Path to output file

--skip-completed

Will skip the extraction if it is already completed.

Default:

False

--compress-threads <compress_threads>

Number of threads for encoding

Default:

3

--compress-chunk-size <compress_chunk_size>

Chunk size for .avi compression

Default:

3000

--compress <compress>

Convert .dat to .avi after successful extraction

Default:

False

--frame-trim <frame_trim>

Frames to trim from beginning and end of data

Default:

0, 0

--model-smoothing-clips <model_smoothing_clips>

Model smoothing clips

Default:

0, 0

--angle-hampel-sig <angle_hampel_sig>

Angle filter sig

Default:

3

--angle-hampel-span <angle_hampel_span>

Angle filter span

Default:

0

--centroid-hampel-sig <centroid_hampel_sig>

Hampel filter sig

Default:

3

--centroid-hampel-span <centroid_hampel_span>

Hampel filter span

Default:

0

--pixel-format <pixel_format>

Pixel format for reading in .avi and .mkv videos

Default:

gray16le

--movie-dtype <movie_dtype>

Data type for raw frames read in for extraction

Default:

<i2

--frame-dtype <frame_dtype>

Data type for processed frames

Default:

uint8

Options:

uint8 | uint16

--write-movie <write_movie>

Write a results output movie including an extracted mouse

Default:

True

--chunk-overlap <chunk_overlap>

Frames overlapped in each chunk. Useful for cable tracking

Default:

0

-t, --temporal-filter-size <temporal_filter_size>

Time prefilter kernel (median filter, must be odd)

Default:

0

-s, --spatial-filter-size <spatial_filter_size>

Space prefilter kernel (median filter, must be odd)

Default:

3

--tail-filter-shape <tail_filter_shape>

Tail filter shape

Default:

ellipse

--tail-filter-size <tail_filter_size>

Tail filter size

Default:

9, 9

--tail-filter-iters <tail_filter_iters>

Number of tail filter iterations

Default:

1

--cable-filter-size <cable_filter_size>

Cable filter size (in pixels)

Default:

5, 5

--cable-filter-shape <cable_filter_shape>

Cable filter shape (rectangle or ellipse)

Default:

rectangle

--cable-filter-iters <cable_filter_iters>

Number of cable filter iterations

Default:

0

--tracking-model-init <tracking_model_init>

Method for tracking model initialization

Default:

raw

--tracking-model-segment <tracking_model_segment>

Segment likelihood mask from tracking model

Default:

True

--tracking-model-ll-clip <tracking_model_ll_clip>

Clip log-likelihoods below this value

Default:

-100

--tracking-model-mask-threshold <tracking_model_mask_threshold>

Threshold on log-likelihood to include pixels for centroid and angle calculation

Default:

-16

--tracking-model-ll-threshold <tracking_model_ll_threshold>

Threshold on log-likelihood for pixels to use for update during tracking

Default:

-100

--use-tracking-model <use_tracking_model>

Use an expectation-maximization style model to aid mouse tracking. Useful for data with cables

Default:

False

--use-cc <use_cc>

Extract features using largest connected components.

Default:

True

--widen-radius <widen_radius>

Number of pixels to increase/decrease radius by when graduating bucket walls.

Default:

0

--graduate-walls <graduate_walls>

Graduates and dilates the background image to compensate for slanted bucket walls. _/

Default:

False

--flip-classifier-smoothing <flip_classifier_smoothing>

Number of frames to smooth flip classifier probabilities

Default:

51

--flip-classifier <flip_classifier>

path to the flip classifier used to properly orient the mouse (.pkl file)

--compute-raw-scalars

Compute scalar values from raw cropped frames.

Default:

False

--detected-true-depth <detected_true_depth>

Option to override automatic depth estimation during extraction. This is only a debugging parameter, for cases where dilate_iterations > 1, otherwise has no effect. Either “auto” or an int value.

Default:

auto

--max-height <max_height>

Max mouse height threshold from floor (mm)

Default:

120

--min-height <min_height>

Min mouse height threshold from floor (mm)

Default:

10

-n, --num-frames <num_frames>

Number of frames to extract. Will extract full session if set to None.

-c, --crop-size <crop_size>

Width and height of cropped mouse image

Default:

80, 80

Arguments

INPUT_FILE

Required argument

find-roi

Finds the ROI (the arena) and background to subtract from frames when extracting.

moseq2-extract find-roi [OPTIONS] INPUT_FILE

Options

-p, --progress-bar

Show verbose progress bars.

Default:

False

--config-file <config_file>
--recompute-bg <recompute_bg>

Overwrite previously computed background image

Default:

False

--use-plane-bground

Use a plane fit for the background. Useful when mice don’t move much

Default:

False

--output-dir <output_dir>

Output directory to save the results h5 file

Default:

proc

--noise-tolerance <noise_tolerance>

Extent of noise to accept during RANSAC Plane ROI computation.

Default:

30

--erode-iterations <erode_iterations>

Number of erosion iterations to decrease bucket floor size.

Default:

0

--bg-roi-erode <bg_roi_erode>

Size of cv2 Structure Element to erode roi.

Default:

1, 1

--dilate-iterations <dilate_iterations>

Number of dilation iterations to increase bucket floor size.

Default:

1

--bg-sort-roi-by-position-max-rois <bg_sort_roi_by_position_max_rois>

The number of maximum ROIs sorted by area

Default:

2

--bg-sort-roi-by-position <bg_sort_roi_by_position>

Sort ROIs by position

Default:

False

--bg-roi-fill-holes <bg_roi_fill_holes>

Fill holes in ROI

Default:

True

--bg-roi-gradient-kernel <bg_roi_gradient_kernel>

Kernel size for Sobel gradient filtering

Default:

7

--bg-roi-gradient-threshold <bg_roi_gradient_threshold>

Gradient must be less than threshold to include points

Default:

3000

--bg-roi-gradient-filter <bg_roi_gradient_filter>

Use graident filter to exclude walls for detected ROI

Default:

False

--bg-roi-depth-range <bg_roi_depth_range>

Range to search for floor of arena (in mm)

Default:

650, 750

--manual-set-depth-range

Flag to deactivate auto depth range setting.

Default:

False

--camera-type <camera_type>

Camera type used for recording for auto-sets bg-roi-weights to precomputed values for different camera types. Possible types: [“kinect”, “azure”, “realsense”]

Default:

auto

Options:

auto | kinect | azure | realsense

--bg-roi-weights <bg_roi_weights>

ROI feature weighting (area, extent, dist to center)

Default:

1, 0.1, 1

--bg-roi-index <bg_roi_index>

Index of which detected ROI mask to use

Default:

0

--bg-roi-shape <bg_roi_shape>

Shape to use to detect roi (ellipse or rect)

Default:

ellipse

--bg-roi-dilate <bg_roi_dilate>

Size of StructuringElement to dilate roi

Default:

10, 10

Arguments

INPUT_FILE

Required argument

generate-config

Generates a configuration file (config.yaml) that holds editable options for extraction parameters.

moseq2-extract generate-config [OPTIONS]

Options

-o, --output-file <output_file>
Default:

config.yaml

--camera-type <camera_type>

specify the camera type (k2 or azure), default is k2

Default:

k2

generate-index

Generates an index file (moseq2-index.yaml) that contains all extracted session metadata.

moseq2-extract generate-index [OPTIONS]

Options

-i, --input-dir <input_dir>

Directory to find h5 files

Default:

/home/wingillis/dev/moseq/moseq2-extract/docs

-o, --output-file <output_file>

Location for storing index

Default:

/home/wingillis/dev/moseq/moseq2-extract/docs/moseq2-index.yaml

GUI Module

GUI front-end operations accessible from a jupyter notebook.

moseq2_extract.gui.download_flip_command(output_dir, config_file='', selection=1)

Download flip classifier and saves its path to config file (config.yaml)

Args: output_dir (str): path to output directory to save flip classifier config_file (str): path to config file (config.yaml) selection (int): index of which flip file to download (default is Adult male C57 classifer)

Returns:

moseq2_extract.gui.generate_index_command(input_dir, output_file)

Generate Index File (moseq2-index.yaml) based on aggregated sessions

Args: input_dir (str): path to folder with aggregated results output_file (str): path to index file

Returns: output_file (str): path to index file.

moseq2_extract.gui.get_selected_sessions(to_extract, extract_all)

Return either selected sessions to extract, or all the sessions given user input, the function will

Args: to_extract (list): list of paths to sessions to extract extract_all (bool): boolean to include all sessions and skip user-input prompt.

Returns: to_extract (list): new list of selected sessions to extract.

General Utilities Module

General utility functions throughout the extract package.

moseq2_extract.util.build_path(keys: dict, format_string: str, snake_case=True) → str

Produce a new file name using keys collected from extraction h5 files.

Args: keys (dict): dictionary specifying which keys used to produce the new file name format_string (str): the string to reformat using the keys dictionary i.e. ‘{subject_name}_{session_name}’. snake_case (bool): flag to save the files with snake_case

Returns: out (str): a newly formatted filename useable with any operating system

moseq2_extract.util.camel_to_snake(s)

Convert CamelCase to snake_case

Args: s (str): CamelCase string to convert to snake_case.

Returns: (str): string in snake_case

moseq2_extract.util.check_filter_sizes(config_data)

Ensure spatial and temporal filter kernel sizes are odd numbers.

Args: config_data (dict): a dictionary holding all extraction parameters

Returns: config_data (dict): Updated configuration dict

moseq2_extract.util.clean_dict(dct)

Standardize types of dict value.

Args: dct (dict): dict object with mixed type value objects.

Returns: out (dict): dict object with list value objects.

moseq2_extract.util.clean_file_str(file_str: str, replace_with: str = '-') → str

Removes invalid characters for a file name from a string.

Args: file_str (str): filename substring to replace replace_with (str): value to replace str with

Returns: out (str): cleaned file string

moseq2_extract.util.click_param_annot(click_cmd)

Return a dict that maps option names to help strings from a click.Command instance.

Args: click_cmd (click.Command): command to annotate

Returns: annotations (dict): dictionary of options and their help messages

moseq2_extract.util.command_with_config(config_file_param_name)

Override default CLI variables with the values contained within the config.yaml being passed.

Args: config_file_param_name (str): path to config file.

Returns: custom_command_class (function): decorator function to update click.Command parameters with the config_file parameter values.

moseq2_extract.util.convert_pxs_to_mm(coords, resolution=(512, 424), field_of_view=(70.6, 60), true_depth=673.1)

Converts x, y coordinates in pixel space to mm.

Args: coords (list): list of x,y pixel coordinates resolution (tuple): image dimensions field_of_view (tuple): width and height scaling params true_depth (float): detected true depth

Returns: new_coords (list): x,y coordinates in mm

moseq2_extract.util.convert_raw_to_avi_function(input_file, chunk_size=2000, fps=30, delete=False, threads=3)

Compress depth file (.dat, ‘.mkv’) to avi file.

Args: input_file (str): path to depth file chunk_size (int): size of chunks to process at a time fps (int): frames per second delete (bool): flag for deleting original depth file threads (int): number of threads to write video.

moseq2_extract.util.detect_and_set_camera_parameters(config_data, input_file=None)

Read the camera type and info and set the bg_roi_weights to the precomputed values. If camera_type is None, function will assume kinect is used.

Args: config_data (dict): dictionary containing all input parameters. input_file (str): path to raw depth file

Returns: config_data (dict): updated dictionary with bg-roi-weights to use for extraction.

moseq2_extract.util.detect_avi_file(finfo)
Detect the camera type by comparing the read video resolution with known

outputted dimensions of different camera types.

Args: finfo (dict): dictionary containing the file metadata,

Returns: detected (str): name of the detected camera type.

moseq2_extract.util.dict_to_h5(h5, dic, root='/', annotations=None)

Save an dict to an h5 file, mounting at root. Keys are mapped to group names recursively.

Args: h5 (h5py.File instance): h5py.file object to operate on dic (dict): dictionary of data to write root (string): group on which to add additional groups and datasets annotations (dict): annotation data to add to corresponding h5 datasets. Should contain same keys as dic.

moseq2_extract.util.escape_path(path)

Return a path to return to original base directory.

Args: path (str): path to current working dir

Returns: path (str): path to original base_dir

moseq2_extract.util.filter_warnings(func)
Applies warnings.simplefilter() to ignore warnings when

running the main gui functionaity in a Jupyter Notebook. The function will filter out: yaml.error.UnsafeLoaderWarning, FutureWarning and UserWarning.

Args: func (function): function to silence enclosed warnings.

Returns: apply_warning_filters (func): Returns passed function after warnings filtering is completed.

moseq2_extract.util.gen_batch_sequence(nframes, chunk_size, overlap, offset=0)

Generates batches used to chunk videos prior to extraction.

Args: nframes (int): total number of frames chunk_size (int): the number of desired chunk size overlap (int): number of overlapping frames offset (int): frame offset

Returns: out (list): the list of batches

moseq2_extract.util.generate_missing_metadata(sess_dir, sess_name)

Generate metadata.json with default avlues for session that does not already include one.

Args: sess_dir (str): Path to session directory to create metadata.json file in. sess_name (str): Session Name to set the metadata SessionName.

Returns:

moseq2_extract.util.get_bucket_center(img, true_depth, threshold=650)

Find Centroid coordinates of circular bucket.

Args: img (np.ndaarray): original background image. true_depth (float): distance value from camera to bucket floor (automatically pre-computed) threshold (float): distance values to accept region into detected circle. (used to reduce fall noise interference)

Returns: cX (int): x-coordinate of circle centroid cY (int): y-coordinate of circle centroid

moseq2_extract.util.get_frame_range_indices(trim_beginning, trim_ending, nframes)

Compute the total number of frames to be extracted, and find the start and end indices.

Args: trim_beginning (int): number of frames to remove from beginning of recording trim_ending (int): number of frames to remove from ending of recording nframes (int): total number of requested frames to extract

Returns: nframes (int): total number of frames to extract first_frame_idx (int): index of the frame to begin extraction from last_frame_idx (int): index of the last frame in the extraction

moseq2_extract.util.get_strels(config_data)

Get dictionary object of cv2 StructuringElements for image filtering given a dict of configurations parameters.

Args: config_data (dict): dict containing cv2 Structuring Element parameters

Returns: str_els (dict): dict containing cv2 StructuringElements used for image filtering

moseq2_extract.util.graduate_dilated_wall_area(bground_im, config_data, strel_dilate, output_dir)

Creates a gradient to represent the dilated (now visible) bucket wall regions. Only is used if background is dilated to capture larger rodents in convex shaped buckets (_/).

Args: bground_im (np.ndarray): the computed background image. config_data (dict): dictionary containing helper user configuration parameters. strel_dilate (cv2.structuringElement): dilation structuring element used to dilate background image. output_dir (str): path to save newly computed background to use.

Returns: bground_im (np.ndarray): the new background image with a gradient around the floor from high to low depth values.

moseq2_extract.util.h5_to_dict(h5file, path) → dict

Load h5 contents to dictionary object.

Args: h5file (str or h5py.File): file path to the given h5 file or the h5 file handle path (str): path to the base dataset within the h5 file

Returns: out (dict): a dict with h5 file contents with the same path structure

moseq2_extract.util.load_found_session_paths(input_dir, exts)

Find all depth files with the specified extension recursively in input directory.

Args: input_dir (str): path to project base directory holding all the session sub-folders. exts (list or str): list of extensions to search for, or a single extension in string form.

Returns: files (list): sorted list of all paths to found depth files

moseq2_extract.util.load_metadata(metadata_file)

Load metadata from session metadata.json file.

Args: metadata_file (str): path to metadata file

Returns: metadata (dict): metadata dictionary of JSON contents

moseq2_extract.util.load_textdata(data_file, dtype=<class 'numpy.float32'>)

Loads timestamp from txt/csv file.

Args: data_file (str): path to timestamp file dtype (dtype): data type of timestamps

Returns: data (np.ndarray): timestamp data timestamps (numpy.array): the array for the timestamps

moseq2_extract.util.load_timestamps(timestamp_file, col=0, alternate=False)

Read timestamps from space delimited text file for timestamps.

Args: timestamp_file (str): path to timestamp file col (int): column in ts file read. alternate (boolean): specified if timestamps were saved in a csv file. False means txt file and True means csv file.

Returns: ts (1D array): list of timestamps

moseq2_extract.util.make_gradient(width, height, h, k, a, b, theta=0)

Create gradient around bucket floor representing slanted wall values.

Args: width (int): bounding box width height (int) bounding box height h (int): centroid x coordinate k (int): centroid y coordinate a (int): x-radius of drawn ellipse b (int): y-radius of drawn ellipse theta (float): degree to rotate ellipse in radians. (has no effect if drawing a circle)

Returns: np.ndarray: numpy array with weighted values from 0.08 -> 0.8 representing the proportion of values to create a gradient from. 0.8 being the proportioned values closest to the circle wall.

moseq2_extract.util.mouse_threshold_filter(h5file, thresh=0)

Filter frames in h5 files by threshold value.

Args: h5file (str): path to h5 file thresh (int): threshold at which to apply filter

Returns: (3d-np boolean array): array of regions to include after threshold filter.

moseq2_extract.util.read_yaml(yaml_file)

Read yaml file into a dictionary

Args: yaml_file (str): path to yaml file

Returns: return_dict (dict): dict of yaml contents

moseq2_extract.util.recursive_find_h5s(root_dir='/home/wingillis/dev/moseq/moseq2-extract/docs', ext='.h5', yaml_string='{}.yaml')

Recursively find h5 files, along with yaml files with the same basename

Args: root_dir (str): path to base directory to begin recursive search in. ext (str): extension to search for yaml_string (str): string for filename formatting when saving data

Returns: h5s (list): list of found h5 files dicts (list): list of found metadata files yamls (list): list of found yaml files

moseq2_extract.util.recursive_find_unextracted_dirs(root_dir='/home/wingillis/dev/moseq/moseq2-extract/docs', session_pattern='session_\\d+\\.(?:tgz|tar\\.gz)', extension='.dat', yaml_path='proc/results_00.yaml', metadata_path='metadata.json', skip_checks=False)

Recursively find unextracted (or incompletely extracted) directories

Args: root_dir (str): path to base directory to start recursive search for unextracted folders. session_pattern (str): folder name pattern to search for extension (str): file extension to search for yaml_path (str): path to respective extracted metadata metadata_path (str): path to relative metadata.json files skip_checks (bool): indicates whether to check if the files exist at the given relative paths

Returns: proc_dirs (1d-list): list of paths to each unextracted session’s proc/ directory

moseq2_extract.util.scalar_attributes()

Gets scalar attributes dict with names paired with descriptions.

Returns: attributes (dict): a dictionary of metadata keys and descriptions.

moseq2_extract.util.select_strel(string='e', size=(10, 10))

Returns structuring element of specified shape.

Args: string (str): string to indicate whether to use ellipse or rectangle size (tuple): size of structuring element

Returns: strel (cv2.StructuringElement): selected cv2 StructuringElement to use in video filtering or ROI dilation/erosion.

moseq2_extract.util.set_bground_to_plane_fit(bground_im, plane, output_dir)

Replaces median-computed background image with plane fit. Only occurs if config_data[‘use_plane_bground’] == True.

Args: bground_im (numpy.ndarray): Background image computed via median value in each pixel of depth video. plane (numpy.ndarray): Computed ROI Plane using RANSAC. output_dir (str): Path to write updated background image to.

Returns: bground_im (numpy.ndarray): The background image.

moseq2_extract.util.strided_app(a, L, S)

Create subarrays of an array with a given stride and window length.

Args: a (np.ndarray) - original array L (int) - Window Length S (int) - Stride size

Returns: (np.ndarray) - array of subarrays

moseq2_extract.util.time_str_for_filename(time_str: str) → str

Process the timestamp to be used in the filename.

Args: time_str (str): time str to format

Returns: out (str): formatted timestamp str

Subpackages