moseq2_app.stat package

Submodules

Stats - Controller Module

Main interactive model syllable statistics results application functionality.

class moseq2_app.stat.controller.InteractiveSyllableStats(index_path, model_path, df_path, info_path, max_sylls, load_parquet)

Bases: SyllableStatWidgets

interactive_stat_helper(): Computes and saves the all the relevant syllable information to be displayed.

interactive_syll_stats_grapher(stat, sort, groupby, errorbar, sessions, ctrl_group, exp_group, hyp_test="KW & Dunn's", thresh='usage')

handle ipywidgets interactions and update the currently displayed Bokeh plot.

Args: stat (str or ipywidgets.DropDown): Statistic to plot: [‘usage’, ‘distance to center’] sort (str or ipywidgets.DropDown): Statistic to sort syllables by (in descending order). groupby (str or ipywidgets.DropDown): Data to plot; either group averages, or individual session data. errorbar (str or ipywidgets.DropDown): Error bar to display. [‘None’, ‘CI 95%’ ,’SEM’, ‘STD’] sessions (list or ipywidgets.MultiSelect): List of selected sessions to display data from. ctrl_group (str or ipywidgets.DropDown): Name of control group to compute group difference sorting with. exp_group (str or ipywidgets.DropDown): Name of comparative group to compute group difference sorting with. hyp_test (str or ipywidgets.DropDown): Name of hypothesis testing method to use to compute significant syllables. thresh (str or ipywidgets.DropDown): Name of statistic to threshold the graph by using the Bokeh Slider

run_selected_hypothesis_test(hyp_test_name, stat, ctrl_group, exp_group)

compute the significant syllables for a given pair of groups given the hypothesis test on the data.

Args: hyp_test_name (str): Name of hypothesis test to run. options=[‘KW & Dunns’, ‘Z-Test’, ‘T-Test’, ‘Mann-Whitney’] stat (str): Syllable statistic to compute hypothesis test on. options=[‘usage’, ‘distance to center’, ‘similarity’, ‘difference’] ctrl_group (str): Name of control group to compute group difference sorting with. exp_group (str): Name of comparative group to compute group difference sorting with.

Returns: sig_sylls (list): list of significant syllables to mark on plotted statistics figure

class moseq2_app.stat.controller.InteractiveTransitionGraph(model_path, index_path, info_path, df_path, max_sylls, plot_vertically, load_parquet)

Bases: TransitionGraphWidgets

compute_entropies(labels, label_group)

Compute individual syllable entropy and transition entropy rates for all sessions with in a label_group.

Args: labels (2d list): list of session syllable labels over time. label_group (list): list of groups computing entropies for.

compute_entropy_differences(): Compute cross group entropy/entropy-rate differences and cast them to OrderedDict objects

initialize_transition_data(): Perform all necessary pre-processing to compute the transition graph data and syllable metadata to display via HoverTool.

interactive_transition_graph_helper(layout, edge_threshold, usage_threshold)

generate all the transition graphs given the currently selected thresholding values, then display them in a Jupyter notebook or web page.

Args: layout (string) scalar_color (string) edge_threshold (tuple or ipywidgets.FloatRangeSlider): Transition probability range to include in graphs. usage_threshold (tuple or ipywidgets.FloatRangeSlider): Syllable usage range to include in graphs. speed_threshold (tuple or ipywidgets.FloatRangeSlider): Syllable speed range to include in graphs.

Stats - View Module

View module to facilitate graphing of interactive statistics tools

moseq2_app.stat.view.bokeh_plotting(df, stat, sorting, mean_df=None, groupby='group', errorbar='SEM', syllable_families=None, sort_name='usage', thresh='usage', sig_sylls=[])

Generate a Bokeh plot with interactive tools such as the HoverTool

Args: df (pd.DataFrame): Mean syllable statistic DataFrame. stat (str): Statistic to plot sorting (list): List of the current/selected syllable ordering groupby (str): Value to group data by. Either by unique group name, session name, or subject name. errorbar (str): Error bar type to display sort_name (str): Syllable sorting name displayed in title. thresh (str): Statistic to threshold syllables by using the Range Slider

Returns: p (bokeh figure): Displayed stat plot with optional color pickers.

moseq2_app.stat.view.colorscale(hexstr, scalefactor)

Scale a hex string by scalefactor. Returns scaled hex string.

To darken the color, use a float value between 0 and 1. To brighten the color, use a float value greater than 1.

>>> colorscale("#DF3C3C", .5)
#6F1E1E
>>> colorscale("#52D24F", 1.6)
#83FF7E
>>> colorscale("#4F75D2", 1)
#4F75D2

moseq2_app.stat.view.draw_stats(fig, df, groups, colors, sorting, groupby, stat, errorbar, line_dash='solid', thresh_stat='usage', sig_sylls=[])

iterate through the given DataFrame and plots the data grouped by sepecified column (‘group’, ‘SessionName’, ‘SubjectName’), with the errorbars

Args: fig (bokeh figure): Figure to draw line plot glyphs on df (pd.DataFrame): DataFrame containing all relevant data to plot groups (list of str): List of group names to iterate by colors (list of str): List of group-corresponding colors to iterate by sorting (list of int): Pre-selected syllable index order groupby (str): string that indicates which DataFrame column is being grouped. stat (str): String that indicates the statistic that is being plotted. errorbar (str): String that indicates the type of error bars to be plotted.

Returns: pickers (list of ColorPickers): List of interactive color picker widgets to update the graph colors. slider (bokeh.models.RangeSlider): RangeSlider object used to threshold/filter the displayed syllables.

moseq2_app.stat.view.format_graphs(graphs, group)

Format multiple transition graphs to be stacked in vertical column-order with graph positions corresponding to the difference graphs.

Args: graphs (list): list of generated Bokeh figures. group (list): list of unique groups

Returns: formatted_plots (2D list): list of lists corresponding to rows of figures being plotted.

moseq2_app.stat.view.format_plot(plot)

Turn off all major and minor x,y ticks on the transition plot graphs

Args: plot (bokeh Plot): Current graph being generated

moseq2_app.stat.view.format_stat_plot(p, df, searchbox, slider, sorting)

Edit the bokeh figures x-axis such that the syllable labels are also displayed, and are slanted 45 degrees. Set the legend to be interactive where users can hide line plots by clicking on their legend item. create the bokeh gridplot to hold all of the displayed widgets above the graph to display.

Args: p (bokeh.Figure): bokeh figure with all the glyphs already drawn. df (pd.DataFrame): DataFrame containing all relevant data to plot slider (bokeh.models.RangeSlider): slider object with the currently displayed values used to filter out syllables in the callback function. pickers (list of ColorPickers): List of interactive color picker widgets to insert into a gridplot to display. sorting (list): list of syllable index values to resort the dataframe by.

Returns: graph_n_pickers (bokeh.layout.column): Bokeh Layout object of the widgets and figure to display.

moseq2_app.stat.view.format_trans_graph_edges(graph, neighbor_edge_colors, difference_graph=False)

Compute the colors and widths of all the transition edges between nodes.

Args: graph (nx.DiGraph): networkx graph to read edge weights from and compute widths and colors with. neighbor_edge_colors (dict): dictionary of node transition tuples mapped to corresponding colors (str). difference_graph (bool): indicates whether to color the edges based on the transition difference between two groups.

Returns: edge_color (dict): dict of edge tuple(node1, node2) object mapped to string values describing edge colors. edge_width (dict): dict of edge tuple(node1, node2) object mapped to float values describing edge widths. selected_edge_colors (dict): dict of edge tuple(node1, node2) object mapped to string values describing neighboring edge colors.

moseq2_app.stat.view.get_aux_stat_dfs(df, group, sorting, groupby='group', errorbar='CI 95%', stat='usage')

Compute the group-specific syllable statistics dataframe, and the selected error values to later draw the line plot and error bars.

Args: df (pd.DataFrame): DataFrame containing all relevant data to plot. group (str): group name to get auxiliary statistics dataframe for. sorting (list): list of syllable index values to resort the dataframe by. groupby (str): column to group the syllable stats by. errorbar (str): name of the error bar type to compute values for. stat (str): name of the statistic to plot.

Returns: aux_df (pd.DataFrame): dataframe that only contains the selected group’s mean statistics. stat_err (pd.DataFrame): dataframe that contains the error values for the selected statistic. aux_err (pd.DataFrame): dataframe that contains the error values for all the statistics. errs_x (list): list of x-indices to plot the error bar lines within. errs_y (list): list of y-indices to plot the error bar lines within.

moseq2_app.stat.view.get_ci_vect_vectorized(x, n_boots=10000, n_samp=None, function=<function nanmean>, pct=5)

Compute min and max values within a (default) 95th percentile of the inputted syllable statistic.

Args: x (pandas Series): 1D list of syllable statistics. n_boots (int): Number of bootstrapped examples to generate to compute the percentile. n_samp (int): Number of inputted syllables. function (numpy function): Function to apply to bootstrapped data. pct (int): Percentile to compute.

Returns: percentile (2d numpy array): an array of min and max values for each syllable’s stat value

moseq2_app.stat.view.get_datasources(aux_df, aux_sem, sem, labels, desc, cm_paths, errs_x, errs_y, stat)

Creates Bokeh ColumnDataSources that will be used to draw all the bokeh glyphs (circle, line plots, and errorbars).

Args: aux_df (pd.DataFrame): DataFrame that only contains the selected group’s mean statistics. aux_sem (pd.DataFrame): DataFrame that contains the error values for all the statistics. sem (pd.DataFrame): DataFrame that contains the error values for the selected statistic. labels (numpy array): Syllable label list sorted by the given sorting order. desc (numpy array): Syllable description list sorted by the given sorting order. cm_paths (numpy array): Syllable crowd movie path list sorted by the given sorting order. errs_x (list): List of x-indices to plot the error bar lines within. errs_y (list): List of y-indices to plot the error bar lines within. stat (str): Statistic to display.

Returns: source (bokeh.models.ColumnDataSource): Bokeh data source of mean syllable stats src_dict (dict): dict version of the ColumnDataSource object that will be passed to the JS Callback. err_source (bokeh.models.ColumnDataSource): Bokeh data source of syllable stats error values. err_dict (dict): dict version of the errorbar ColumnDataSource object that will be passed to the JS Callback.

moseq2_app.stat.view.get_difference_legend_items(plot, edge_width, group_name)

Create the difference graph legend items with the min and max transition probabilities for both the up and down-regulated transition probabilities.

Args: plot (bokeh.figure): Bokeh plot to add legend to. edge_width (dict): Dictionary of edge widths group_name (str): Difference graph title.

Returns: diff_items (list): List of LegendItem objects to display

moseq2_app.stat.view.get_group_node_syllable_info(syll_info, node_indices)

Read the given syllable information dict in the ordering provided by the node_indices previously computed via nx.DiGraph.nodes

Args: syll_info (dict): dict of syllable label information to read. node_indices (list): ordering of syllables to read from the syll info dict

Returns: labels (list): 1d list of syllable labels corresponding to each node index descs (list): 1d list of syllable descriptions corresponding to each node index cm_paths (list): 1d list of syllable crowd movie relpaths corresponding to each node index

moseq2_app.stat.view.get_legend_items(plot, edge_width, group_name, difference_graph=False)

generate two legend items to describe the transition plots.

Args: plot (bokeh figure): bokeh generated figure to add legends to. edge_width (dict): dict of edge tuple(node1, node2) object mapped to float values describing edge widths. group_name (str): name of currently plotted group difference_graph (bool): indicator for whether the currently plotted graph is a difference graph between two groups.

Returns: main_legend (bokeh Legend instance): Legend containing colors describing incoming and outgoing transitions. info_legend (bokeh Legend instance): Legend containing edge widths corresponding to min and max transition probabilities.

moseq2_app.stat.view.get_minmax_tp(edge_width, diff=False)

Compute the min and max transition probabilities given the rescaled edge-widths.

Args: edge_width (dict): dict of syllables paired with drawn edge widths diff (bool): indicates whether to compute min/max transition probs. for up and down-regulated syllables.

Returns: min_tp (float): min transition probability (min_down_tp if diff=True) max_tp (float): max transition probability (max_down_tp if diff=True) min_up_tp (float): min transition probability in up-regulated syllable max_up_tp (float): max transition probability in up-regulated syllable

moseq2_app.stat.view.get_neighbors(graph, node_indices, group_name)

Compute the incoming and outgoing syllable entropies, entropy rates, previous nodes and neighboring nodes for all the nodes included in node_indices.

Args: graph (networkx DiGraph): Generated DiGraph to convert to Bokeh glyph and plot. node_indices (list): List of node indices included in the given graph group_name (str): Graph’s group name.

Returns: prev_states (list): List of previous nodes for each node index in the graph. next_states (list): List of successor nodes/syllables for each node in the graph neighbor_edge_colors (list): List of colors determining whether an edge is incoming or outgoing from each node.

moseq2_app.stat.view.get_node_labels(plots, graph_renderer, rendered_graphs, graph, node_indices)

create a LabelSet instance to display each group’s corresponding syllable numbers of the correct corresponding nodes on each plot/for each group.

Args: plots (list): list of plots currently generated in the plot_interactive_transition_graph loop. graph_renderer (bokeh.plotting.GraphRenderer instance): the canvas of the current group’s transition graph to add the information to. rendered_graphs (list): list of GraphRenderers currently generated in the plot_interactive_transition_graph loop. graph (nx.DiGraph): networkx graph to read the node coordinates from. node_indices (list): ordering of syllables to read from the syll info dict

Returns: labels (bokeh.models.LabelSet instance): glyph to render on the graph such that the nodes are numbered.

moseq2_app.stat.view.get_syllable_info(df, sorting)

Return the labels, descriptions and crowd movie paths for all the syllables to display in the x-axis, and hover tool.

Args: df (pd.DataFrame): DataFrame containing all relevant data to plot. sorting (list): list of syllable index values to resort the dataframe by.

Returns: labels (numpy array): syllable label list sorted by the given sorting order. desc (numpy array): syllable description list sorted by the given sorting order. cm_paths (numpy array): syllable crowd movie path list sorted by the given sorting order.

moseq2_app.stat.view.get_trans_graph_group_stats(node_indices, usages, scalars)

Compute and returns all the syllable statistics to display and filter the graph using the GUI.

Args: node_indices (list): list of plotted node indices as serialized from a networkX.Digraph.nodes array usages (list): list of syllable usages corresponding to the node_indices. scalars (dict): dict of syllable scalar values

Returns: group_stats (dict): packed dict of syllable scalar strings mapped to 1d lists of the values corresponding to the node_indices.

moseq2_app.stat.view.plot_dendrogram(index_file, model_path, syll_info_path, save_dir, max_syllable=40, color_by_cluster=False)

plot a static dentrogram

Args:: index_file (str): path to index file model_path (str): path to the model syll_info_path (str): path to syll_info save_dir (str): max_syllable (int, optional): _description_. Defaults to 40. color_by_cluster (bool, optional): _description_. Defaults to False.

moseq2_app.stat.view.plot_interactive_transition_graph(graphs, pos, group, group_names, usages, syll_info, incoming_transition_entropy, outgoing_transition_entropy, scalars, scalar_color='default', plot_vertically=False, legend_loc='above')

Convert the computed networkx transition graphs to Bokeh glyph objects that can be interacted with and updated throughout run-time.

Args: graphs (list of nx.DiGraphs): list of created networkx graphs. pos (nx.Layout): shared node position coordinates layout object. group (list): list of unique group names. group_names (list): list of names for all the generated transition graphs + difference graphs usages (list of OrdreredDicts): list of OrderedDicts containing syllable usages. syll_info (dict): dict of syllable label information to display with HoverTool scalars (dict): dict of syllable scalar information to display with HoverTool

moseq2_app.stat.view.set_fill_color(scalar_color, data_dict)

Set the node fill coloring based on the selected scalar value.

Args: scalar_color (str): name of scalar to color nodes by. data_dict (dict): dict containing dicts of scalar_df keys and their corresponding values to create the linear color map from.

Returns: fill_color (str or list): list of colors per node, or single color (white) empty (bool): indicator for whether to display a color bar.

moseq2_app.stat.view.set_grouping_colors(df, groupby)

return the unique group names, and their associated colors to use when plotting the default figure.

Args: df (pd.DataFrame): DataFrame containing all relevant data to plot. groupby (str): column to group the syllable stats by.

Returns: groups (list): list of group names to plot line plots for. group_colors (list): list of colors corresponding to each plotted group. colors (list): list of all the colors used to plot the glyphs

moseq2_app.stat.view.set_node_colors_and_sizes(graph, usages, node_indices, difference_graph=False)

Compute the colors and sizes of all the transition nodes.

Args: graph (nx.DiGraph): networkx graph to set the new node attributes to. usages (list): list of syllable usages corresponding to the node_indices. node_indices (list): list of plotted node indices as serialized from a networkX.Digraph.nodes array difference_graph (bool): indicates whether to color the edges based on the transition difference between two groups.

moseq2_app.stat.view.set_node_labels(x, y, syllable)

create a LabelSet instance to render the syllables numbers on each graphed transition node.

Args: x (list): list of x coordinates corresponding to each node in the current graph y (list): list of y coordinates corresponding to each node in the current graph syllable (list): list of syllable numbers corresponding to each node in the current graph

Returns: labels (bokeh.models.LabelSet instance): glyph to render on the graph such that the nodes are numbered.

moseq2_app.stat.view.setup_graph_hover_renderers(graph_renderer, group_stats, node_indices)

Add all the information enclosed in the group_stats dict to the currently plotted transition graph

Args: graph_renderer (bokeh.plotting.GraphRenderer instance): the canvas of the current group’s transition graph to add the information to. group_stats (dict): packed dict of syllable scalars to add to the hover tool. node_indices (list): ordering of syllables to read from the syll info dict

Returns: graph_renderer (bokeh.plotting.GraphRenderer instance): updated reference of the GraphRenderer instance.

moseq2_app.stat.view.setup_hovertool(renderers, callback=None)

Initialize hover tool with tooltips showing all the syllable information and the crowd movies upon hovering over a syllable circle glyph.

Args: renderers (list bokeh.Renderer Instances): drawn bokeh glyph representing syllables, that will be updated in the callback function callback (bokeh.models.CustomJS): javascript callback function to embed to hover tool objects to preserve alignment.

Returns: hover (bokeh.models.HoverTool): hover tool to embed into the created figure.

moseq2_app.stat.view.setup_node_and_edge_interactions(graph_renderer, group_stats, scalar_color)

Add the interactive functionality to hovering and tapping on the nodes and edges of each transition graph.

Args: graph_renderer (bokeh.plotting.GraphRenderer instance): the canvas of the current group’s transition graph to add the information to. group_stats (dict): packed dict of syllable scalars to add to the hover tool. scalar_color (str): name of scalar to color nodes by.

Returns: graph_renderer (bokeh.plotting.GraphRenderer instance): updated reference of the GraphRenderer instance. color_bar (bokeh.models.ColorBar instance): a color bar Bokeh glyph to add to the current graph if the scalar colors are not all white.

moseq2_app.stat.view.setup_slider(src_dict, err_dict, err_source, slider, circle, line, thresh_stat='usage')

Initializes the CustomJS script callback function used to update the plot upon changing the value of the RangeSlider.

Args: src_dict (dict): dict object containing all the stats information contained in the main ColumnDataSource. err_dict (dict): dict object containing all the error margins for the src_dict statistics. err_source (bokeh.models.ColumnDataSource): data source to update based on slider values. slider (bokeh.models.RangeSlider): slider object with the currently displayed values used to filter out syllables in the callback function. circle (bokeh.glyph circle): drawn bokeh glyph representing syllables, that will be updated in the callback function line (bokeh.glyph line): drawn line connecting all the circle nodes in the bokeh figure. thresh_stat (str): name of the statistic to threshold by.

Returns: callback (bokeh.models.CustomJS): javascript callback function to embed to slider and hover tool objects.

moseq2_app.stat.view.setup_syllable_search(src_dict, err_dict, err_source, searchbox, circle, line)

Initializes the CustomJS script callback function used to update the plot upon changing the value of the TextInput.

Args: src_dict (dict): dict object containing all the stats information contained in the main ColumnDataSource. err_dict (dict): dict object containing all the error margins for the src_dict statistics. err_source (bokeh.models.ColumnDataSource): data source to update based on slider values. searchbox (bokeh.models.TextInput): text input widget for users to input syllable labels to find. circle (bokeh.glyph circle): drawn bokeh glyph representing syllables, that will be updated in the callback function line (bokeh.glyph line): drawn line connecting all the circle nodes in the bokeh figure.

Returns: callback (bokeh.models.CustomJS): javascript callback function to embed to search bar and hover tool objects.

moseq2_app.stat.view.setup_trans_graph_tooltips(plot)

display a hover tool, tap tool, and a box select tool to the plot

Args: plot (bokeh figure): bokeh generated figure to add tools to.

Returns:

Stats - Widgets Module

Widgets module containing classes with components for each of the interactive syllable statistics tools.

class moseq2_app.stat.widgets.SyllableStatBokehCallbacks(condition=''): Bases: object

class moseq2_app.stat.widgets.SyllableStatWidgets

Bases: object

clear_on_click(b=None)

Clear the cell output

Args: b (button click)

on_grouping_update(event)

Update the MultipleSelect widget upon selecting groupby == SubjectName or SessionName. Hides it if groupby == group.

Args: event (user clicks new grouping)

class moseq2_app.stat.widgets.TransitionGraphWidgets