Sample Analysis

The Sample Analysis dashlet, which could be added via the dashboards tab, allows a per-sample analysis using a gradcam like analysis. It returns the input, overlayed with an attention-like heatmap of the features that effects the model decision for a specific label, per model layer.

This allows to not only understand what effects the model's final decision, but also how does this decompose into more granular features on earlier layers. This is

Sample Analysis

Analysis of a sample returns results from a variety of explainability and error-analysis algorithms, in addition to all its visualizations defined by the Visualizers.

Running Sample Analysis

Running Sample Analysis from Population Exploration

1. Click a dot within the Population Exploration view and selecting a sample.

2. Click "Analyze Sample" on the lower bar.

3. Select sample ID and algorithm

4. Hit "Analyze Sample" again

Running an analysis from the sample analysis dashlet

1. Adding the Sample Analysis dashlet.

2. Make sure that a target model run is selected.

3. Select Data Slice, sample ID, and algorithm.

4. Click "Analyze".

After the analysis is complete, you can further explore the model's response to the sample. The list of Visualizers and their outputs can be found on the left, and the error-analysis visualizations can be found on the right.

The Sample Analysis Dashlet

The sample selection is composed of 3 distinct sections:

1. The sample selection panel that show which sample are selected and allow to take the following actions using the its top buttons:

   1. Select All samples

   2. Add another sample to analysis

   3. Fetch similar samples

2. The viewer tab is similar to the sample visualization in population exploration. The default visualizer that is chosen is the grad-cam-like visualizer (i.e. input with attention heatmap overlayed). All other visualizers are available as well

3. The controller panel where it is possible to select:

   1. Label: the grad-cam like is computed with respect to

   2. Connection: i.e. on which input is the heatmap overlayed

   3. Depth: The layer on which the grad-cam-like heatmap is computed. By default the last layer is selected. Moving the selection to custom and moving the controller to the left shows the gradcam-like heatmap on earlier. The depth is computed by the amount of layers between the respective input and the layer.