Visualization Modules

The overcomplete.visualization module provides a set of tools for analyzing and visualizing top-concept activations in batches of images. These tools help for understanding which part of an images contribute for some concept.

Overview

The visualization module includes functions for: - Overlaying heatmaps onto images to highlight top-concept activations. - Displaying the most representative images for a given concept. - Applying contour visualizations to emphasize highly activating regions. - Zooming into the hottest points of a heatmap. - Highlighting evidence areas using percentile-based heatmap thresholding.

Example Usage

from overcomplete.visualization import (overlay_top_heatmaps,
evidence_top_images, zoom_top_images, contour_top_image)
# lets imagine we have 100 images and 10k concepts maps
# and we want to visualize concept 3
images = torch.randn(100, 3, 256, 256)
heatmaps = torch.randn(100, 14, 14, 10_000)

# heatmap + transparency (recommended)
overlay_top_heatmaps(images, heatmaps, concept_id=3,
                     cmap='jet', alpha=0.35)
# transparency based
evidence_top_images(images, heatmaps, concept_id=3)
# zoom into max activating crops
zoom_top_images(images, heatmaps, concept_id=3, zoom_size=100)
# contour of most important part (boundary)
contour_top_image(images, heatmaps, concept_id=3)

For more details, see the module in Overcomplete.visualization.

overlay_top_heatmaps(images,
                     heatmaps,
                     concept_id,
                     cmap=None,
                     alpha=0.35)

Visualize the top activating image for a concepts and overlay the associated heatmap.

Parameters

• images : torch.Tensor or PIL.Image or np.ndarray

  • Batch of input images of shape (batch_size, channels, height, width).

• z_heatmaps : torch.Tensor or np.ndarray

  • Batch of heatmaps corresponding to the input images of shape (batch_size, height, width, num_concepts).

• concept_id : int

  • Index of the concept to visualize.

• cmap : str, optional

  • Colormap for the heatmap, by default 'jet'.

• alpha : float, optional

  • Transparency of the heatmap overlay, by default 0.35.

evidence_top_images(images,
                    heatmaps,
                    concept_id,
                    percentiles=None)

Visualize the top activating image for a concept and highlight the top activating pixels.

Parameters

• images : torch.Tensor or PIL.Image or np.ndarray

  • Batch of input images of shape (batch_size, channels, height, width).

• heatmaps : torch.Tensor or np.ndarray

  • Batch of heatmaps corresponding to the input images of shape (batch_size, height, width, num_concepts).

• concept_id : int

  • Index of the concept to visualize.

• percentiles : list of int, optional

  • List of percentiles to highlight, by default None.

zoom_top_images(images,
                heatmaps,
                concept_id,
                zoom_size=100)

Zoom into the hottest point in the heatmaps for a specific concept.

Parameters

• images : torch.Tensor or PIL.Image or np.ndarray

  • Batch of input images of shape (batch_size, channels, height, width).

• heatmaps : torch.Tensor or np.ndarray

  • Batch of heatmaps corresponding to the input images of shape (batch_size, height, width, num_concepts).

• concept_id : int

  • Index of the concept to visualize.

• zoom_size : int, optional

  • Size of the zoomed area around the hottest point, by default 100.

contour_top_image(images,
                  heatmaps,
                  concept_id,
                  percentiles=None,
                  cmap='viridis',
                  linewidth=1.0)

Contour the best images for a specific concept using heatmap percentiles.

Parameters

• images : torch.Tensor or PIL.Image or np.ndarray

  • Batch of input images of shape (batch_size, channels, height, width).

• heatmaps : torch.Tensor or np.ndarray

  • Batch of heatmaps corresponding to the input images of shape (batch_size, height, width, num_concepts).

• concept_id : int

  • Index of the concept to visualize.

• percentiles : list of int, optional

  • List of percentiles to contour, by default [70].

• cmap : str, optional

  • Colormap for the contours, by default "viridis".

• linewidth : float, optional

  • Width of the contour lines, by default 1.0.