[Agustin-Picard]

This PR adds input-to-concept localization to HolisticCraft.

The new API answers:

Which input regions drive the activation of concept k?

It does so by exposing the fitted latent extractor and factorizer as a model that returns one concept activation score per concept, then applying a standard Xplique black-box attribution method to the selected score.

This complements the existing visualization of latent concept activation maps; it does not replace it.

The implementation is currently validated with Rise and SobolAttributionMethod and is designed to work with compatible black-box attribution methods.

Motivation

HolisticCraft already produces concept coefficients coeffs_u.

When the latent representation is spatial, these coefficients form concept activation maps:

U_k(x)

which can be resized and overlaid on the input.

With the default spatial mean reducer, the concept score is
s_k(x) = mean_{i,j} U_k(x)_{i,j}. Resizing U_k(x) is therefore a CAM-style
map for the concept score: it displays the spatial map underlying that score
directly, without gradients or class-prediction weights. This works well for CNN
architectures, but may not render well for very specific deep architectures,
large receptive fields, attention-based models, multi-scale representations,
or other cases where a latent position does not correspond
directly to a local input region.

This PR adds the complementary input-to-concept path. It computes a concept
score, then a black-box explainer perturbs input regions to attribute that score
to the input:

input
  -> latent activations
  -> factorizer.encode(...)
  -> concept activation map U_k(x)
  -> reducer R
  -> concept activation score s_k(x)
  -> black-box input attribution
  -> concept localization map

Terminology

The implementation and documentation use four distinct quantities:

Quantity	Definition	Question answered
Concept activation map	Spatial factorization coefficients U_k(x), stored in coeffs_u; CAM-style map for the spatial mean concept score	Where and how strongly is concept k activated in latent space?
Concept activation score	Scalar reduction s_k(x) = R(U_k(x))	How strongly is concept k activated overall?
Concept importance	Attribution of the task prediction to concept activations	How much do concept activations contribute to the task prediction?
Concept localization map	Attribution of a concept activation score to the input	Which input regions drive the concept activation score?

For perturbation-based methods, the two attribution paths vary different quantities:

Concept importance (Sobol): perturb latent concept activations
    -> measure the task prediction -> attribute it to concepts

Concept localization (RISE/Sobol): perturb input regions
    -> measure concept activation score s_k(x) -> attribute it to input regions

Concept importance also supports gradient-based methods; the first path
describes Sobol-based importance.

For CNN activations with shape (N, H', W', C), factorization produces coeffs_u with shape (N, H', W', K), so each U_k(x) is spatial.

Spatial ViT tokens can similarly form a concept activation map when mapped back to their patch grid.

For global representations with shape (N, K), there is no spatial concept activation map; the coefficient for each concept is already a global activation score.

Public API

make_concept_localizer()

localizer = craft.make_concept_localizer(
    concept_reducer="mean",
)

The returned callable maps a batch of input images to:

(batch_size, number_of_concepts)

concept activation scores.

Built-in reducers are:

"mean"
"sum"
"max"

A custom callable can also be provided as long as it returns one scalar per input and concept.

The localizer preserves signed concept coefficients. Users who intentionally want magnitude-based scores can provide an appropriate custom reducer.

The concrete TensorFlow/PyTorch localizer implementations are private implementation details; the public entry point is HolisticCraft.make_concept_localizer().

attribute_concepts_to_inputs()

concept_maps = craft.attribute_concepts_to_inputs(
    images,
    partial_explainer=partial_explainer,
    concept_ids=[0, 3, 7],
    concept_reducer="mean",
)

This method:

1. validates the requested concepts;
2. creates the concept-score localizer;
3. instantiates the configured black-box explainer once;
4. uses one-hot concept targets;
5. attributes each requested concept activation score to the input.

For image inputs, the result has shape:

(N, H, W, number_of_concepts)

Channel k always corresponds to concept k.

If only a subset of concepts is requested, uncomputed channels are filled entirely with NaN, so they cannot be confused with valid zero attribution.

TensorFlow callers use native channel-last inputs. PyTorch callers pass native channel-first tensors to the high-level method; the framework wrapper handles the layout conversion required by the attribution stack.

Why localization is black-box only

Input-to-concept localization evaluates the fitted factorizer on perturbed inputs through:

factorizer.encode(...)

This path is not guaranteed to be differentiable.

For this reason, attribute_concepts_to_inputs() currently supports black-box attribution methods only.

Gradient-based methods remain supported for the separate concept importance path, where concept activations are decoded toward the task prediction.

No task operator is required for concept localization because one-hot targets directly select a concept activation score.

If a non-None operator is supplied in the PartialExplainer, it is ignored with a warning.

Factorizers used for localization must support out-of-sample encode(). A non-inductive factorizer raises an explicit runtime error when localization is requested.

Example

Sobol

from xplique.attributions import SobolAttributionMethod
from xplique.concepts import PartialExplainer

sobol = PartialExplainer(
    SobolAttributionMethod,
    grid_size=8,
    nb_design=32,
    perturbation_function="inpainting",
)

concept_maps = craft.attribute_concepts_to_inputs(
    model_images,
    partial_explainer=sobol,
    concept_ids=[0, 3, 7],
    concept_reducer="mean",
)

craft.display_images_per_concept(
    display_images,
    concept_maps=concept_maps,
    order=[0, 3, 7],
)

RISE

RISE uses the same public localization API; only the PartialExplainer configuration changes:

from xplique.attributions import Rise
from xplique.concepts import PartialExplainer

rise = PartialExplainer(
    Rise,
    nb_samples=2000,
    grid_size=7,
    preservation_probability=0.5,
    mask_value=0.0,
)

concept_maps = craft.attribute_concepts_to_inputs(
    model_images,
    partial_explainer=rise,
    concept_ids=[0, 3, 7],
)

Visualization integration

The existing display API now supports both interpretation modes.

Concept activation map visualization

Existing behavior remains available through coeffs_u:

coeffs_u = craft.transform(model_images)

craft.display_images_per_concept(
    display_images,
    coeffs_u=coeffs_u,
)

For spatial coefficients and the default mean score, this is a CAM-style
visualization of the concept score in latent space. No input perturbations are
used to produce this map.

Concept localization maps

Input attribution maps can instead be provided with:

craft.display_images_per_concept(
    display_images,
    concept_maps=concept_maps,
)

display_top_images_per_concept() also accepts concept_maps, with an important semantic distinction:

• coeffs_u determines which images rank highest for a concept;
• concept_maps determines the localization overlay.

Concept localization magnitude therefore does not replace concept activation as the ranking signal.

If a localization map already matches the input resolution, display_concept_heatmap() avoids an unnecessary resize.

Signed localization maps are currently displayed by absolute magnitude. Neither concept activation maps nor concept localization maps should be interpreted as segmentation masks.

Backward compatibility

The feature is opt-in.

Existing usage of:

display_images_per_concept(...)
display_top_images_per_concept(...)
estimate_importance(...)
compute_explanation_per_concept(...)
get_topk_images_per_concept(...)
ConceptDecoder

retains its previous behavior.

No existing call triggers black-box input attribution unless attribute_concepts_to_inputs() is explicitly requested.

Tests

The PR includes coverage for:

• concept activation score reducers: mean, sum, max, and custom callables;
• spatial, token, and global concept coefficient layouts;
• TensorFlow/PyTorch concept-score parity;
• TensorFlow and PyTorch black-box localization smoke tests;
• concept selection and one-hot target construction;
• output shape and finite-value validation;
• partial localization with NaN channels for uncomputed concepts;
• invalid/duplicate concept IDs and invalid image batches;
• ignored explicit operator behavior;
• black-box-only validation;
• non-inductive factorizer failures;
• localization-map display behavior;
• preservation of coefficient-based top-image ranking;
• a semantic perturbation test showing that localization follows the concept's spatial dependency.

Cross-framework parity tests and framework-agnostic localization regressions are kept separate from the PyTorch-specific classification tests to keep the test responsibilities explicit.

Documentation

The HolisticCraft documentation now:

• defines concept activation maps, concept activation scores, concept importance, and concept localization maps consistently;
• explains when a concept activation map is a CAM-style visualization of the concept score;
• includes a compact end-to-end workflow tied to the actual API;
• explains the distinction between latent activation-map visualization and input attribution;
• explains CNN, spatial-token, and global coefficient layouts;
• documents the black-box-only restriction and factorizer encode() requirement;
• provides Sobol and RISE localization examples;
• documents the interaction between coeffs_u, concept_maps, and top-image ranking.

Updated notebook:

https://drive.google.com/file/d/19eg2uU95A_LW1hQ1ENrhGI41lLK35UHu/view?usp=sharing

[AntoninPoche]

There are few artifacts due to AI usage:

• Different names for the same thing
• Overcomplex code
• Lack of comments
• Many tests with weird structure, this will surely force future change to follow an arbitraly defined architecture. We called that technical debt the last time. (I did not review in detail the regression one).

Nonetheless, it seems to do the job, depends how much effort you want to put into it.

[Agustin-Picard]

I've updated the notebook following your suggestions @AntoninPoche : https://drive.google.com/file/d/19eg2uU95A_LW1hQ1ENrhGI41lLK35UHu/view?usp=sharing

[Agustin-Picard]

Thanks again @AntoninPoche ! Since the last review, I pushed a broader cleanup/simplification pass addressing most of the points you raised:

• Harmonized the terminology and clarified the distinction between concept importance (perturb/attribute concept activations → task prediction) and concept localization (perturb input → concept activation score). Removed all mentions of coefficient maps, as concept activation maps is much clearer.
• Simplified the implementation further (as you suggested): there is now a single shared concept localizer in HolisticCraft for TF and torch; and the extra Torch wrapper layer was removed. The Torch backend now only handles the NHWC → NCHW conversion through _prepare_localizer_inputs (and is basically a no-op for TF).
• Simplified the display/localization plumbing and removed unnecessary test scaffolding.
• Reorganized the tests into dedicated localization/parity coverage, removed duplicated TF/Torch localizer tests, and kept parity checks for mean / sum / max (including CUDA when available).
• Updated the documentation and notebook accordingly (keeping mostly what you had proposed, with maybe 2 or 3 modifications).

I've also adjusted the PR's description to reflect the current API!

[AntoninPoche]

LGTM!

Much clearer on the documentation and efficient in the code!