Metraipolis › archive › dc_ec77dda0ad23

Mechanism-Level Evaluation for Vision-Language Models: Controlled Activation-Replacement Diagnosis of Gender Bias — arXiv:2609.16651

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.16651
License (as archived)
CC0-1.0
Archived
2026-10-15T21:52:12.997Z
SHA-256 of the archived text
f7c8e525d5540137a5632661e1cab66abe91f610dd65a27a4607ef516bce48a4
Stable URL of this page
https://metraipolis.com/documento/dc_ec77dda0ad23

How to cite

Mechanism-Level Evaluation for Vision-Language Models: Controlled Activation-Replacement Diagnosis of Gender Bias — arXiv:2609.16651 — arXiv (en), https://arxiv.org/abs/2609.16651 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ec77dda0ad23 (Metraipolis archive dc_ec77dda0ad23, sha256 f7c8e525d5540137a5632661e1cab66abe91f610dd65a27a4607ef516bce48a4)
BibTeX
@misc{dcec77dda0ad23,
  title = {Mechanism-Level Evaluation for Vision-Language Models: Controlled Activation-Replacement Diagnosis of Gender Bias — arXiv:2609.16651},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.16651},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ec77dda0ad23; sha256 f7c8e525d5540137a5632661e1cab66abe91f610dd65a27a4607ef516bce48a4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.16651. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Behavioral benchmarking reveals \emph{what} biases exist in vision-language models but not \emph{which internal components} are most sensitive to targeted intervention, precluding principled intervention. We argue for mechanism-level evaluation as a necessary complement, demonstrating causal mediation analysis as a diagnostic instrument for gender bias. We decompose gender-cue effects into controlled indirect effects attributable to specific-layer activations and direct effects through all other pathways, producing layer-by-layer mechanistic signatures. Across six models spanning three architectural families (LLaVA-1.5, LLaVA-NeXT, InstructBLIP at 7B/13B) and two 8B-scale architectures, three findings emerge: language-layer activations exhibit the greatest output sensitivity under controlled intervention, with the direct component often carrying the opposite sign; architectural choices redistribute layer-wise sensitivity to activation replacement; and counterfactual scores diverge from surface-level scores, exposing implicit associations. Systematic ablation validates internal consistency. An intervention experiment finds that the average indirect effect (AIE) and downstream intervention effectiveness are only weakly correlated (Pearson $r = 0.33$), and the layer with the second-largest AIE produces near-zero bias change---indicating that mechanistic diagnosis captures activation-replacement sensitivity but does not, by itself, identify optimal intervention targets. These results show mechanism-level evaluation captures architecture-specific sensitivity patterns that behavioral benchmarks cannot; pairing both should become standard NLP practice. Code: this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.MM).
Categorie arXiv: Multimedia (cs.MM).
Pagina del lavoro: https://arxiv.org/abs/2609.16651 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T21:36:31.124Z dalla pagina https://arxiv.org/catchup/cs.MM/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).