Anej Svete

Anej Svete

PhD Student in Natural Language Processing.

A Geometric Notion of Causal Probing

Published in arXiv, 2023

We propose a formal definition of intrinsic information about a concept (feature) in a subspace of a language model’s representation space. We propose a counterfactual approach that avoids the failure mode of spurious correlations by treating components in the subspace and its orthogonal complement independently.

Download the paper here

Citation BibTeX:

@article{guerner2023geometric,
      title={A Geometric Notion of Causal Probing}, 
      author={Clément Guerner and Anej Svete and Tianyu Liu and Alexander Warstadt and Ryan Cotterell},
      year={2023},
      eprint={2307.15054},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      journal = {arXiv preprint arXiv:2307.15054},
      url={https://arxiv.org/abs/2307.15054}
}