Anej Svete

Anej Svete

PhD Student in Natural Language Processing.

Recurrent Neural Language Models as Probabilistic Finite-state Automata

Published in EMNLP 2023, 2023

We study what classes of such probability distributions RNN LMs can represent and show that simple RNNs are equivalent to a subclass of probabilistic finite-state automata, and can thus model a strict subset of probability distributions expressible by finite-state models.

Download the paper here

Citation BibTeX:

@inproceedings{svete-etal-2023-recurrent,
    title = "Recurrent Neural Language Models as Probabilistic Finite-state Automata",
    author = "Svete, Anej  and
      Cotterell, Ryan",
    booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2023",
    address = "Singapore, Singapore",
    publisher = "Association for Computational Linguistics",
}