Portrait de Ian Porada n'est pas disponible

Ian Porada

poradaia@mila.quebec
Doctorat - McGill University
Superviseur⋅e principal⋅e
Jackie Cheung
Site web
Github
Google Scholar

Publications

Investigating Failures to Generalize for Coreference Resolution Models
Ian Porada
Alexandra Olteanu
Kaheer Suleman
Adam Trischler
Jackie Cheung
Coreference resolution models are often evaluated on multiple datasets. Datasets vary, however, in how coreference is realized -- i.e., how … (voir plus)the theoretical concept of coreference is operationalized in the dataset -- due to factors such as the choice of corpora and annotation guidelines. We investigate the extent to which errors of current coreference resolution models are associated with existing differences in operationalization across datasets (OntoNotes, PreCo, and Winogrande). Specifically, we distinguish between and break down model performance into categories corresponding to several types of coreference, including coreferring generic mentions, compound modifiers, and copula predicates, among others. This break down helps us investigate how state-of-the-art models might vary in their ability to generalize across different coreference types. In our experiments, for example, models trained on OntoNotes perform poorly on generic mentions and copula predicates in PreCo. Our findings help calibrate expectations of current coreference resolution models; and, future work can explicitly account for those types of coreference that are empirically associated with poor generalization when developing models.
2023-03-16
ArXiv (prépublication)
doi.org
arxiv.org
Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge
Ian Porada
Alessandro Sordoni
Jackie Cheung
2022-07-01
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (publié)
doi.org
arxiv.org
Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge
Ian Porada
Alessandro Sordoni
Jackie Cheung
Transformer models pre-trained with a masked-language-modeling objective (e.g., BERT) encode commonsense knowledge as evidenced by behaviora… (voir plus)l probes; however, the extent to which this knowledge is acquired by systematic inference over the semantics of the pre-training corpora is an open question. To answer this question, we selectively inject verbalized knowledge into the pre-training minibatches of BERT and evaluate how well the model generalizes to supported inferences after pre-training on the injected knowledge. We find generalization does not improve over the course of pre-training BERT from scratch, suggesting that commonsense knowledge is acquired from surface-level, co-occurrence patterns rather than induced, systematic reasoning.
2021-12-16
ArXiv (preprint)
doi.org
arxiv.org
Modeling Event Plausibility with Consistent Conceptual Abstraction
Ian Porada
Kaheer Suleman
Adam Trischler
Jackie Cheung
2021-06-01
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (publié)
doi.org
arxiv.org
ADEPT: An Adjective-Dependent Plausibility Task
Ali Emami
Ian Porada
Alexandra Olteanu
Kaheer Suleman
Adam Trischler
Jackie Cheung
2021-01-01
Annual Meeting of the Association for Computational Linguistics (publié)
doi.org
Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text
Ian Porada
Kaheer Suleman
Jackie Cheung
2019-11-01
Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing (publié)
doi.org
arxiv.org