Portrait de Ian Porada

Ian Porada

Doctorat - McGill
Superviseur⋅e principal⋅e
Jackie Cheung
Sujets de recherche
Raisonnement
Traitement du langage naturel
Site web
Google Scholar
GitHub

Publications

Investigating Failures to Generalize for Coreference Resolution Models
Ian Porada
Alexandra Olteanu
Kaheer Suleman
Adam Trischler
Jackie Cheung
Coreference resolution models are often evaluated on multiple datasets. Datasets vary, however, in how coreference is realized -- i.e., how … (voir plus)the theoretical concept of coreference is operationalized in the dataset -- due to factors such as the choice of corpora and annotation guidelines. We investigate the extent to which errors of current coreference resolution models are associated with existing differences in operationalization across datasets (OntoNotes, PreCo, and Winogrande). Specifically, we distinguish between and break down model performance into categories corresponding to several types of coreference, including coreferring generic mentions, compound modifiers, and copula predicates, among others. This break down helps us investigate how state-of-the-art models might vary in their ability to generalize across different coreference types. In our experiments, for example, models trained on OntoNotes perform poorly on generic mentions and copula predicates in PreCo. Our findings help calibrate expectations of current coreference resolution models; and, future work can explicitly account for those types of coreference that are empirically associated with poor generalization when developing models.
2024-08-01
Findings of the Association for Computational Linguistics ACL 2024 (publié)
doi.org
arxiv.org
Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge
Ian Porada
Alessandro Sordoni
Jackie Cheung
2022-07-01
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (publié)
doi.org
arxiv.org
Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge
Ian Porada
Alessandro Sordoni
Jackie Cheung
Transformer models pre-trained with a masked-language-modeling objective (e.g., BERT) encode commonsense knowledge as evidenced by behaviora… (voir plus)l probes; however, the extent to which this knowledge is acquired by systematic inference over the semantics of the pre-training corpora is an open question. To answer this question, we selectively inject verbalized knowledge into the pre-training minibatches of BERT and evaluate how well the model generalizes to supported inferences after pre-training on the injected knowledge. We find generalization does not improve over the course of pre-training BERT from scratch, suggesting that commonsense knowledge is acquired from surface-level, co-occurrence patterns rather than induced, systematic reasoning.
2021-12-16
ArXiv (preprint)
doi.org
arxiv.org
Modeling Event Plausibility with Consistent Conceptual Abstraction
Ian Porada
Kaheer Suleman
Adam Trischler
Jackie Cheung
2021-06-01
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (publié)
doi.org
arxiv.org
ADEPT: An Adjective-Dependent Plausibility Task
Ali Emami
Ian Porada
Alexandra Olteanu
Kaheer Suleman
Adam Trischler
Jackie Cheung
2021-01-01
Annual Meeting of the Association for Computational Linguistics (publié)
doi.org
META-Learning State-based Eligibility Traces for More Sample-Efficient Policy Evaluation
Mingde Zhao
Sitao Luan
Ian Porada
Xiao-Wen Chang
Doina Precup
Temporal-Difference (TD) learning is a standard and very successful reinforcement learning approach, at the core of both algorithms that lea… (voir plus)rn the value of a given policy, as well as algorithms which learn how to improve policies. TD-learning with eligibility traces provides a way to boost sample efficiency by temporal credit assignment, i.e. deciding which portion of a reward should be assigned to predecessor states that occurred at different previous times, controlled by a parameter
2020-01-01
AAMAS (publié)
dblp.uni-trier.de
Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text
Ian Porada
Kaheer Suleman
Jackie Cheung
2019-11-01
Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing (publié)
doi.org
arxiv.org