Portrait de Antoine Bordes n'est pas disponible

Antoine Bordes

Alumni

Publications

C AUSAL R: Causal Reasoning over Natural Language Rulebases
Jason Weston
Antoine Bordes
Sumit Chopra
Thomas Wolf
Lysandre Debut
Julien Victor Sanh
Clement Chaumond
Anthony Delangue
Pier-339 Moi
Tim ric Cistac
R´emi Rault
Morgan Louf
Funtow-900 Joe
Sam Davison
Patrick Shleifer
Von Platen
Clara Ma
Yacine Jernite
Julien Plu
Canwen Xu … (voir 6 de plus)
Zhilin Yang
Peng Qi
Saizheng Zhang
Yoshua Bengio
William W Cohen
Russ Salakhutdinov
Transformers have been shown to be able to 001 perform deductive reasoning on a logical rule-002 base containing rules and statements writte… (voir plus)n 003 in natural language. Recent works show that 004 such models can also produce the reasoning 005 steps (i.e., the proof graph ) that emulate the 006 model’s logical reasoning process. But these 007 models behave as a black-box unit that emu-008 lates the reasoning process without any causal 009 constraints in the reasoning steps, thus ques-010 tioning the faithfulness. In this work, we frame 011 the deductive logical reasoning task as a causal 012 process by defining three modular components: 013 rule selection, fact selection, and knowledge 014 composition. The rule and fact selection steps 015 select the candidate rule and facts to be used 016 and then the knowledge composition combines 017 them to generate new inferences. This ensures 018 model faithfulness by assured causal relation 019 from the proof step to the inference reasoning. 020 To test our causal reasoning framework, we 021 propose C AUSAL R where the above three com-022 ponents are independently modeled by trans-023 formers. We observe that C AUSAL R is robust 024 to novel language perturbations, and is com-025 petitive with previous works on existing rea-026 soning datasets. Furthermore, the errors made 027 by C AUSAL R are more interpretable due to 028 the multi-modular approach compared to black-029 box generative models. 1 030
SPE: Symmetrical Prompt Enhancement for Factual Knowledge Retrieval
James M. Crawford
Matthew L. Ginsberg
Jacob Devlin
Ming-Wei Chang
Kenton Lee
Xavier Glorot
Antoine Bordes
Yoshua Bengio
Alex Graves
Abdel rahman Mohamed
Adi Haviv
Jonathan Berant
Amir Globerson
Chloe Kiddon
Pedro M. Domingos
Brian Lester
Rami Al-rfou'
Noah Constant. 2021
Pengfei Liu
Weizhe Yuan … (voir 6 de plus)
Jinlan Fu
Zhengbao Jiang
Xiao Liu
Yanan Zheng
Zhengxiao Du
Ming Ding
Pretrained language models (PLMs) have 001 been shown to accumulate factual knowledge 002 from their unsupervised pretraining proce-003 dure… (voir plus)s (Petroni et al., 2019). Prompting is an 004 effective way to query such knowledge from 005 PLMs. Recently, continuous prompt methods 006 have been shown to have a larger potential 007 than discrete prompt methods in generating ef-008 fective queries (Liu et al., 2021a). However, 009 these methods do not consider symmetry of 010 the task. In this work, we propose Symmet-011 rical Prompt Enhancement (SPE), a continu-012 ous prompt-based method for fact retrieval that 013 leverages the symmetry of the task. Our results 014 on LAMA, a popular fact retrieval dataset, 015 show significant improvement of SPE over pre-016 vious prompt methods