Publications

On Codex Prompt Engineering for OCL Generation: An Empirical Study
Seif Abukhalaf
Mohammad Hamdaqa
Foutse Khomh
The Object Constraint Language (OCL) is a declarative language that adds constraints and object query expressions to Meta-Object Facility (M… (see more)OF) models. OCL can provide precision and conciseness to UML models. Nevertheless, the unfamiliar syntax of OCL has hindered its adoption by software practitioners. LLMs, such as GPT-3, have made significant progress in many NLP tasks, such as text generation and semantic parsing. Similarly, researchers have improved on the downstream tasks by fine-tuning LLMs for the target task. Codex, a GPT-3 descendant by OpenAI, has been fine-tuned on publicly available code from GitHub and has proven the ability to generate code in many programming languages, powering the AI-pair programmer Copilot. One way to take advantage of Codex is to engineer prompts for the target downstream task. In this paper, we investigate the reliability of the OCL constraints generated by Codex from natural language specifications. To achieve this, we compiled a dataset of 15 UML models and 168 specifications from various educational resources. We manually crafted a prompt template with slots to populate with the UML information and the target task in the prefix format to complete the template with the generated OCL constraint. We used both zero- and few-shot learning methods in the experiments. The evaluation is reported by measuring the syntactic validity and the execution accuracy metrics of the generated OCL constraints. Moreover, to get insight into how close or natural the generated OCL constraints are compared to human-written ones, we measured the cosine similarity between the sentence embedding of the correctly generated and human-written OCL constraints. Our findings suggest that by enriching the prompts with the UML information of the models and enabling few-shot learning, the reliability of the generated OCL constraints increases. Furthermore, the results reveal a close similarity based on sentence embedding between the generated OCL constraints and the human-written ones in the ground truth, implying a level of clarity and understandability in the generated OCL constraints by Codex.
2023-05-14
2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR) (published)
doi.org
arxiv.org
Conditional Permutation Invariant Flows
Berend Zwartsenberg
Adam Ścibior
Matthew Niedoba
Vasileios Lioutas
Yunpeng Liu
Justice Sefas
Setareh Dabiri
Jonathan Wilder Lavington
Trevor Campbell
Frank N. Wood
We present a novel, conditional generative probabilistic model of set-valued data with a tractable log density. This model is a continuous n… (see more)ormalizing flow governed by permutation equivariant dynamics. These dynamics are driven by a learnable per-set-element term and pairwise interactions, both parametrized by deep neural networks. We illustrate the utility of this model via applications including (1) complex traffic scene generation conditioned on visually specified map information, and (2) object bounding box generation conditioned directly on images. We train our model by maximizing the expected likelihood of labeled conditional data under our flow, with the aid of a penalty that ensures the dynamics are smooth and hence efficiently solvable. Our method significantly outperforms non-permutation invariant baselines in terms of log likelihood and domain-specific metrics (offroad, collision, and combined infractions), yielding realistic samples that are difficult to distinguish from real data.
2023-05-14
TMLR (accepted)
doi.org
openreview.net
Fast and Attributed Change Detection on Dynamic Graphs with Density of States
Shenyang Huang
Jacob Danovitch
Guillaume Rabusseau
Reihaneh Rabbany
2023-05-14
arXiv (preprint)
doi.org
arxiv.org
UnityLint: A Bad Smell Detector for Unity
Matteo Bosco
Pasquale Cavoto
Augusto Ungolo
Biruk Asmare Muse
Foutse Khomh
Vittoria Nardone
Massimiliano Di Penta
The video game industry is particularly rewarding as it represents a large portion of the software development market. However, working in t… (see more)his domain may be challenging for developers, not only because of the need for heterogeneous skills (from software design to computer graphics), but also for the limited body of knowledge in terms of good and bad design and development principles, and the lack of tool support to assist them. This tool demo proposes UnityLint, a tool able to detect 18 types of bad smells in Unity video games. UnityLint builds upon a previously-defined and validated catalog of bad smells for video games. The tool, developed in C# and available both as open-source and binary releases, is composed of (i) analyzers that extract facts from video game source code and metadata, and (ii) smell detectors that leverage detection rules to identify smells on top of the extracted facts.Tool: https://github.com/mdipenta/UnityCodeSmellAnalyzerTeaser Video: https://youtu.be/HooegxZ8H6g
2023-05-14
2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC) (published)
doi.org
MatSci-NLP: Evaluating Scientific Language Models on Materials Science Language Tasks Using Text-to-Schema Modeling
Yurun Song
Santiago Miret
Bang Liu
2023-05-13
ArXiv (preprint)
doi.org
arxiv.org
RIGAA at the SBFT 2023 Tool Competition - Cyber-Physical Systems Track
Dmytro Humeniuk
Foutse Khomh
Giuliano Antoniol
Testing and verification of autonomous systems is critically important. In the context of SBFT 2023 CPS testing tool competition, we present… (see more) our tool RIGAA for generating virtual roads to test an autonomous vehicle lane keeping assist system. RIGAA combines reinforcement learning as well as evolutionary search to generate test scenarios. It has achieved the second highest final score among 5 other submitted tools.
2023-05-13
2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT) (published)
doi.org
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava
Abhinav Rastogi
Abhishek Rao
Abu Awal Md Shoeb
Abubakar Abid
Adam Fisch
Adam R. Brown
Adam Santoro
Aditya Gupta
Adrià Garriga-Alonso
Agnieszka Kluska
Aitor Lewkowycz
Akshat Agarwal
Alethea Power
Alex Ray
Alex Warstadt
Alexander W. Kocurek
Ali Safaya
Ali Tazarv
Alice Xiang … (see 432 more)
Alicia Parrish
Allen Nie
Aman Hussain
Amanda Askell
Amanda Dsouza
Ambrose Slone
Ameet Rahane
Anantharaman S. Iyer
Anders Johan Andreassen
Andrea Madotto
Andrea Santilli
Andreas Stuhlmüller
Andrew M. Dai
Andrew La
Andrew Lampinen
Andy Zou
Angela Jiang
Angelica Chen
Anh Vuong
Animesh Gupta
Anna Gottardi
Antonio Norelli
Anu Venkatesh
Arash Gholamidavoodi
Arfa Tabassum
Arul Menezes
Arun Kirubarajan
Asher Mullokandov
Ashish Sabharwal
Austin Herrick
Avia Efrat
Aykut Erdem
Ayla Karakaş
B. Ryan Roberts
Bao Sheng Loe
Barret Zoph
Bartłomiej Bojanowski
Batuhan Özyurt
Behnam Hedayatnia
Behnam Neyshabur
Benjamin Inden
Benno Stein
Berk Ekmekci
Bill Yuchen Lin
Blake Howald
Bryan Orinion
Cameron Diao
Cameron Dour
Catherine Stinson
Cedrick Argueta
Cesar Ferri
Chandan Singh
Charles Rathkopf
Chenlin Meng
Chitta Baral
Chiyu Wu
Chris Callison-Burch
Christopher Waites
Christian Voigt
Christopher D Manning
Christopher Potts
Cindy Ramirez
Clara E. Rivera
Clemencia Siro
Colin Raffel
Courtney Ashcraft
Cristina Garbacea
Damien Sileo
Dan Garrette
Dan Hendrycks
Dan Kilman
Dan Roth
C. Daniel Freeman
Daniel Khashabi
Daniel Levy
Daniel Moseguí González
Danielle Perszyk
Danny Hernandez
Danqi Chen
Daphne Ippolito
Dar Gilboa
David Dohan
David Drakard
David Jurgens
Debajyoti Datta
Deep Ganguli
Denis Emelin
Denis Kleyko
Deniz Yuret
Derek Chen
Derek Tam
Dieuwke Hupkes
Diganta Misra
Dilyar Buzan
Dimitri Coelho Mollo
Diyi Yang
Dong-Ho Lee
Dylan Schrader
Ekaterina Shutova
Ekin Dogus Cubuk
Elad Segal
Eleanor Hagerman
Elizabeth Barnes
Elizabeth Donoway
Ellie Pavlick
Emanuele Rodolà
Emma Lam
Eric Chu
Eric Tang
Erkut Erdem
Ernie Chang
Ethan A Chi
Ethan Dyer
Ethan Jerzak
Ethan Kim
Eunice Engefu Manyasi
Evgenii Zheltonozhskii
Fanyue Xia
Fatemeh Siar
Fernando Martínez-Plumed
Francesca Happé
François Chollet
Frieda Rong
Gaurav Mishra
Genta Indra Winata
Gerard de Melo
Germán Kruszewski
Giambattista Parascandolo
Giorgio Mariani
Gloria Xinyue Wang
Gonzalo Jaimovitch-Lopez
Gregor Betz
Guy Gur-Ari
Hana Galijasevic
Hannah Kim
Hannah Rashkin
Hannaneh Hajishirzi
Harsh Mehta
Hayden Bogar
Henry Shevlin
Henry Francis Anthony Shevlin
Hinrich Schuetze
Hiromu Yakura
Hongming Zhang
Hugh Mee Wong
Ian Ng
Isaac Noble
Jaap Jumelet
Jack Geissinger
Jackson Kernion
Jacob Hilton
Jaehoon Lee
Jaime Fernández Fisac
James B Simon
James Koppel
James Zheng
James Zou
Jan Kocon
Jana Thompson
Janelle Wingfield
Jared Kaplan
Jarema Radom
Jascha Sohl-Dickstein
Jason Phang
Jason Wei
Jason Yosinski
Jekaterina Novikova
Jelle Bosscher
Jennifer Marsh
Jeremy Kim
Jeroen Taal
Jesse Engel
Jesujoba Oluwadara Alabi
Jiacheng Xu
Jiaming Song
Jillian Tang
Joan Waweru
John Burden
John Miller
John U. Balis
Jonathan Batchelder
Jonathan Berant
Jörg Frohberg
Jos Rozen
Jose Hernandez-Orallo
Joseph Boudeman
Joseph Guerr
Joseph Jones
Joshua B. Tenenbaum
Joshua S. Rule
Joyce Chua
Kamil Kanclerz
Karen Livescu
Karl Krauth
Karthik Gopalakrishnan
Katerina Ignatyeva
Katja Markert
Kaustubh Dhole
Kevin Gimpel
Kevin Omondi
Kory Mathewson
Kristen Chiafullo
Ksenia Shkaruta
Kumar Shridhar
Kyle McDonell
Kyle Richardson
Laria Reynolds
Leo Gao
Ling Zhang
Liam Dugan
Lianhui Qin
Lidia Contreras-Ochando
Louis-Philippe Morency
Luca Moschella
Lucas Lam
Lucy Noble
Ludwig Schmidt
Luheng He
Luis Oliveros-Colón
Luke Metz
Lütfi Kerem Senel
Maarten Bosma
Maarten Sap
Maartje Ter Hoeve
Maheen Farooqi
Manaal Faruqui
Mantas Mazeika
Marco Baturan
Marco Marelli
Marco Maru
Maria Jose Ramirez-Quintana
Marie Tolkiehn
Mario Giulianelli
Martha Lewis
Martin Potthast
Matthew L Leavitt
Matthias Hagen
Mátyás Schubert
Medina Orduna Baitemirova
Melody Arnaud
Melvin McElrath
Michael Andrew Yee
Michael Cohen
Michael Gu
Michael Ivanitskiy
Michael Starritt
Michael Strube
Michał Swędrowski
Michele Bevilacqua
Michihiro Yasunaga
Mihir Kale
Mike Cain
Mimee Xu
Mirac Suzgun
Mitch Walker
Mo Tiwari
Mohit Bansal
Moin Aminnaseri
Mor Geva
Mozhdeh Gheini
Mukund Varma T
Nanyun Peng
Nathan Andrew Chi
Nayeon Lee
Neta Gur-Ari Krakover
Nicholas Cameron
Nicholas Roberts
Nick Doiron
Nicole Martinez
Nikita Nangia
Niklas Deckers
Niklas Muennighoff
Nitish Shirish Keskar
Niveditha S. Iyer
Noah Constant
Noah Fiedel
Nuan Wen
Oliver Zhang
Omar Agha
Omar Elbaghdadi
Omer Levy
Owain Evans
Pablo Antonio Moreno Casares
Parth Doshi
Pascale Fung
Paul Pu Liang
Paul Vicol
Pegah Alipoormolabashi
Peiyuan Liao
Percy Liang
Peter W Chang
Peter Eckersley
Phu Mon Htut
Pinyu Hwang
Pi-Bei Hwang
Piotr Miłkowski
Piyush Patil
Pouya Pezeshkpour
Priti Oli
Qiaozhu Mei
Qing Lyu
Qinlang Chen
Rabin Banjade
Rachel Etta Rudolph
Raefer Gabriel
Rahel Habacker
Ramon Risco
Raphaël Millière
Rhythm Garg
Richard Barnes
Rif A. Saurous
Riku Arakawa
Robbe Raymaekers
Robert Frank
Rohan Sikand
Roman Novak
Roman Sitelew
Ronan Le Bras
Rosanne Liu
Rowan Jacobs
Rui Zhang
Russ Salakhutdinov
Ryan Andrew Chi
Seungjae Ryan Lee
Ryan Stovall
Ryan Teehan
Rylan Yang
Sahib Singh
Saif Mohammad
Sajant Anand
Sam Dillavou
Sam Shleifer
Sam Wiseman
Samuel Gruetter
Samuel R. Bowman
Samuel Stern Schoenholz
Sanghyun Han
Sanjeev Kwatra
Sarah A. Rous
Sarik Ghazarian
Sayan Ghosh
Sean Casey
Sebastian Bischoff
Sebastian Gehrmann
Sebastian Schuster
Sepideh Sadeghi
Shadi Hamdan
Sharon Zhou
Shashank Srivastava
Sherry Shi
Shikhar Singh
Shima Asaadi
Shixiang Shane Gu
Shubh Pachchigar
Shubham Toshniwal
Shyam Upadhyay
Shyamolima Shammie Debnath
Siamak Shakeri
Simon Thormeyer
Simone Melzi
Siva Reddy
Sneha Priscilla Makini
Soo-Hwan Lee
Spencer Torene
Sriharsha Hatwar
Stanislas Dehaene
Stefan Divic
Stefano Ermon
Stella Biderman
Stephanie Lin
Stephen Prasad
Steven Piantadosi
Stuart Shieber
Summer Misherghi
Svetlana Kiritchenko
Swaroop Mishra
Tal Linzen
Tal Schuster
Tao Li
Tao Yu
Tariq Ali
Tatsunori Hashimoto
Te-Lin Wu
Théo Desbordes
Theodore Rothschild
Thomas Phan
Tianle Wang
Tiberius Nkinyili
Timo Schick
Timofei Kornev
Titus Tunduny
Tobias Gerstenberg
Trenton Chang
Trishala Neeraj
Tushar Khot
Tyler Shultz
Uri Shaham
Vedant Misra
Vera Demberg
Victoria Nyamai
Vikas Raunak
Vinay Venkatesh Ramasesh
vinay uday prabhu
Vishakh Padmakumar
Vivek Srikumar
William Fedus
William Saunders
William Zhang
Wout Vossen
Xiang Ren
Xiaoyu Tong
Xinran Zhao
Xinyi Wu
Xudong Shen
Yadollah Yaghoobzadeh
Yair Lakretz
Yangqiu Song
Yasaman Bahri
Yejin Choi
Yichi Yang
Sophie Hao
Yiding Hao
Yifu Chen
Yonatan Belinkov
Yufang Hou
Yuntao Bai
Zachary Seid
Zhuoye Zhao
Zijian Wang
Zijie J. Wang
Zirui Wang
Ziyi Wu
2023-05-10
TMLR (accepted)
openreview.net
openreview.net
Domain Agnostic Image-to-image Translation using Low-Resolution Conditioning
Mohamed Abderrahmen Abid
Arman Afrasiyabi
Ihsen Hedhli
Jean‐François Lalonde
Christian Gagné
2023-05-07
ArXiv (preprint)
doi.org
arxiv.org
ConceptFusion: Open-set Multimodal 3D Mapping
Krishna Murthy
Alihusein Kuwajerwala
Qiao Gu
Mohd Omama
Tao Chen
Alaa Maalouf
Shuang Li
Ganesh Subramanian Iyer
Soroush Saryazdi
Nikhil Varma Keetha
Ayush Tewari
Joshua B. Tenenbaum
Celso M de Melo
Madhava Krishna
Liam Paull
Florian Shkurti
Antonio Torralba
Building 3D maps of the environment is central to robot navigation, planning, and interaction with objects in a scene. Most existing approac… (see more)hes that integrate semantic concepts with 3D maps largely remain confined to the closed-set setting: they can only reason about a finite set of concepts, pre-defined at training time. Further, these maps can only be queried using class labels, or in recent work, using text prompts. We address both these issues with ConceptFusion, a scene representation that is: (i) fundamentally open-set, enabling reasoning beyond a closed set of concepts (ii) inherently multi-modal, enabling a diverse range of possible queries to the 3D map, from language, to images, to audio, to 3D geometry, all working in concert. ConceptFusion leverages the open-set capabilities of today’s foundation models pre-trained on internet-scale data to reason about concepts across modalities such as natural language, images, and audio. We demonstrate that pixel-aligned open-set features can be fused into 3D maps via traditional SLAM and multi-view fusion approaches. This enables effective zero-shot spatial reasoning, not needing any additional training or finetuning, and retains long-tailed concepts better than supervised approaches, outperforming them by more than 40% margin on 3D IoU. We extensively evaluate ConceptFusion on a number of real-world datasets, simulated home environments, a real-world tabletop manipulation task, and an autonomous driving platform. We showcase new avenues for blending foundation models with 3D open-set multimodal mapping.
2023-05-05
ICRA.org/2023/Workshop/Pretraining4Robotics (published)
doi.org
openreview.net
Scope Restriction for Scalable Real-Time Railway Rescheduling: An Exploratory Study
Erik L. Nygren
Christian Eichenberger
Emma Frejinger
With the aim to stimulate future research, we describe an exploratory study of a railway rescheduling problem. A widely used approach in pra… (see more)ctice and state of the art is to decompose these complex problems by geographical scope. Instead, we propose defining a core problem that restricts a rescheduling problem in response to a disturbance to only trains that need to be rescheduled, hence restricting the scope in both time and space. In this context, the difficulty resides in defining a scoper that can predict a subset of train services that will be affected by a given disturbance. We report preliminary results using the Flatland simulation environment that highlights the potential and challenges of this idea. We provide an extensible playground open-source implementation based on the Flatland railway environment and Answer-Set Programming.
2023-05-04
ArXiv (preprint)
doi.org
arxiv.org
Single-cell analysis reveals inflammatory interactions driving macular degeneration
Manik Kuchroo
Marcello DiStasio
Eric Song
Eda Calapkulu
Le Zhang
Maryam Ige
Amar H. Sheth
Abdelilah Majdoubi
Madhvi Menon
Alexander Tong
Abhinav Godavarthi
Yu Xing
Scott Gigante
Holly R. Steach
Jessie Huang
Guillaume Huguet
Janhavi Narain
Kisung You
George Mourgkos
Rahul M. Dhodapkar … (see 5 more)
Matthew Hirn
Bastian Rieck
Guy Wolf
Smita Krishnaswamy
Brian P. Hafler
Due to commonalities in pathophysiology, age-related macular degeneration (AMD) represents a uniquely accessible model to investigate thera… (see more)pies for neurodegenerative diseases, leading us to examine whether pathways of disease progression are shared across neurodegenerative conditions. Here we use single-nucleus RNA sequencing to profile lesions from 11 postmortem human retinas with age-related macular degeneration and 6 control retinas with no history of retinal disease. We create a machine-learning pipeline based on recent advances in data geometry and topology and identify activated glial populations enriched in the early phase of disease. Examining single-cell data from Alzheimer’s disease and progressive multiple sclerosis with our pipeline, we find a similar glial activation profile enriched in the early phase of these neurodegenerative diseases. In late-stage age-related macular degeneration, we identify a microglia-to-astrocyte signaling axis mediated by interleukin-1β which drives angiogenesis characteristic of disease pathogenesis. We validated this mechanism using in vitro and in vivo assays in mouse, identifying a possible new therapeutic target for AMD and possibly other neurodegenerative conditions. Thus, due to shared glial states, the retina provides a potential system for investigating therapeutic approaches in neurodegenerative diseases.
2023-05-04
Nature Communications (published)
doi.org
Improving Ecological Connectivity Assessments with Transfer Learning and Function Approximation
Michael D. Catchen
Michelle Lin
Timothée Poisot
David Rolnick
Andrew Gonzalez
This is a conference paper presented at the ICLR 2023 "Machine Learning for Remote Sensing" workshop.Protecting and restoring ecological con… (see more)nectivity is essential to climate change adaptation, and necessary if species are to shift their geographic distributions to track their suitable climatic conditions over the coming century. Despite the increasing availability of near real-time and high resolution data for landcover change, current connectivity planning projects are hindered by the computational time required to run connectivity analyses at realistic geographic scales with realistic models of movement. This bottleneck precludes application of optimization algorithms to prioritize ecological restoration to maintain and improve connectivity. Here we propose we can make progress toward overcoming these challenges using machine-learning methods. Our proposed methods will enable rapid optimization of connectivity prioritization and extend its application to many more species than is currently possible. We conclude by illustrating how this project will contribute to efforts to apply connectivity conservation using an example of ongoing restoration in southern Québec.
2023-05-03
Machine Learning for Remote Sensing @ ICLR (published)
doi.org