Publications

Idiolect: A Reconfigurable Voice Coding Assistant
Breandan Considine
Nicholas Albion
Xujie Si
This paper presents Idiolect, an open source 1 IDE plugin for voice coding and a novel approach to building bots that allows for users to de… (voir plus)fine custom commands on-the-fly. Unlike traditional chatbots, Idiolect does not pretend to be an omniscient virtual assistant but rather a reconfigurable voice programming system that empowers users to create their own commands and actions dynamically, without rebuilding or restarting the application. We offer an experience report describing the tool itself, illustrate some example use cases, and reflect on several lessons learned during the tool’s development.
2023-05-19
2023 IEEE/ACM 5th International Workshop on Bots in Software Engineering (BotSE) (publié)
doi.org
arxiv.org
Neural Bee Colony Optimization: A Case Study in Public Transit Network Design
Andrew Holliday
Gregory Dudek
In this work we explore the combination of metaheuristics and learned neural network solvers for combinatorial optimization. We do this in t… (voir plus)he context of the transit network design problem, a uniquely challenging combinatorial optimization problem with real-world importance. We train a neural network policy to perform single-shot planning of individual transit routes, and then incorporate it as one of several sub-heuristics in a modified Bee Colony Optimization (BCO) metaheuristic algorithm. Our experimental results demonstrate that this hybrid algorithm outperforms the learned policy alone by up to 20% and the original BCO algorithm by up to 53% on realistic problem instances. We perform a set of ablations to study the impact of each component of the modified algorithm.
2023-05-17
ArXiv (prépublication)
doi.org
arxiv.org
NollySenti: Leveraging Transfer Learning and Machine Translation for Nigerian Movie Sentiment Classification
Iyanuoluwa Shode
David Ifeoluwa Adelani
Jing Peng
Anna Feldman
Africa has over 2000 indigenous languages but they are under-represented in NLP research due to lack of datasets. In recent years, there hav… (voir plus)e been progress in developing labelled corpora for African languages. However, they are often available in a single domain and may not generalize to other domains. In this paper, we focus on the task of sentiment classification for cross-domain adaptation. We create a new dataset, Nollywood movie reviews for five languages widely spoken in Nigeria (English, Hausa, Igbo, Nigerian Pidgin, and Yoruba). We provide an extensive empirical evaluation using classical machine learning methods and pre-trained language models. By leveraging transfer learning, we compare the performance of cross-domain adaptation from Twitter domain, and cross-lingual adaptation from English language. Our evaluation shows that transfer from English in the same target domain leads to more than 5% improvement in accuracy compared to transfer from Twitter in the same language. To further mitigate the domain difference, we leverage machine translation from English to other Nigerian languages, which leads to a further improvement of 7% over cross-lingual evaluation. While machine translation to low-resource languages are often of low quality, our analysis shows that sentiment related words are often preserved.
2023-05-17
ArXiv (prépublication)
doi.org
arxiv.org
The potential for co-operatives to mitigate AI ethics catastrophes: perspectives from media analysis
David Marino
AJung Moon
Would the world have seen less AI-related scandals if more AI companies operated as co-operatives? As a response to multiple high profile te… (voir plus)ch scandals within the last decade, there has been an increased call for introducing more accountability in the AI industry. However, it is unclear to what degree the proposed efforts have been or will be effective in practice. The question remains whether these incremental, multi-stakeholder AI ethics efforts are in fact trying to address a fundamentally systemic issue inherent to the existing corporate power structure. As an attempt to address this question, we gain an understanding of the major themes in high profile AI-related catastrophes in the last four years (2018–2021) through an inductive media analysis. We then investigate how the principle of democratic gov-ernance and distributive executive power - core to co-operative organization structure - could have prevented or mitigated the contributing factors of the reported events. We find that the vast majority (71%) of the recent AI ethics scandals are not the result of a lack of knowledge or tools, but attributed to power dynamics that hinder the ability of internal stakeholders from taking action. We present the co-operative governance structure as a possible mitigating solution to addressing future AI ethics catastrophes, and provide a critical look at practical challenges inherent to AI co-operatives.
2023-05-17
Ethics: An International Journal of Social, Political, and Legal Philosophy (publié)
doi.org
A MC-based anthropomorphic test case for commissioning model-based dose calculation in interstitial breast 192-Ir HDR brachytherapy.
Vasiliki Peppa
Rowan M. Thomson
S. Enger
Gabriel P. Fonseca
Choonik Lee
Joseph N. E. Lucero
Firas Mourtada
Frank‐André Siebert
Javier Vijande
Panagiotis Papagiannis
PURPOSE To provide the first clinical test case for commissioning of 192 Ir brachytherapy model-based dose calculation algorithms (MBDCAs) a… (voir plus)ccording to the AAPM TG-186 report workflow. ACQUISITION AND VALIDATION METHODS A computational patient phantom model was generated from a clinical multi-catheter 192 Ir HDR breast brachytherapy case. Regions of interest (ROIs) were contoured and digitized on the patient CT images and the model was written to a series of DICOM CT images using MATLAB. The model was imported into two commercial treatment planning systems (TPSs) currently incorporating an MBDCA. Identical treatment plans were prepared using a generic 192 Ir HDR source and the TG-43-based algorithm of each TPS. This was followed by dose to medium in medium calculations using the MBDCA option of each TPS. Monte Carlo (MC) simulation was performed in the model using three different codes and information parsed from the treatment plan exported in DICOM radiation therapy (RT) format. Results were found to agree within statistical uncertainty and the dataset with the lowest uncertainty was assigned as the reference MC dose distribution. DATA FORMAT AND USAGE NOTES The dataset is available online at http://irochouston.mdanderson.org/rpc/BrachySeeds/BrachySeeds/index.html,https://doi.org/10.52519/00005. Files include the treatment plan for each TPS in DICOM RT format, reference MC dose data in RT Dose format, as well as a guide for database users and all files necessary to repeat the MC simulations. POTENTIAL APPLICATIONS The dataset facilitates the commissioning of brachytherapy MBDCAs using TPS embedded tools and establishes a methodology for the development of future clinical test cases. It is also useful to non-MBDCA adopters for intercomparing MBDCAs and exploring their benefits and limitations, as well as to brachytherapy researchers in need of a dosimetric and/or a DICOM RT information parsing benchmark. Limitations include specificity in terms of radionuclide, source model, clinical scenario, and MBDCA version used for its preparation.
2023-05-16
Medical Physics (publié)
doi.org
Raising the Bar for Certified Adversarial Robustness with Diffusion Models
Thomas Altstidl
David Dobre
Bjoern Eskofier
Gauthier Gidel
Leo Schwinn
Certified defenses against adversarial attacks offer formal guarantees on the robustness of a model, making them more reliable than empirica… (voir plus)l methods such as adversarial training, whose effectiveness is often later reduced by unseen attacks. Still, the limited certified robustness that is currently achievable has been a bottleneck for their practical adoption. Gowal et al. and Wang et al. have shown that generating additional training data using state-of-the-art diffusion models can considerably improve the robustness of adversarial training. In this work, we demonstrate that a similar approach can substantially improve deterministic certified defenses. In addition, we provide a list of recommendations to scale the robustness of certified training approaches. One of our main insights is that the generalization gap, i.e., the difference between the training and test accuracy of the original model, is a good predictor of the magnitude of the robustness improvement when using additional generated data. Our approach achieves state-of-the-art deterministic robustness certificates on CIFAR-10 for the
2023-05-16
arXiv (prépublication)
doi.org
arxiv.org
Responses of pyramidal cell somata and apical dendrites in mouse visual cortex over multiple days
Colleen J. Gillon
Jérôme A. Lecoq
Jason E. Pina
Ruweida Ahmed
Yazan N. Billeh
Shiella Caldejon
Peter Groblewski
Timothy M. Henley
India Kato
Eric Lee
Jennifer Luviano
Kyla Mace
Chelsea Nayan
Thuyanh V. Nguyen
Kat North
Jed Perkins
Sam Seid
Matthew T. Valley
Ali Williford
Yoshua Bengio … (voir 3 de plus)
Timothy P. Lillicrap
Joel Zylberberg
Blake A. Richards
The apical dendrites of pyramidal neurons in sensory cortex receive primarily top-down signals from associative and motor regions, while cel… (voir plus)l bodies and nearby dendrites are heavily targeted by locally recurrent or bottom-up inputs from the sensory periphery. Based on these differences, a number of theories in computational neuroscience postulate a unique role for apical dendrites in learning. However, due to technical challenges in data collection, little data is available for comparing the responses of apical dendrites to cell bodies over multiple days. Here we present a dataset collected through the Allen Institute Mindscope’s OpenScope program that addresses this need. This dataset comprises high-quality two-photon calcium imaging from the apical dendrites and the cell bodies of visual cortical pyramidal neurons, acquired over multiple days in awake, behaving mice that were presented with visual stimuli. Many of the cell bodies and dendrite segments were tracked over days, enabling analyses of how their responses change over time. This dataset allows neuroscientists to explore the differences between apical and somatic processing and plasticity.
2023-05-16
Scientific Data (publié)
doi.org
Eliminating Space Scanning: Fast mmWave Beam Alignment with UWB Radios
Ju Wang
X. T. Chen
Xue Liu
Gregory Dudek
Due to their large bandwidth and impressive data speed, millimeter-wave (mmWave) radios are expected to play a key role in the 5G and beyond… (voir plus) (e.g., 6G) communication networks. Yet, to release mmWave’s true power, the highly directional mmWave beams need to be aligned perfectly. Most existing beam alignment methods adopt an exhaustive or semi-exhaustive space scanning, which introduces up to seconds of delays. To eliminate the need for complex space scanning, this article presents an Ultra-wideband (UWB)-assisted mmWave communication framework, which leverages the co-located UWB antennas to estimate the best angles for mmWave beam alignment. One major challenge of applying this idea in the real world is the barrier of limited antenna numbers. Commercial-Off-The-Shelf (COTS) devices are usually equipped with only a small number of UWB antennas, which are not enough for the existing algorithms to provide an accurate angle estimation. To solve this challenge, we design a novel Multi-Frequency MUltiple SIgnal Classification (MF-MUSIC) algorithm, which extends the classic MUltiple SIgnal Classification (MUSIC) algorithm to the frequency domain and overcomes the antenna limitation barrier in the spatial domain. Extensive real-world experiments and numerical simulations illustrate the advantage of the proposed MF-MUSIC algorithm. MF-MUSIC uses only three antennas to achieve an accurate angle estimation, which is a mere 0.15° (or a relative difference of 3.6%) different from the state-of-the-art 16-antenna-based angle estimation method.
2023-05-15
ACM Transactions on Sensor Networks (publié)
doi.org
On Codex Prompt Engineering for OCL Generation: An Empirical Study
Seif Abukhalaf
Mohammad Hamdaqa
Foutse Khomh
The Object Constraint Language (OCL) is a declarative language that adds constraints and object query expressions to Meta-Object Facility (M… (voir plus)OF) models. OCL can provide precision and conciseness to UML models. Nevertheless, the unfamiliar syntax of OCL has hindered its adoption by software practitioners. LLMs, such as GPT-3, have made significant progress in many NLP tasks, such as text generation and semantic parsing. Similarly, researchers have improved on the downstream tasks by fine-tuning LLMs for the target task. Codex, a GPT-3 descendant by OpenAI, has been fine-tuned on publicly available code from GitHub and has proven the ability to generate code in many programming languages, powering the AI-pair programmer Copilot. One way to take advantage of Codex is to engineer prompts for the target downstream task. In this paper, we investigate the reliability of the OCL constraints generated by Codex from natural language specifications. To achieve this, we compiled a dataset of 15 UML models and 168 specifications from various educational resources. We manually crafted a prompt template with slots to populate with the UML information and the target task in the prefix format to complete the template with the generated OCL constraint. We used both zero- and few-shot learning methods in the experiments. The evaluation is reported by measuring the syntactic validity and the execution accuracy metrics of the generated OCL constraints. Moreover, to get insight into how close or natural the generated OCL constraints are compared to human-written ones, we measured the cosine similarity between the sentence embedding of the correctly generated and human-written OCL constraints. Our findings suggest that by enriching the prompts with the UML information of the models and enabling few-shot learning, the reliability of the generated OCL constraints increases. Furthermore, the results reveal a close similarity based on sentence embedding between the generated OCL constraints and the human-written ones in the ground truth, implying a level of clarity and understandability in the generated OCL constraints by Codex.
2023-05-14
2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR) (publié)
doi.org
arxiv.org
Conditional Permutation Invariant Flows
Berend Zwartsenberg
Adam Ścibior
Matthew Niedoba
Vasileios Lioutas
Yunpeng Liu
Justice Sefas
Setareh Dabiri
Jonathan Wilder Lavington
Trevor Campbell
Frank N. Wood
We present a novel, conditional generative probabilistic model of set-valued data with a tractable log density. This model is a continuous n… (voir plus)ormalizing flow governed by permutation equivariant dynamics. These dynamics are driven by a learnable per-set-element term and pairwise interactions, both parametrized by deep neural networks. We illustrate the utility of this model via applications including (1) complex traffic scene generation conditioned on visually specified map information, and (2) object bounding box generation conditioned directly on images. We train our model by maximizing the expected likelihood of labeled conditional data under our flow, with the aid of a penalty that ensures the dynamics are smooth and hence efficiently solvable. Our method significantly outperforms non-permutation invariant baselines in terms of log likelihood and domain-specific metrics (offroad, collision, and combined infractions), yielding realistic samples that are difficult to distinguish from real data.
2023-05-14
TMLR (accepté)
doi.org
openreview.net
Fast and Attributed Change Detection on Dynamic Graphs with Density of States
Shenyang Huang
Jacob Danovitch
Guillaume Rabusseau
Reihaneh Rabbany
2023-05-14
arXiv (prépublication)
doi.org
arxiv.org
UnityLint: A Bad Smell Detector for Unity
Matteo Bosco
Pasquale Cavoto
Augusto Ungolo
Biruk Asmare Muse
Foutse Khomh
Vittoria Nardone
Massimiliano Di Penta
The video game industry is particularly rewarding as it represents a large portion of the software development market. However, working in t… (voir plus)his domain may be challenging for developers, not only because of the need for heterogeneous skills (from software design to computer graphics), but also for the limited body of knowledge in terms of good and bad design and development principles, and the lack of tool support to assist them. This tool demo proposes UnityLint, a tool able to detect 18 types of bad smells in Unity video games. UnityLint builds upon a previously-defined and validated catalog of bad smells for video games. The tool, developed in C# and available both as open-source and binary releases, is composed of (i) analyzers that extract facts from video game source code and metadata, and (ii) smell detectors that leverage detection rules to identify smells on top of the extracted facts.Tool: https://github.com/mdipenta/UnityCodeSmellAnalyzerTeaser Video: https://youtu.be/HooegxZ8H6g
2023-05-14
2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC) (publié)
doi.org