Publications

TopiOCQA: Open-domain Conversational Question Answering with Topic Switching
Vaibhav Adlakha
Shehzaad Dhuliawala
Kaheer Suleman
Harm de Vries
Siva Reddy
2022-04-12
Transactions of the Association for Computational Linguistics (published)
doi.org
arxiv.org
Can neurogenesis act as a neural regularizer?
Lina M. Tran
Adam Santoro
Lulu Liu
Sheena A. Josselyn
Blake A. Richards
Paul W. Frankland
New neurons are continuously generated in the subgranular zone of the dentate gyrus throughout adulthood. These new neurons gradually integr… (see more)ate into hippocampal circuits, forming new naïve synapses. Viewed from this perspective, these new neurons may represent a significant source of ‘wiring’ noise in hippocampal networks. In machine learning, such noise injection is commonly used as a regularization technique. Regularization techniques help prevent overfitting training data, and allow models to generalize learning to new, unseen data. Using a computational modeling approach, here we ask whether a neurogenesis-like process similarly acts as a regularizer, facilitating generalization in a category learning task. In a convolutional neural network (CNN) trained on the CIFAR-10 object recognition dataset, we modeled neurogenesis as a replacement/turnover mechanism, where weights for a randomly chosen small subset of neurons in a chosen hidden layer were re-initialized to new values as the model learned to categorize 10 different classes of objects. We found that neurogenesis enhanced generalization on unseen test data compared to networks with no neurogenesis. Moreover, neurogenic networks either outperformed or performed similarly to networks with conventional noise injection (i.e., dropout, weight decay, and neural noise). These results suggest that neurogenesis can enhance generalization in hippocampal learning through noise-injection, expanding on the roles that neurogenesis may have in cognition. In deep neural networks, various forms of noise injection are used as regularization techniques to prevent overfitting and promote generalization on unseen test data. Here, we were interested in whether adult neurogenesis– the lifelong production of new neurons in the hippocampus– might similarly function as a regularizer in the brain. We explored this question computationally, assessing whether implementing a neurogenesis-like process in a hidden layer within a convolutional neural network trained in a category learning task would prevent overfitting and promote generalization. We found that neurogenesis regularization was as least as effective as, or more effective than, conventional regularizers (i.e., dropout, weight decay and neural noise) in improving model performance. These results suggest that optimal levels of hippocampal neurogenesis may improve memory-guided decision making by preventing overfitting, thereby promoting the formation of more generalized memories that can be applied in a broader range of circumstances. We outline how these predictions may be evaluated behaviorally in rodents with altered hippocampal neurogenesis.
2022-04-09
bioRxiv (preprint)
doi.org
How to certify machine learning based safety-critical systems? A systematic literature review
Florian Tambon
Gabriel Laberge
Le An
Amin Nikanjam
Paulina Stevia Nouwou Mindom
Yann Pequignot
Foutse Khomh
Giulio Antoniol
Ettore Merlo
Francois Laviolette
2022-04-09
IEEE/ACM Conference on Automated Software Engineering (published)
doi.org
arxiv.org
GCNFusion: An efficient graph convolutional network based model for information diffusion
Bahareh Fatemi
Soheila Molaei
Shirui Pan
Samira Abbasgholizadeh Rahimi
2022-04-08
Expert Systems with Applications (unknown)
doi.org
Clones in Deep Learning Code: What, Where, and Why?
Hadhemi Jebnoun
Md Saidur Rahman
Foutse Khomh
Biruk Asmare Muse
Deep Learning applications are becoming increasingly popular. Developers of deep learning systems strive to write more efficient code. Deep … (see more)learning systems are constantly evolving, imposing tighter development timelines and increasing complexity, which may lead to bad design decisions. A copy-paste approach is widely used among deep learning developers because they rely on common frameworks and duplicate similar tasks. Developers often fail to properly propagate changes to all clones fragments during a maintenance activity. To our knowledge, no study has examined code cloning practices in deep learning development. Given the negative impacts of clones on software quality reported in the studies on traditional systems, it is very important to understand the characteristics and potential impacts of code clones on deep learning systems. To this end, we use the NiCad tool to detect clones from 59 Python, 14 C# and 6 Java-based deep learning systems and an equal number of traditional software systems. We then analyze the frequency and distribution of code clones in deep learning and traditional systems. We do further analysis of the distribution of code clones using location-based taxonomy. We also study the correlation between bugs and code clones to assess the impacts of clones on the quality of the studied systems. Finally, we introduce a code clone taxonomy related to deep learning programs and identify the deep learning system development phases in which cloning has the highest risk of faults. Our results show that code cloning is a frequent practice in deep learning systems and that deep learning developers often clone code from files in distant repositories in the system. In addition, we found that code cloning occurs more frequently during DL model construction. And that hyperparameters setting is the phase during which cloning is the riskiest, since it often leads to faults.
2022-04-07
Empirical Software Engineering (published)
doi.org
arxiv.org
Rapid, automated nerve histomorphometry through open-source artificial intelligence
Simeon Christian Daeschler
Marie-Hélène Bourget
Dorsa Derakhshan
Vasudev Sharma
Stoyan Ivaylov Asenov
Tessa Gordon
Julien Cohen-Adad
Gregory Howard Borschel
We aimed to develop and validate a deep learning model for automated segmentation and histomorphometry of myelinated peripheral nerve fibers… (see more) from light microscopic images. A convolutional neural network integrated in the AxonDeepSeg framework was trained for automated axon/myelin segmentation using a dataset of light-microscopic cross-sectional images of osmium tetroxide-stained rat nerves including various axonal regeneration stages. In a second dataset, accuracy of automated segmentation was determined against manual axon/myelin labels. Automated morphometry results, including axon diameter, myelin sheath thickness and g-ratio were compared against manual straight-line measurements and morphometrics extracted from manual labels with AxonDeepSeg as a reference standard. The neural network achieved high pixel-wise accuracy for nerve fiber segmentations with a mean (± standard deviation) ground truth overlap of 0.93 (± 0.03) for axons and 0.99 (± 0.01) for myelin sheaths, respectively. Nerve fibers were identified with a sensitivity of 0.99 and a precision of 0.97. For each nerve fiber, the myelin thickness, axon diameter, g-ratio, solidity, eccentricity, orientation, and individual x -and y-coordinates were determined automatically. Compared to manual morphometry, automated histomorphometry showed superior agreement with the reference standard while reducing the analysis time to below 2.5% of the time needed for manual morphometry. This open-source convolutional neural network provides rapid and accurate morphometry of entire peripheral nerve cross-sections. Given its easy applicability, it could contribute to significant time savings in biomedical research while extracting unprecedented amounts of objective morphologic information from large image datasets.
2022-04-07
Scientific Reports (published)
doi.org
Predicting Visual Improvement After Macular Hole Surgery: A Combined Model Using Deep Learning and Clinical Features
Alexandre Lachance
Mathieu Godbout
Fares Antaki
Mélanie Hébert
Serge Bourgault
Mathieu Caissie
Éric Tourville
Audrey Durand
Ali Dirani
2022-04-05
Translational Vision Science & Technology (published)
doi.org
Comparison of multi-center MRI protocols for visualizing the spinal cord gray matter
Julien Cohen-Adad
Eva Alonso-Ortiz
Stephanie Alley
Maria Marcella Laganà
Francesca Baglio
Signe Johanna Vannesjo
Haleh Karbasforoushan
Maryam Seif
Alan C. Seifert
Junqian Xu
Joo-Won Kim
René Labounek
Lubomír Vojtíšek
Marek Dostál
Jan Valosek
Rebecca S. Samson
Francesco Grussu
Marco Battiston
Claudia A. M. Gandini Wheeler-Kingshott
Marios C. Yiannakas … (see 4 more)
Guillaume Gilbert
Torben Schneider
Brian Johnson
Ferrán Prados
We propose quality assessment criteria and metrics for gray‐matter visualization and apply them to different protocols. The proposed crite… (see more)ria and metrics, the analyzed protocols, and our open‐source code can serve as a benchmark for future optimization of spinal cord gray‐matter imaging protocols.
2022-04-04
Magnetic Resonance in Medicine (unknown)
doi.org
E VALUATING G ENERALIZATION IN GF LOW N ETS FOR M OLECULE D ESIGN
Andrei Cristian Nica
Moksh J. Jain
Emmanuel Bengio
Cheng-Hao Liu
Maksym Korablyov
Michael M. Bronstein
Yoshua Bengio
Deep learning bears promise for drug discovery problems such as de novo molecular design. Generating data to train such models is a costly a… (see more)nd time-consuming process, given the need for wet-lab experiments or expensive simulations. This problem is compounded by the notorious data-hungriness of machine learning algorithms. In small molecule generation the recently proposed GFlowNet method has shown good performance in generating diverse high-scoring candidates, and has the interesting advantage of being an off-policy offline method. Finding an appropriate generalization evaluation metric for such models, one predictive of the desired search performance (i.e. finding high-scoring diverse candidates), will help guide online data collection for such an algorithm. In this work, we develop techniques for evaluating GFlowNet performance on a test set, and identify the most promising metric for predicting generalization. We present empirical results on several small-molecule design tasks in drug discovery, for several GFlowNet training setups, and we find a metric strongly correlated with diverse high-scoring batch generation. This metric should be used to identify the best generative model from which to sample batches of molecules to be evaluated.
2022-04-04
ICLR.cc/2022/Workshop/MLDD (poster)
openreview.net
openreview.net
Application of AI in community based primary health care: Systematic review and critical appraisal
S. A. Rahimi
Patrick Archambault
Herve Tchala Vignon Zomahoun
Sam Chandavong
Marie-Pierre Gagnon
Sabrina M. Wong
Gauri Sharma
Lyse Langlois
Nathalie Rheault
Yves Couturier
Jean Légaré
2022-04-03
Systematic review, meta-analysis, or scoping review (published)
doi.org
TRACKING AND PREDICTING COVID-19 RADIOLOGICAL TRAJECTORY USING DEEP LEARNING ON CHEST X-RAYS: INITIAL ACCURACY TESTING
Simon Duchesne
Daniel Gourdeau
Patrick Archambault
Carl Chartrand‐Lefebvre
Louis Dieumegarde
Reza Forghani
Christian Gagné
Alexandre Hains
David Hornstein
H. Le
Simon Lemieux
Marie‐Hélène Lévesque
Diego R. Martín
Lorne Rosenbloom
An Tang
F. Vecchio
Olivier Potvin
Nathalie Duchesne
Decision scores and ethically mindful algorithms are being established to adjudicate mechanical ventilation in the context of potential reso… (see more)urces shortage due to the current onslaught of COVID-19 cases. There is a need for a reproducible and objective method to provide quantitative information for those scores. Towards this goal, we present a retrospective study testing the ability of a deep learning algorithm at extracting features from chest x-rays (CXR) to track and predict radiological evolution. We trained a repurposed deep learning algorithm on the CheXnet open dataset (224,316 chest X-ray images of 65,240 unique patients) to extract features that mapped to radiological labels. We collected CXRs of COVID-19-positive patients from two open-source datasets (last accessed on April 9, 2020)(Italian Society for Medical and Interventional Radiology and MILA). Data collected form 60 pairs of sequential CXRs from 40 COVID patients (mean age ± standard deviation: 56 ± 13 years; 23 men, 10 women, seven not reported) and were categorized in three categories: “Worse”, “Stable”, or “Improved” on the basis of radiological evolution ascertained from images and reports. Receiver operating characteristic analyses, Mann-Whitney tests were performed. On patients from the CheXnet dataset, the area under ROC curves ranged from 0.71 to 0.93 for seven imaging features and one diagnosis. Deep learning features between “Worse” and “Improved” outcome categories were significantly different for three radiological signs and one diagnostic (“Consolidation”, “Lung Lesion”, “Pleural effusion” and “Pneumonia”; all P 0.05). Features from the first CXR of each pair could correctly predict the outcome category between “Worse” and “Improved” cases with 82.7% accuracy. CXR deep learning features show promise for classifying the disease trajectory. Once validated in studies incorporating clinical data and with larger sample sizes, this information may be considered to inform triage decisions.
2022-04-03
Scientific Reports (published)
doi.org
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation
Kushal Arora
Layla El Asri
Hareesh Bahuleyan
Jackie CK Cheung
Current language generation models suffer from issues such as repetition, incoherence, and hallucinations. An often-repeated hypothesis for … (see more)this brittleness of generation models is that it is caused by the training and the generation procedure mismatch, also referred to as exposure bias. In this paper, we verify this hypothesis by analyzing exposure bias from an imitation learning perspective. We show that exposure bias leads to an accumulation of errors during generation, analyze why perplexity fails to capture this accumulation of errors, and empirically show that this accumulation results in poor generation quality.
2022-04-02
ArXiv (preprint)
doi.org
arxiv.org