Publications

Static graph approximations of dynamic contact networks for epidemic forecasting

Razieh Shirzadkhani

Shenyang Huang

Abby Leung

2024-05-22

Scientific Reports (published)

Investigation of dosimetric characteristics of radiochromic film in response to alpha particles emitted from Americium-241.

Victor D. Diaz‐Martinez

Mélodie Cyr

S. Devic

Nada Tomic

David F. Lewis

Shirin A. Enger

BACKGROUND In radiotherapy, it is essential to deliver prescribed doses to tumors while minimizing damage to surrounding healthy tissue. Acc… (see more)urate measurements of absorbed dose are required for this purpose. Gafchromic® external beam therapy (EBT) radiochromic films have been widely used in radiotherapy. While the dosimetric characteristics of the EBT3 model film have been extensively studied for photon and charged particle beams (protons, electrons, and carbon ions), little research has been done on α

2024-05-20

Medical Physics (Lancaster) (published)

doi.org

Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving

Aniket Rajiv Didolkar

Anirudh Goyal

Nan Rosemary Ke

Siyuan Guo

Michal Valko

Timothy P Lillicrap

Danilo Jimenez Rezende

Yoshua Bengio

Michael Curtis Mozer

Sanjeev Arora

Metacognitive knowledge refers to humans' intuitive knowledge of their own thinking and reasoning processes. Today's best LLMs clearly posse… (see more)ss some reasoning processes. The paper gives evidence that they also have metacognitive knowledge, including ability to name skills and procedures to apply given a task. We explore this primarily in context of math reasoning, developing a prompt-guided interaction procedure to get a powerful LLM to assign sensible skill labels to math questions, followed by having it perform semantic clustering to obtain coarser families of skill labels. These coarse skill labels look interpretable to humans. To validate that these skill labels are meaningful and relevant to the LLM's reasoning processes we perform the following experiments. (a) We ask GPT-4 to assign skill labels to training questions in math datasets GSM8K and MATH. (b) When using an LLM to solve the test questions, we present it with the full list of skill labels and ask it to identify the skill needed. Then it is presented with randomly selected exemplar solved questions associated with that skill label. This improves accuracy on GSM8k and MATH for several strong LLMs, including code-assisted models. The methodology presented is domain-agnostic, even though this article applies it to math problems.

2024-05-20

ArXiv (preprint)

doi.org

arxiv.org

Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving

Aniket Rajiv Didolkar

Anirudh Goyal

Nan Rosemary Ke

Siyuan Guo

Michal Valko

Timothy P Lillicrap

Danilo Jimenez Rezende

Yoshua Bengio

Michael Curtis Mozer

Sanjeev Arora

Metacognitive knowledge refers to humans' intuitive knowledge of their own thinking and reasoning processes. Today's best LLMs clearly posse… (see more)ss some reasoning processes. The paper gives evidence that they also have metacognitive knowledge, including ability to name skills and procedures to apply given a task. We explore this primarily in context of math reasoning, developing a prompt-guided interaction procedure to get a powerful LLM to assign sensible skill labels to math questions, followed by having it perform semantic clustering to obtain coarser families of skill labels. These coarse skill labels look interpretable to humans. To validate that these skill labels are meaningful and relevant to the LLM's reasoning processes we perform the following experiments. (a) We ask GPT-4 to assign skill labels to training questions in math datasets GSM8K and MATH. (b) When using an LLM to solve the test questions, we present it with the full list of skill labels and ask it to identify the skill needed. Then it is presented with randomly selected exemplar solved questions associated with that skill label. This improves accuracy on GSM8k and MATH for several strong LLMs, including code-assisted models. The methodology presented is domain-agnostic, even though this article applies it to math problems.

2024-05-20

ArXiv (preprint)

doi.org

arxiv.org

BitPruning: Learning Bitlengths for Aggressive and Accurate Quantization

Miloš Nikolić

Ghouthi Boukli Hacene

Ciaran Bannon

Alberto Delmas Lascorz

Matthieu Courbariaux

Omar Mohamed Awad

Yoshua Bengio

Isak Edo Vivancos

Vincent Gripon

Andreas Moshovos

Neural networks have demonstrably achieved state-of-the art accuracy using low-bitlength integer quantization, yielding both execution time … (see more)and energy benefits on existing hardware designs that support short bitlengths. However, the question of finding the minimum bitlength for a desired accuracy remains open. We introduce a training method for minimizing inference bitlength at any granularity while maintaining accuracy. Namely, we propose a regularizer that penalizes large bitlength representations throughout the architecture and show how it can be modified to minimize other quantifiable criteria, such as number of operations or memory footprint. We demonstrate that our method learns thrifty representations while maintaining accuracy. With ImageNet, the method produces an average per layer bitlength of 4.13, 3.76 and 4.36 bits on AlexNet, ResNet18 and MobileNet V2 respectively, remaining within 2.0%, 0.5% and 0.5% of the base TOP-1 accuracy.

2024-05-19

2024 IEEE International Symposium on Circuits and Systems (ISCAS) (published)

doi.org

arxiv.org

Coordination among leaf and fine-root traits along a strong natural soil fertility gradient

Xavier Guilbeault-Mayers

Hans Lambers

Étienne Laliberté

2024-05-18

Plant and Soil (published)

doi.org

Coordination among leaf and fine-root traits along a strong natural soil fertility gradient

Xavier Guilbeault-Mayers

Hans Lambers

Étienne Laliberté

2024-05-18

Plant and Soil (published)

doi.org

Implementing a Hierarchical Deep Learning Approach for Simulating multilevel Auction Data

Marcelin Joanis

Andrea Lodi

Igor Sadoune

2024-05-18

Computational Economics (published)

doi.org

arxiv.org

Towards Modular LLMs by Building and Reusing a Library of LoRAs

Oleksiy Ostapenko

Zhan Su

Edoardo Ponti

Laurent Charlin

Nicolas Le Roux

Matheus Pereira

Lucas Caccia

Alessandro Sordoni

The growing number of parameter-efficient adaptations of a base large language model (LLM) calls for studying whether we can reuse such trai… (see more)ned adapters to improve performance for new tasks. We study how to best build a library of adapters given multi-task data and devise techniques for both zero-shot and supervised task generalization through routing in such library. We benchmark existing approaches to build this library and introduce model-based clustering, MBC, a method that groups tasks based on the similarity of their adapter parameters, indirectly optimizing for transfer across the multi-task dataset. To re-use the library, we present a novel zero-shot routing mechanism, Arrow, which enables dynamic selection of the most relevant adapters for new inputs without the need for retraining. We experiment with several LLMs, such as Phi-2 and Mistral, on a wide array of held-out tasks, verifying that MBC-based adapters and Arrow routing lead to superior generalization to new tasks. We make steps towards creating modular, adaptable LLMs that can match or outperform traditional joint training.

2024-05-18

ArXiv (preprint)

arxiv.org

Automatic segmentation of Organs at Risk in Head and Neck cancer patients from CT and MRI scans

Sébastien Quetin

Andrew Heschl

Mauricio Murillo

Rohit Murali

Shirin A. Enger

Farhad Maleki

Background and purpose: Deep Learning (DL) has been widely explored for Organs at Risk (OARs) segmentation; however, most studies have focus… (see more)ed on a single modality, either CT or MRI, not both simultaneously. This study presents a high-performing DL pipeline for segmentation of 30 OARs from MRI and CT scans of Head and Neck (H&N) cancer patients. Materials and methods: Paired CT and MRI-T1 images from 42 H&N cancer patients alongside annotation for 30 OARs from the H&N OAR CT&MR segmentation challenge dataset were used to develop a segmentation pipeline. After cropping irrelevant regions, rigid followed by non-rigid registration of CT and MRI volumes was performed. Two versions of the CT volume, representing soft tissues and bone anatomy, were stacked with the MRI volume and used as input to an nnU-Net pipeline. Modality Dropout was used during the training to force the model to learn from the different modalities. Segmentation masks were predicted with the trained model for an independent set of 14 new patients. The mean Dice Score (DS) and Hausdorff Distance (HD) were calculated for each OAR across these patients to evaluate the pipeline. Results: This resulted in an overall mean DS and HD of 0.777 +- 0.118 and 3.455 +- 1.679, respectively, establishing the state-of-the-art (SOTA) for this challenge at the time of submission. Conclusion: The proposed pipeline achieved the best DS and HD among all participants of the H&N OAR CT and MR segmentation challenge and sets a new SOTA for automated segmentation of H&N OARs.

2024-05-17

ArXiv (preprint)

doi.org

arxiv.org

GFETM: Genome Foundation-based Embedded Topic Model for scATAC-seq Modeling

Yimin Fan

Adrien Osakwe

Shi Han

Yu Li

Jun Ding

Yue Li

2024-05-17