Publications

Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models
Zhong Zhang
Bang Liu
Junming Shao
2023-01-01
ACL (1) (published)
doi.org
arxiv.org
Formal and Empirical Studies of Counting Behaviour in ReLU RNNs.
Nadine El-Naggar
Andrew Ryzhikov
Laure Daviaud
Pranava Madhyastha
Tillman Weyde
François Coste
Faissal Ouardi
Guillaume Rabusseau
2023-01-01
International Conference on Graphics and Interaction (published)
dblp.uni-trier.de
Formalizing locality for normative synaptic plasticity models
Colin Bredenberg
Ezekiel Williams
Cristina Savin
Blake Richards
Guillaume Lajoie
openreview.net
Formation of Giant Plasma Membrane Vesicles for Biological and Medical Applications: A Review
Yangping Li
Songyang Liu
Wanyu Xu
Kemin Wang
Feng-jiang He
Jianbo Liu
Plasma membrane vesicles (PMVs) are micron-sized biomembrane vesicles that are isolated directly from living cells. They retain the lipid an… (see more)d protein complexity of the plasma membrane of the parent cell...
2023-01-01
Sensors & Diagnostics (published)
doi.org
Formation of Giant Plasma Membrane Vesicles for Biological and Medical Applications: A Review
Yang Li
Songyang Liu
Wanyu Xu
Kemin Wang
Fengjiao He
Jianbo Liu
Plasma membrane vesicles (PMVs) are micron-sized biomembrane vesicles that are isolated directly from living cells. They retain the lipid an… (see more)d protein complexity of the plasma membrane of the parent cell...
2023-01-01
Sensors & Diagnostics (published)
doi.org
From Words to Blocks: Building Objects by Grounding Language Models with Reinforcement Learning
Michael Ahn
Anthony Brohan
Noah Brown
liang Dai
Dan Su
Holy Lovenia Ziwei Bryan Wilie
Tiezheng Yu
Willy Chung
Quyet V. Do
Paul Barde
Tristan Karch
Derek Nowrouzezahrai
C. Bonial
Mitchell Abrams
David R. Traum
Hyung Won
Le Hou
Shayne Longpre
Yi Zoph
William Tay … (see 32 more)
Eric Fedus
Xuezhi Li
Lasse Espeholt
Hubert Soyer
Remi Munos
Karen Si-801
Vlad Mnih
Tom Ward
Yotam Doron
Wenlong Huang
Pieter Abbeel
Deepak Pathak
Julia Kiseleva
Ziming Li
Mohammad Aliannejadi
Shrestha Mohanty
Maartje Ter Hoeve
Mikhail Burtsev
Alexey Skrynnik
Artem Zholus
A. Panov
Kavya Srinet
A. Szlam
Yuxuan Sun
Katja Hofmann
Marc-Alexandre Côté
Ahmed Hamid Awadallah
Linar Abdrazakov
Igor Churin
Putra Manggala
Kata Naszádi
Michiel Van Der Meer
Leveraging pre-trained language models to gen-001 erate action plans for embodied agents is an 002 emerging research direction. However, exe… (see more)-003 cuting instructions in real or simulated envi-004 ronments necessitates verifying the feasibility 005 of actions and their relevance in achieving a 006 goal. We introduce a novel method that in-007 tegrates a language model and reinforcement 008 learning for constructing objects in a Minecraft-009 like environment, based on natural language 010 instructions. Our method generates a set of 011 consistently achievable sub-goals derived from 012 the instructions and subsequently completes the 013 associated sub-tasks using a pre-trained RL pol-014 icy. We employ the IGLU competition, which 015 is based on the Minecraft-like simulator, as our 016 test environment, and compare our approach 017 to the competition’s top-performing solutions. 018 Our approach outperforms existing solutions in 019 terms of both the quality of the language model 020 and the quality of the structures built within the 021 IGLU environment. 022
2023-01-01
(published)
www.semanticscholar.org
Functional architecture of the aging brain
Roni Setton
Laetitia Mwilambwe-Tshilobo
Manesh Girn
Amber W. Lockrow
Giulia Baracchini
Alexander J. Lowe
Benjamin N. Cassidy
Jian Li
Wen-Ming Luh
Danilo Bzdok
Richard M. Leahy
Tian Ge
Daniel S. Margulies
Bratislav Mišić
Boris C Bernhardt
W. Dale Stevens
Felipe De Brigard
Prantik Kundu
Gary R. Turner
R. Nathan Spreng
The intrinsic functional connectome can reveal how a lifetime of learning and lived experience is represented in the functional architecture… (see more) of the aging brain. We investigated whether network dedifferentiation, a hallmark of brain aging, reflects a global shift in network dynamics, or comprises network-specific changes that reflect the changing landscape of aging cognition. We implemented a novel multi-faceted strategy involving multi-echo fMRI acquisition and de-noising, individualized cortical parcellation, and multivariate (gradient and edge-level) functional connectivity methods. Twenty minutes of resting-state fMRI data and cognitive assessments were collected in younger (n=181) and older (n=120) adults. Dimensionality in the BOLD signal was lower for older adults, consistent with global network dedifferentiation. Functional connectivity gradients were largely age-invariant. In contrast, edge-level connectivity showed widespread changes with age, revealing discrete, network-specific dedifferentiation patterns. Visual and somatosensory regions were more integrated within the functional connectome; default and frontoparietal regions showed greater coupling; and the dorsal attention network was less differentiated from transmodal regions. Associations with cognition suggest that the formation and preservation of integrated, large-scale brain networks supports complex cognitive abilities. However, into older adulthood, the connectome is dominated by large-scale network disintegration, global dedifferentiation and network-specific dedifferentiation associated with age-related cognitive change.
2023-01-01
(published)
www.semanticscholar.org
FusionRetro: Molecule Representation Fusion via Reaction Graph for Retrosynthetic Planning
Songtao Liu
Zhengkai Tu
Minkai Xu
Zuobai Zhang
Peilin Zhao
Jian Tang
Rex Ying
Lu Lin
Dinghao Wu
Retrosynthetic planning is a fundamental problem in drug discovery and organic chemistry, which aims to find a complete multi-step syntheti… (see more)c route from a set of starting materials to the target molecule, determining crucial process flow in chemical production. Existing approaches combine single-step retrosynthesis models and search algorithms to find synthetic routes. However, these approaches generally consider the two pieces in a decoupled manner, taking only the product as the input to predict the reactants per planning step and largely ignoring the important context information from other intermediates along the synthetic route. In this work, we perform a series of experiments to identify the limitations of this decoupled view and propose a novel retrosynthesis framework that also exploits context information for retrosynthetic planning. We view synthetic routes as reaction graphs, and propose to incorporate the context by three principled steps: encode molecules into embeddings, aggregate information over routes, and readout to predict reactants. The whole framework can be efficiently optimized in an end-to-end fashion. Comprehensive experiments show that by fusing in context information over routes, our model sig-nificantly improves the performance of retrosyn-thetic planning over baselines that are not context-aware, especially for long synthetic routes.
2023-01-01
(published)
www.semanticscholar.org
FusionRetro: Molecule Representation Fusion via In-Context Learning for Retrosynthetic Planning
Songtao Liu
Zhengkai Tu
Minkai Xu
Zuobai Zhang
Lu Lin
Rex Ying
Zhitao Ying
Jian Tang
Peilin Zhao
Dinghao Wu
2023-01-01
ICML (published)
openreview.net
openreview.net
Gap Minimization for Knowledge Sharing and Transfer
Boyu Wang
Jorge A. Mendez
Changjian Shui
Fan Zhou
Di Wu
Gezheng Xu
Christian Gagné
Eric R. Eaton
Learning from multiple related tasks by knowledge sharing and transfer has become increasingly relevant over the last two decades. In order … (see more)to successfully transfer information from one task to another, it is critical to understand the similarities and differences between the domains. In this paper, we introduce the notion of \emph{performance gap}, an intuitive and novel measure of the distance between learning tasks. Unlike existing measures which are used as tools to bound the difference of expected risks between tasks (e.g.,
arxiv.org
General Purpose AI Systems in the AI Act: Trying to Fit a Square Peg Into a Round Hole
Claire Boine
David Rolnick
2023-01-01
SSRN Electronic Journal (published)
doi.org
Generating QM1B with PySCF$_{\text{IPU}}$
Alexander Mathiasen
Hatem Helal
Kerstin Klaeser
Paul Balanca
Josef Dean
Carlo Luschi
Dominique Beaini
Andrew William Fitzgibbon
Dominic Masters
openreview.net