Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models
Amal Rannen-Triki
Jorg Bornschein
Marcus Hutter
Andr'as Gyorgy
Alexandre Galashov
Yee Whye Teh
Michalis K. Titsias
We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation. While it is… (voir plus) generally known that this approach improves the overall predictive performance, especially when considering distributional shift between training and evaluation data, we here emphasize the perspective that online adaptation turns parameters into temporally changing states and provides a form of context-length extension with memory in weights, more in line with the concept of memory in neuroscience. We pay particular attention to the speed of adaptation (in terms of sample efficiency),sensitivity to the overall distributional drift, and the computational overhead for performing gradient computations and parameter updates. Our empirical study provides insights on when online adaptation is particularly interesting. We highlight that with online adaptation the conceptual distinction between in-context learning and fine tuning blurs: both are methods to condition the model on previously observed tokens.
Socially Assistive Robots for patients with Alzheimer's Disease: A scoping review.
Vania Karami
Mark J. Yaffe
Genevieve Gore
Socially Assistive Robots for patients with Alzheimer's Disease: A scoping review.
Vania Karami
Mark J. Yaffe
Genevieve Gore
Socially Assistive Robots for patients with Alzheimer's Disease: A scoping review.
Vania Karami
Mark J. Yaffe
Genevieve Gore
Socially Assistive Robots for patients with Alzheimer's Disease: A scoping review.
Vania Karami
Mark J. Yaffe
Genevieve Gore
Sources of richness and ineffability for phenomenally conscious states
Xu Ji
Eric Elmoznino
George Deane
Axel Constant
Jonathan Simon
Substitution of dietary monounsaturated fatty acids from olive oil for saturated fatty acids from lard increases low-density lipoprotein apolipoprotein B-100 fractional catabolic rate in subjects with dyslipidemia associated with insulin resistance: a randomized controlled trial
Louis-Charles Desjardins
Francis Brière
André J Tremblay
Maryka Rancourt-Bouchard
Jean-Philippe Drouin-Chartier
Valéry Lemelin
Amélie Charest
Ernst J Schaefer
Benoit Lamarche
Patrick Couture
Substitution of dietary monounsaturated fatty acids from olive oil for saturated fatty acids from lard increases LDL apolipoprotein B-100 fractional catabolic rate in subjects with dyslipidemia associated with insulin resistance: a randomized controlled trial.
Louis-Charles Desjardins
Francis Brière
André J Tremblay
Maryka Rancourt-Bouchard
Jean-Philippe Drouin-Chartier
Valéry Lemelin
Amélie Charest
Ernst J Schaefer
Benoit Lamarche
Patrick Couture
The effects of nature-based vs. indoor settings on the adaptability, performance and affect of calisthenics exercisers. A registered report.
Henrique Brito
Henrique Lopes
Daniel Carrilho
Adriano Carvalho
Duarte Araújo
The effects of nature-based vs. indoor settings on the adaptability, performance and affect of calisthenics exercisers. A registered report.
Henrique Brito
Henrique Lopes
Daniel Carrilho
Adriano Carvalho
Duarte Araújo
The « jingle-jangle fallacy » of empathy: Delineating affective, cognitive and motor components of empathy from behavioral synchrony using a virtual agent
Julia Ayache
Alexander Sumich
D. Kuss
Darren Rhodes
Nadja Heym
Towards a connection between the capacitated vehicle routing problem and the constrained centroid-based clustering
Abdelhakim Abdellaoui
Issmail ElHallaoui
Efficiently solving a vehicle routing problem (VRP) in a practical runtime is a critical challenge for delivery management companies. This p… (voir plus)aper explores both a theoretical and experimental connection between the Capacitated Vehicle Routing Problem (CVRP) and the Constrained Centroid-Based Clustering (CCBC). Reducing a CVRP to a CCBC is a synonym for a transition from an exponential to a polynomial complexity using commonly known algorithms for clustering, i.e K-means. At the beginning, we conduct an exploratory analysis to highlight the existence of such a relationship between the two problems through illustrative small-size examples and simultaneously deduce some mathematically-related formulations and properties. On a second level, the paper proposes a CCBC based approach endowed with some enhancements. The proposed framework consists of three stages. At the first step, a constrained centroid-based clustering algorithm generates feasible clusters of customers. This methodology incorporates three enhancement tools to achieve near-optimal clusters, namely: a multi-start procedure for initial centroids, a customer assignment metric, and a self-adjustment mechanism for choosing the number of clusters. At the second step, a traveling salesman problem (T SP) solver is used to optimize the order of customers within each cluster. Finally, we introduce a process relying on routes cutting and relinking procedure, which calls upon solving a linear and integer programming model to further improve the obtained routes. This step is inspired by the ruin&recreate algorithm. This approach is an extension of the classical cluster-first, route-second method and provides near-optimal solutions on well-known benchmark instances in terms of solution quality and computational runtime, offering a milestone in solving VRP.