Publications

Robust and Versatile Bipedal Jumping Control through Reinforcement Learning

Zhongyu Li

Xue Bin Peng

Pieter Abbeel

Sergey Levine

Glen Berseth

Koushil Sreenath

2023-07-10

Robotics: Science and Systems XIX (published)

doi.org

arxiv.org

Overcoming the Technical Challenges of Coordinating Distributed Load Resources at Scale

Johanna Mathieu

Ian Hiskens

Ioannis Marios Granitsas

Oluwagbemileke Oyefeso

Gregory Ledva

Sebastian Nugroho

Salman Nazir

Scott Hinson

Suzanne Russo

Steve Mock

Rachel Jenkins

Jill Harlow

Grant Fisher

Drew Geller

Duncan Callaway

Phillippe Phanivong

Antoine Lesage-Landry

2023-07-08

(published)

doi.org

Capacity Planning in Stable Matching: An Application to School Choice

Federico Bobbio

Margarida Carvalho

Andrea Lodi

Ignacio Rios

Alfredo Torrico

Centralized mechanisms are becoming the standard approach to solve several assignment problems. Examples include the allocation of students … (see more)to schools (school choice), high-school graduates to colleges, residents to hospitals and refugees to cities. In most of these markets, a desirable property of the assignment is stability, which guarantees that no pair of agents has incentive to circumvent the matching. Using school choice as our matching market application, we introduce the problem of jointly allocating a school capacity expansion and finding the best stable matching for the students in the expanded market. We analyze theoretically the problem, focusing on the trade-off behind the multiplicity of student-optimal assignments, and the problem complexity. Since the theoretical intractability of the problem precludes the adaptation of classical approaches to solve it efficiently, we generalize existent mathematical programming formulations of stability constraints to our setting. These generalizations result in integer quadratically-constrained programs, which are computationally hard to solve. In addition, we propose a novel mixed-integer linear programming formulation that is exponentially-large on the problem size. We show that the stability constraints can be separated in linear time, leading to an effective cutting-plane method. We evaluate the performance of our approaches in a detailed computational study, and we find that our cutting-plane method outperforms mixed-integer programming solvers applied to existent formulations extended to our problem setting. We also propose two heuristics that are effective for large instances of the problem. Finally, we use the Chilean school choice system data to demonstrate the impact of capacity planning under stability conditions. Our results show that each additional school seat can benefit multiple students. On the one hand, we can focus on access by prioritizing extra seats that benefit previously unassigned students; on the other hand, we can focus on merit by allocating extra seats that benefit several students via chains of improvement. These insights empower the decision-maker in tuning the matching algorithm to provide a fair application-oriented solution.

2023-07-07

Proceedings of the 24th ACM Conference on Economics and Computation (published)

doi.org

Deep Multirepresentation Learning for Data Clustering.

Mohammadreza Sadeghi

Narges Armanfard

Deep clustering incorporates embedding into clustering in order to find a lower-dimensional space suitable for clustering tasks. Conventiona… (see more)l deep clustering methods aim to obtain a single global embedding subspace (aka latent space) for all the data clusters. In contrast, in this article, we propose a deep multirepresentation learning (DML) framework for data clustering whereby each difficult-to-cluster data group is associated with its own distinct optimized latent space and all the easy-to-cluster data groups are associated with a general common latent space. Autoencoders (AEs) are employed for generating cluster-specific and general latent spaces. To specialize each AE in its associated data cluster(s), we propose a novel and effective loss function which consists of weighted reconstruction and clustering losses of the data points, where higher weights are assigned to the samples more probable to belong to the corresponding cluster(s). Experimental results on benchmark datasets demonstrate that the proposed DML framework and loss function outperform state-of-the-art clustering approaches. In addition, the results show that the DML method significantly outperforms the SOTA on imbalanced datasets as a result of assigning an individual latent space to the difficult clusters.

2023-07-07

IEEE Transactions on Neural Networks and Learning Systems (published)

doi.org

Scaling Laws Do Not Scale

Fernando Diaz

Michael Madaio

Recent work has proposed a power law relationship, referred to as ``scaling laws,'' between the performance of artificial intelligence (AI) … (see more)models and aspects of those models' design (e.g., dataset size). In other words, as the size of a dataset (or model parameters, etc) increases, the performance of a given model trained on that dataset will correspondingly increase. However, while compelling in the aggregate, this scaling law relationship overlooks the ways that metrics used to measure performance may be precarious and contested, or may not correspond with how different groups of people may perceive the quality of models' output. In this paper, we argue that as the size of datasets used to train large AI models grows, the number of distinct communities (including demographic groups) whose data is included in a given dataset is likely to grow, each of whom may have different values. As a result, there is an increased risk that communities represented in a dataset may have values or preferences not captured by (or in the worst case, at odds with) the metrics used to evaluate model performance for scaling laws. We end the paper with implications for AI scaling laws -- that models may not, in fact, continue to improve as the datasets get larger -- at least not for all people or communities impacted by those models.

2023-07-05

ArXiv (preprint)

doi.org

arxiv.org

Generative Flow Networks: a Markov Chain Perspective

Tristan Deleu

Yoshua Bengio

2023-07-04

ArXiv (preprint)

doi.org

arxiv.org

Better Training of GFlowNets with Local Credit and Incomplete Trajectories

Ling Pan

Nikolay Malkin

Dinghuai Zhang

Yoshua Bengio

Generative Flow Networks or GFlowNets are related to Monte-Carlo Markov chain methods (as they sample from a distribution specified by an en… (see more)ergy function), reinforcement learning (as they learn a policy to sample composed objects through a sequence of steps), generative models (as they learn to represent and sample from a distribution) and amortized variational methods (as they can be used to learn to approximate and sample from an otherwise intractable posterior, given a prior and a likelihood). They are trained to generate an object

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (published)

doi.org

openreview.net

Bidirectional Learning for Offline Model-based Biological Sequence Design

Can Chen

Yingxue Zhang

Xue (Steve) Liu

Mark Coates

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (published)

doi.org

openreview.net

Bigger, Better, Faster: Human-level Atari with human-level efficiency

Max Schwarzer

Johan Samir Obando Ceron

Aaron Courville

Marc Gendron-Bellemare

Rishabh Agarwal

Pablo Samuel Castro

We introduce a value-based RL agent, which we call BBF, that achieves super-human performance in the Atari 100K benchmark. BBF relies on sca… (see more)ling the neural networks used for value estimation, as well as a number of other design choices that enable this scaling in a sample-efficient manner. We conduct extensive analyses of these design choices and provide insights for future work. We end with a discussion about updating the goalposts for sample-efficient RL research on the ALE. We make our code and data publicly available at https://github.com/google-research/google-research/tree/master/bigger_better_faster.

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (published)

doi.org

openreview.net

Bootstrapped Representations in Reinforcement Learning

Charline Le Lan

Stephen Tu

Mark Rowland

Anna Harutyunyan

Rishabh Agarwal

Marc Gendron-Bellemare

Will Dabney

In reinforcement learning (RL), state representations are key to dealing with large or continuous state spaces. While one of the promises of… (see more) deep learning algorithms is to automatically construct features well-tuned for the task they try to solve, such a representation might not emerge from end-to-end training of deep RL agents. To mitigate this issue, auxiliary objectives are often incorporated into the learning process and help shape the learnt state representation. Bootstrapping methods are today's method of choice to make these additional predictions. Yet, it is unclear which features these algorithms capture and how they relate to those from other auxiliary-task-based approaches. In this paper, we address this gap and provide a theoretical characterization of the state representation learnt by temporal difference learning (Sutton, 1988). Surprisingly, we find that this representation differs from the features learned by Monte Carlo and residual gradient algorithms for most transition structures of the environment in the policy evaluation setting. We describe the efficacy of these representations for policy evaluation, and use our theoretical analysis to design new auxiliary learning rules. We complement our theoretical results with an empirical comparison of these learning rules for different cumulant functions on classic domains such as the four-room domain (Sutton et al, 1999) and Mountain Car (Moore, 1990).

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (published)

doi.org

openreview.net

Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?

Boris Knyazev

DOHA HWANG

Simon Lacoste-Julien

2023-07-03

Proceedings of the 40th International Conference on Machine Learning (published)

doi.org

openreview.net