Riashat Islam

Reinforcement Learning for Sequence Design Leveraging Protein Language Models

Jithendaraa Subramanian

Shiva Kanth Sujit

Niloy Irtisam

Umong Sain

Derek Nowrouzezahrai

Samira Ebrahimi Kahou

Riashat Islam

2024-07-03

ArXiv (preprint)

doi.org

arxiv.org

Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Generation.

James Vuckovic

Eric Laufer

Cheng-Hao Liu

Michael M. Bronstein

2024-01-01

Neural Information Processing Systems (published)

dblp.uni-trier.de

Offline Policy Optimization in RL with Variance Regularizaton

Riashat Islam

Samarth Sinha

Homanga Bharadhwaj

Samin Yeasar Arnob

Zhuoran Yang

Animesh Garg

Zhaoran Wang

Lihong Li

Doina Precup

2022-12-29

ArXiv (preprint)

doi.org

arxiv.org

Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning

Hongyu Zang

Xin Li

Romain Laroche

Yoshua Bengio

Remi Tachet des Combes

2022-11-01

ArXiv (preprint)

doi.org

arxiv.org

Discrete Compositional Representations as an Abstraction for Goal Conditioned Reinforcement Learning

Hongyu Zang

Xin Li

Romain Laroche

Yoshua Bengio

Remi Tachet des Combes

Goal-conditioned reinforcement learning (RL) is a promising direction for training agents that are capable of solving multiple tasks and rea… (see more)ch a diverse set of objectives. How to \textit{specify} and \textit{ground} these goals in such a way that we can both reliably reach goals during training as well as generalize to new goals during evaluation remains an open area of research. Defining goals in the space of noisy, high-dimensional sensory inputs is one possibility, yet this poses a challenge for training goal-conditioned agents, or even for generalization to novel goals. We propose to address this by learning compositional representations of goals and processing the resulting representation via a discretization bottleneck, for coarser specification of goals, through an approach we call DGRL. We show that discretizing outputs from goal encoders through a bottleneck can work well in goal-conditioned RL setups, by experimentally evaluating this method on tasks ranging from maze environments to complex robotic navigation and manipulation tasks. Additionally, we show a theoretical result which bounds the expected return for goals not observed during training, while still allowing for specifying goals with expressive combinatorial structure.

openreview.net

Importance of Empirical Sample Complexity Analysis for Offline Reinforcement Learning

Samin Yeasar Arnob

Riashat Islam

Doina Precup

2021-12-31

ArXiv (preprint)

arxiv.org

Randomized Least Squares Policy Optimization

Haque Ishfaq

Zhuoran Yang

Andrei-Stefan Lupu

Viet Bang Nguyen

Lewis Liu

Riashat Islam

Zhaoran Wang

Doina Precup

Policy Optimization (PO) methods with function approximation are one of the most popular classes of Reinforcement Learning (RL) algorithms. … (see more)However, designing provably efﬁcient policy optimization algorithms remains a challenge. Recent work in this area has focused on incorporating upper conﬁdence bound (UCB)-style bonuses to drive exploration in policy optimization. In this paper, we present Randomized Least Squares Policy Optimization (RLSPO) which is inspired by Thompson Sampling. We prove that, in an episodic linear kernel MDP setting, RLSPO achieves (cid:101) O ( d 3 / 2 H 3 / 2 √ T ) worst-case (frequentist) regret, where H is the number of episodes, T is the total number of steps and d is the feature dimension. Finally, we evaluate RLSPO empirically and show that it is competitive with existing provably efﬁcient PO algorithms.

InfoBot: Structured Exploration in ReinforcementLearning Using Information Bottleneck

D. Strouse

Matthew Botvinick

Sergey Levine

InfoBot: Transfer and Exploration via the Information Bottleneck

DJ Strouse

Matthew Botvinick

Sergey Levine

A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postula… (see more)te that in the absence of useful reward signals, an effective exploration strategy should seek out {\it decision states}. These states lie at critical junctions in the state space from where the agent can transition to new, potentially unexplored regions. We propose to learn about decision states from prior experience. By training a goal-conditioned policy with an information bottleneck, we can identify decision states by examining where the model actually leverages the goal state. We find that this simple mechanism effectively identifies decision states, even in partially observed settings. In effect, the model learns the sensory cues that correlate with potential subgoals. In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision states and through new regions of the state space.

2019-01-01

ICLR.cc/2019/Conference (poster)

openreview.net

RE-EVALUATE: Reproducibility in Evaluating Reinforcement Learning Algorithms

Reinforcement learning (RL) has recently achieved tremendous success in solving complex tasks. Careful considerations are made towards repro… (see more)ducible research in machine learning. Reproducibility in RL often becomes more difficult, due to the lack of standard evaluation method and detailed methodology for algorithms and comparisons with existing work. In this work, we highlight key differences in evaluation in RL compared to supervised learning, and discuss specific issues that are often non-intuitive for newcomers. We study the importance of reproducibility in evaluation in RL, and propose an evaluation pipeline that can be decoupled from the algorithm code. We hope such an evaluation pipeline can be standardized, as a step towards robust and reproducible research in RL.

2018-06-27

ICML.cc/2018/RML (poster)

openreview.net