Portrait of Shivakanth Sujit is unavailable

Shivakanth Sujit

Alumni
Website
Google Scholar

Publications

Prioritizing Samples in Reinforcement Learning with Reducible Loss
Shivakanth Sujit
Somjit Nath
Pedro H.M. Braga
Samira Ebrahimi Kahou
Most reinforcement learning algorithms take advantage of an experience replay buffer to repeatedly train on samples the agent has observed i… (see more)n the past. Not all samples carry the same amount of significance and simply assigning equal importance to each of the samples is a naïve strategy. In this paper, we propose a method to prioritize samples based on how much we can learn from a sample. We define the learn-ability of a sample as the steady decrease of the training loss associated with this sample over time. We develop an algorithm to prioritize samples with high learn-ability, while assigning lower priority to those that are hard-to-learn, typically caused by noise or stochasticity. We empirically show that our method is more robust than random sampling and also better than just prioritizing with respect to the training loss, i.e. the temporal difference loss, which is used in prioritized experience replay.
2023-12-11
Neural Information Processing Systems (Accept (poster))
doi.org
openreview.net
Learning Robust Dynamics through Variational Sparse Gating
Arnav Kumar Jain
Shivakanth Sujit
Shruti Joshi
Vincent Michalski
Danijar Hafner
Samira Ebrahimi-Kahou
Learning world models from their sensory inputs enables agents to plan for actions by imagining their future outcomes. World models have pre… (see more)viously been shown to improve sample-efficiency in simulated environments with few objects, but have not yet been applied successfully to environments with many objects. In environments with many objects, often only a small number of them are moving or interacting at the same time. In this paper, we investigate integrating this inductive bias of sparse interactions into the latent dynamics of world models trained from pixels. First, we introduce Variational Sparse Gating (VSG), a latent dynamics model that updates its feature dimensions sparsely through stochastic binary gates. Moreover, we propose a simplified architecture Simple Variational Sparse Gating (SVSG) that removes the deterministic pathway of previous models, resulting in a fully stochastic transition function that leverages the VSG mechanism. We evaluate the two model architectures in the BringBackShapes (BBS) environment that features a large number of moving objects and partial observability, demonstrating clear improvements over prior models.
2022-10-30
Neural Information Processing Systems (accepted)
doi.org
openreview.net