Portrait of Kaustubh Mani is unavailable

Kaustubh Mani

PhD - Université de Montréal
Supervisor
Liam Paull
Research Topics
Reinforcement Learning
Google Scholar
GitHub

Publications

Safety Representations for Safer Policy Learning
Kaustubh Mani
Vincent Mai
Charlie Gauthier
Annie S Chen
Samer B. Nashed
Liam Paull
Reinforcement learning algorithms typically necessitate extensive exploration of the state space to find optimal policies. However, in safet… (see more)y-critical applications, the risks associated with such exploration can lead to catastrophic consequences. Existing safe exploration methods attempt to mitigate this by imposing constraints, which often result in overly conservative behaviours and inefficient learning. Heavy penalties for early constraint violations can trap agents in local optima, deterring exploration of risky yet high-reward regions of the state space. To address this, we introduce a method that explicitly learns state-conditioned safety representations. By augmenting the state features with these safety representations, our approach naturally encourages safer exploration without being excessively cautious, resulting in more efficient and safer policy learning in safety-critical scenarios. Empirical evaluations across diverse environments show that our method significantly improves task performance while reducing constraint violations during training, underscoring its effectiveness in balancing exploration with safety.
2025-01-22
ICLR.cc/2025/Conference (poster)
doi.org
openreview.net
Safety Representations for Safer Policy Learning
Kaustubh Mani
Vincent Mai
Charlie Gauthier
Annie S Chen
Samer B. Nashed
Liam Paull
Reinforcement learning algorithms typically necessitate extensive exploration of the state space to find optimal policies. However, in safet… (see more)y-critical applications, the risks associated with such exploration can lead to catastrophic consequences. Existing safe exploration methods attempt to mitigate this by imposing constraints, which often result in overly conservative behaviours and inefficient learning. Heavy penalties for early constraint violations can trap agents in local optima, deterring exploration of risky yet high-reward regions of the state space. To address this, we introduce a method that explicitly learns state-conditioned safety representations. By augmenting the state features with these safety representations, our approach naturally encourages safer exploration without being excessively cautious, resulting in more efficient and safer policy learning in safety-critical scenarios. Empirical evaluations across diverse environments show that our method significantly improves task performance while reducing constraint violations during training, underscoring its effectiveness in balancing exploration with safety.
2025-01-22
ICLR.cc/2025/Conference (poster)
openreview.net
openreview.net