Portrait of Artur Kuramshin

Artur Kuramshin

Master's Research - Université de Montréal
Supervisor
Glen Berseth
Research Topics
Reinforcement Learning
Representation Learning
Robotics
Website

Publications

RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies
Pranav Atreya
Karl Pertsch
Tony Lee
Moo Jin Kim
Arhan Jain
Artur Kuramshin
Cyrus Neary
Edward S. Hu
Kanav Arora
Kirsty Ellis
Luca Macesanu
Matthew Leonard
Meedeum Cho
Özgür Aslan
Shivin Dass
Tony Wang
Xingfang Yuan
Abhishek Gupta
Dinesh Jayaraman
Glen Berseth … (see 6 more)
Kostas Daniilidis
Roberto Martín-Martín
Youngwoon Lee
Percy Liang
Chelsea Finn
Sergey Levine
Comprehensive, unbiased, and comparable evaluation of modern generalist policies is uniquely challenging: existing approaches for robot benc… (see more)hmarking typically rely on heavy standardization, either by specifying fixed evaluation tasks and environments, or by hosting centralized "robot challenges", and do not readily scale to evaluating generalist policies across a broad range of tasks and environments. In this work, we propose RoboArena, a new approach for scalable evaluation of generalist robot policies in the real world. Instead of standardizing evaluations around fixed tasks, environments, or locations, we propose to crowd-source evaluations across a distributed network of evaluators. Importantly, evaluators can freely choose the tasks and environments they evaluate on, enabling easy scaling of diversity, but they are required to perform double-blind evaluations over pairs of policies. Then, by aggregating preference feedback from pairwise comparisons across diverse tasks and environments, we can derive a ranking of policies. We instantiate our approach across a network of evaluators at seven academic institutions using the DROID robot platform. Through more than 600 pairwise real-robot evaluation episodes across seven generalist policies, we demonstrate that our crowd-sourced approach can more accurately rank the performance of existing generalist policies than conventional, centralized evaluation approaches, while being more scalable, resilient, and trustworthy. We open our evaluation network to the community and hope that it can enable more accessible comparisons of generalist robot policies.
2025-10-07
Proceedings of The 8th Conference on Robot Learning (published)
proceedings.mlr.press
RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies
Pranav Atreya
Karl Pertsch
Tony Lee
Moo Jin Kim
Arhan Jain
Artur Kuramshin
Cyrus Neary
Edward S. Hu
Kanav Arora
Kirsty Ellis
Luca Macesanu
Matthew Leonard
Meedeum Cho
Özgür Aslan
Shivin Dass
Tony Wang
Xingfang Yuan
Abhishek Gupta
Dinesh Jayaraman
Glen Berseth … (see 6 more)
Kostas Daniilidis
Roberto Martín-Martín
Youngwoon Lee
Percy Liang
Chelsea Finn
Sergey Levine
2025-10-07
Proceedings of The 8th Conference on Robot Learning (published)
proceedings.mlr.press
openreview.net
Task Robustness via Re-Labelling Vision-Action Robot Data
Artur Kuramshin
Özgür Aslan
Cyrus Neary
Glen Berseth
2025-09-06
robot-learning.org/CoRL/2025/Workshop/Robot_Data (published)
openreview.net
openreview.net
Task Robustness via Re-Labelling Vision-Action Robot Data
Artur Kuramshin
Özgür Aslan
Cyrus Neary
Glen Berseth
The recent trend in scaling models for robot learning has resulted in impressive policies that can perform various manipulation tasks and ge… (see more)neralize to novel scenarios. However, these policies continue to struggle with following instructions, likely due to the limited linguistic and action sequence diversity in existing robotics datasets. This paper introduces
2025-09-06
robot-learning.org/CoRL/2025/Workshop/Robot_Data (published)
openreview.net
openreview.net