Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control
Zhongyu Li
Xue Bin Peng
Pieter Abbeel
Sergey Levine
Koushil Sreenath
This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal rob… (voir plus)ots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot’s I/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.
The Effects of a Digital Game Simulator versus a Traditional Intervention on Paramedics’ Neonatal Resuscitation Performance
Georg M. Schmölzer
Unsupervised Discovery of Steerable Factors When Graph Deep Generative Models Are Entangled
Shengchao Liu
Chengpeng Wang
Jiarui Lu
Weili Nie
Hanchen Wang
Zhuoxinran Li
Bolei Zhou
Asymmetric stimulus representations bias visual perceptual learning
Pooya Laamerad
Asmara Awada
Christopher C. Pack
The primate visual cortex contains various regions that exhibit specialization for different stimulus properties, such as motion, shape, and… (voir plus) color. Within each region there is often further specialization, such that particular stimulus features, such as horizontal and vertical orientations, are overrepresented. These asymmetries are associated with well-known perceptual biases, but little is known about how they influence visual learning. Most theories would predict that learning is optimal, in the sense that it is unaffected by these asymmetries. But other approaches to learning would result in specific patterns of perceptual biases. To distinguish between these possibilities, we trained human observers to discriminate between expanding and contracting motion patterns, which have a highly asymmetrical representation in visual cortex. Observers exhibited biased percepts of these stimuli, and these biases were affected by training in ways that were often suboptimal. We simulated different neural network models and found that a learning rule that involved only adjustments to decision criteria, rather than connection weights, could account for our data. These results suggest that cortical asymmetries influence visual perception and that human observers often rely on suboptimal strategies for learning.
A comparison of RL-based and PID controllers for 6-DOF swimming robots: hybrid underwater object tracking
Faraz Lotfi
Khalil Virji
Nicholas Dudek
Protocol for fever control using external cooling in mechanically ventilated patients with septic shock: SEPSISCOOL II randomised controlled trial
Armelle Guénégou-Arnoux
Juliette Murris
Stéphane Bechet
Camille Jung
Johann Auchabie
Julien Dupeyrat
Nadia Anguel
Pierre Asfar
Julio Badie
Dorothée Carpentier
Benjamin Chousterman
Jeremy Bourenne
Agathe Delbove
Jérôme Devaquet
Nicolas Deye
Anne-Florence Dureau
Jean-Baptiste Lascarrou
Stephane Legriel
Christophe Guitton … (voir 14 de plus)
Caroline Jannière-Nartey
Jean-Pierre Quenot
Jean-Claude Lacherade
Julien Maizel
Armand Mekontso Dessap
Bruno Mourvillier
Philippe Petua
Gaetan Plantefeve
Jean-Christophe Richard
Alexandre Robert
Clément Saccheri
Ly Van Phach Vong
Sandrine Katsahian
Frédérique Schortgen
Trait‐matching models predict pairwise interactions across regions, not food web properties
Dominique Caron
Ulrich Brose
Miguel Lurgi
F. Guillaume Blanchet
Dominique Gravel
Trait‐matching models predict pairwise interactions across regions, not food web properties
Dominique Caron
Ulrich Brose
Miguel Lurgi
F. Guillaume Blanchet
Dominique Gravel
Food webs are essential for understanding how ecosystems function, but empirical data on the interactions that make up these ecological netw… (voir plus)orks are lacking for most taxa in most ecosystems. Trait‐based models can fill these data gaps, but their ability to do so has not been widely tested. We test how well these models can extrapolate to new ecological communities both in terms of pairwise predator–prey interactions and higher level food web attributes (i.e. species position, food web‐level properties).Canada, Europe, Tanzania.Current.Terrestrial vertebrates.We train trait‐based models of pairwise trophic interactions on four independent vertebrate food webs (Canadian tundra, Serengeti, alpine south‐eastern Pyrenees and Europe) and evaluate how well these models predict pairwise interactions and network properties of each food web.We find that, overall, trait‐based models predict most interactions and their absence correctly. Performance was best for training and testing on the same food web (AUC > 0.90) and declined with environmental and phylogenetic distances with the strongest loss of performance for the tundra‐Serengeti ecosystems (AUC > 0.75). Network metrics were less well‐predicted than single interactions by our models with predicted food webs being more connected, less modular, and with higher mean trophic levels than observed.Theory predicts that the variability observed in food webs can be explained by differences in trait distributions and trait‐matching relationships. Our finding that trait‐based models can predict many trophic interactions, even in contrasting environments, adds to the growing body of evidence that there are general constraints on interactions and that trait‐based methods can serve as a useful first approximation of food webs in unknown areas. However, food webs are more than the sum of their parts, and predicting network attributes will likely require models that simultaneously predict individual interactions and community constraints.
Efficient Data-Driven MPC for Demand Response of Commercial Buildings
Marie-Christine Par'e
Vasken Dermardiros
Model predictive control (MPC) has been shown to significantly improve the energy efficiency of buildings while maintaining thermal comfort.… (voir plus) Data-driven approaches based on neural networks have been proposed to facilitate system modelling. However, such approaches are generally nonconvex and result in computationally intractable optimization problems. In this work, we design a readily implementable energy management method for small commercial buildings. We then leverage our approach to formulate a real-time demand bidding strategy. We propose a data-driven and mixed-integer convex MPC which is solved via derivative-free optimization given a limited computational time of 5 minutes to respect operational constraints. We consider rooftop unit heating, ventilation, and air conditioning systems with discrete controls to accurately model the operation of most commercial buildings. Our approach uses an input convex recurrent neural network to model the thermal dynamics. We apply our approach in several demand response (DR) settings, including a demand bidding, a time-of-use, and a critical peak rebate program. Controller performance is evaluated on a state-of-the-art building simulation. The proposed approach improves thermal comfort while reducing energy consumption and cost through DR participation, when compared to other data-driven approaches or a set-point controller.
Graphylo: A deep learning approach for predicting regulatory DNA and RNA sites from whole-genome multiple alignments
Dongjoon Lim
Changhyun Baek
Black-Box Access is Insufficient for Rigorous AI Audits
Stephen Casper
Carson Ezell
Charlotte Siegmann
Noam Kolt
Taylor Lynn Curtis
Benjamin Bucknall
Andreas Haupt
Kevin Wei
Jérémy Scheurer
Marius Hobbhahn
Lee Sharkey
Satyapriya Krishna
Marvin Von Hagen
Silas Alberti
Alan Chan
Qinyi Sun
Michael Gerovitch
David Bau
Max Tegmark
Dylan Hadfield-Menell
External audits of AI systems are increasingly recognized as a key mechanism for AI governance. The effectiveness of an audit, however, depe… (voir plus)nds on the degree of access granted to auditors. Recent audits of state-of-the-art AI systems have primarily relied on black-box access, in which auditors can only query the system and observe its outputs. However, white-box access to the system’s inner workings (e.g., weights, activations, gradients) allows an auditor to perform stronger attacks, more thoroughly interpret models, and conduct fine-tuning. Meanwhile, outside-the-box access to training and deployment information (e.g., methodology, code, documentation, data, deployment details, findings from internal evaluations) allows auditors to scrutinize the development process and design more targeted evaluations. In this paper, we examine the limitations of black-box audits and the advantages of white- and outside-the-box audits. We also discuss technical, physical, and legal safeguards for performing these audits with minimal security risks. Given that different forms of access can lead to very different levels of evaluation, we conclude that (1) transparency regarding the access and methods used by auditors is necessary to properly interpret audit results, and (2) white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.
Black-Box Access is Insufficient for Rigorous AI Audits
Stephen Casper
Carson Ezell
Charlotte Siegmann
Noam Kolt
Taylor Lynn Curtis
Benjamin Bucknall
Andreas Haupt
Kevin Wei
Jérémy Scheurer
Marius Hobbhahn
Lee Sharkey
Satyapriya Krishna
Marvin Von Hagen
Silas Alberti
Alan Chan
Qinyi Sun
Michael Gerovitch
David Bau
Max Tegmark
Dylan Hadfield-Menell
External audits of AI systems are increasingly recognized as a key mechanism for AI governance. The effectiveness of an audit, however, depe… (voir plus)nds on the degree of access granted to auditors. Recent audits of state-of-the-art AI systems have primarily relied on black-box access, in which auditors can only query the system and observe its outputs. However, white-box access to the system’s inner workings (e.g., weights, activations, gradients) allows an auditor to perform stronger attacks, more thoroughly interpret models, and conduct fine-tuning. Meanwhile, outside-the-box access to training and deployment information (e.g., methodology, code, documentation, data, deployment details, findings from internal evaluations) allows auditors to scrutinize the development process and design more targeted evaluations. In this paper, we examine the limitations of black-box audits and the advantages of white- and outside-the-box audits. We also discuss technical, physical, and legal safeguards for performing these audits with minimal security risks. Given that different forms of access can lead to very different levels of evaluation, we conclude that (1) transparency regarding the access and methods used by auditors is necessary to properly interpret audit results, and (2) white- and outside-the-box access allow for substantially more scrutiny than black-box access alone.