24.02.14 KB. Ok to add accepted conference paper to spiral© 2014 IEEE.Learning policies that generalize across multiple tasks is an important and challenging research topic in reinforcement learning and robotics. Training individual policies for every single potential task is often impractical, especially for continuous task variations, requiring more principled approaches to share and transfer knowledge among similar tasks. We present a novel approach for learning a nonlinear feedback policy that generalizes across multiple tasks. The key idea is to define a parametrized policy as a function of both the state and the task, which allows learning a single policy that generalizes across multiple known and unknown tasks. Applications of our novel approach to reinforcement and imitation learning in realrobot experiments are shown

Kober J, Bagnell JA, Peters J. Reinforcement learning in robotics: A survey. The International Journal Of Robotics Research. 2013;32(11):1238-1274.Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model-based and model-free as well as between value-function-based and policy-search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research

In robotics, the ultimate goal of reinforcement learning is to endow robots with the ability to learn, improve, adapt and reproduce tasks with dynamically changing constraints based on exploration and autonomous learning. We give a summary of the state-of-the-art of reinforcement learning in the context of robotics, in terms of both algorithms and policy representations. Numerous challenges faced by the policy representation in robotics are identified. Three recent examples for the application of reinforcement learning to real-world robots are described: a pancake flipping task, a bipedal walking energy minimization task and an archery-based aiming task. In all examples, a state-of-the-art expectation-maximization-based reinforcement learning is used, and different policy representations are proposed and evaluated for each task. The proposed policy representations offer viable solutions to six rarely-addressed challenges in policy representations: correlations, adaptability, multi-resolution, globality, multi-dimensionality and convergence. Both the successes and the practical difficulties encountered in these examples are discussed. Based on insights from these particular cases, conclusions are drawn about the state-of-the-art and the future perspective directions for reinforcement learning in robotics.

Reinforcement learning has become a prominent learning paradigm for enabling autonomous robotic systems to acquire intelligent decision-making capabilities through continuous interaction with dynamic environments. Practical deployment of reinforcement learning in robotics remains constrained by high sample complexity, prolonged training duration, computational overhead, and safety limitations associated with extensive trial-and-error learning. Transfer learning has emerged as an effective solution by enabling the reuse of previously acquired knowledge across related tasks, environments, and robotic platforms, thereby improving learning efficiency and accelerating policy optimization. Recent advances in deep learning, representation learning, policy transfer, and domain adaptation have further strengthened the capability of transfer learning to improve generalization, reduce training cost, and support rapid adaptation in complex robotic sapplications. This chapter presents a comprehensive overview of transfer learning techniques designed to accelerate reinforcement learning in robotics. Fundamental concepts of reinforcement learning and transfer learning are introduced before examining knowledge transfer mechanisms, policy transfer strategies, representation learning methods, hybrid transfer learning frameworks, and deep transfer learning architectures. The discussion further explores simulation-to-real transfer, domain adaptation, meta-learning, continual learning, and emerging intelligent learning frameworks that enhance robotic autonomy across diverse operational environments. Benchmark simulation platforms, evaluation metrics, robotic application domains, current research challenges, and future research directions are also examined to provide a structured understanding of recent developments. The presented discussion highlights the role of transfer learning in improving sample efficiency, accelerating convergence, enhancing policy generalization, and reducing computational complexity. The chapter serves as a valuable reference for researchers, academicians, and practitioners seeking efficient knowledge transfer methodologies for developing scalable, adaptive, and reliable reinforcement learning systems for next-generation intelligent robotics.

Autonomous Vehicles (AVs) are required to operate safely and efficiently in dynamic environments. For this, the AVs equipped with Joint Radar-Communications (JRC) functions can enhance the driving safety by utilizing both radar detection and data communication functions. However, optimizing the performance of the AV system with two different functions under uncertainty and dynamic of surrounding environments is very challenging. In this work, we first propose an intelligent optimization framework based on the Markov Decision Process (MDP) to help the AV make optimal decisions in selecting JRC operation functions under the dynamic and uncertainty of the surrounding environment. We then develop an effective learning algorithm leveraging recent advances of deep reinforcement learning techniques to find the optimal policy for the AV without requiring any prior information about surrounding environment. Furthermore, to make our proposed framework more scalable, we develop a Transfer Learning (TL) mechanism that enables the AV to leverage valuable experiences for accelerating the training process when it moves to a new environment. Extensive simulations show that the proposed transferable deep reinforcement learning framework reduces the obstacle miss detection probability by the AV up to 67% compared to other conventional deep reinforcement learning approaches.

The field of robotics has been rapidly developing in recent years, and the work related to training robotic agents with reinforcement learning has been a major focus of research. This survey reviews the application of reinforcement learning for pick-and-place operations, a task that a logistics robot can be trained to complete without support from a robotics engineer. To introduce this topic, we first review the fundamentals of reinforcement learning and various methods of policy optimization, such as value iteration and policy search. Next, factors which have an impact on the pick-and-place task, such as reward shaping, imitation learning, pose estimation, and simulation environment are examined. Following the review of the fundamentals and key factors for reinforcement learning, we present an extensive review of all methods implemented by researchers in the field to date. The strengths and weaknesses of each method from literature are discussed, and details about the contribution of each manuscript to the field are reviewed. The concluding critical discussion of the available literature, and the summary of open problems indicates that experiment validation, model generalization, and grasp pose selection are topics that require additional research.

Population coding is widely regarded as a key mechanism for achieving reliable behavioral responses in the face of neuronal variability. But in standard reinforcement learning a flip-side becomes apparent. Learning slows down with increasing population size since the global reinforcement becomes less and less related to the performance of any single neuron. We show that, in contrast, learning speeds up with increasing population size if feedback about the populationresponse modulates synaptic plasticity in addition to global reinforcement. The two feedback signals (reinforcement and population-response signal) can be encoded by ambient neurotransmitter concentrations which vary slowly, yielding a fully online plasticity rule where the learning of a stimulus is interleaved with the processing of the subsequent one. The assumption of a single additional feedback mechanism therefore reconciles biological plausibility with efficient learning

In percutaneous intervention for treatment of coronary plaques, guidewire navigation is a primary procedure for stent delivery. Steering a flexible guidewire within coronary arteries requires considerable training, and the non-linearity between the control operation and the movement of the guidewire makes precise manipulation difficult. Here, we introduce a deep reinforcement learning(RL) framework for autonomous guidewire navigation in a robot-assisted coronary intervention. Using Rainbow, a segment-wise learning approach is applied to determine how best to accelerate training using human demonstrations with deep Q-learning from demonstrations (DQfD), transfer learning, and weight initialization. `State' for RL is customized as a focus window near the guidewire tip, and subgoals are placed to mitigate a sparse reward problem. The RL agent improves performance, eventually enabling the guidewire to reach all valid targets in `stable' phase. Our framework opens anew direction in the automation of robot-assisted intervention, providing guidance on RL in physical spaces involving mechanical fatigue.

Recent studies have successfully integrated large vision-language models (VLMs) into low-level robotic control by supervised fine-tuning (SFT) with expert robotic datasets, resulting in what we term vision-language-action (VLA) models. Although the VLA models are powerful, how to improve these large models during interaction with environments remains an open question. In this paper, we explore how to further improve these VLA models via Reinforcement Learning (RL), a commonly used fine-tuning technique for large models. However, we find that directly applying online RL to large VLA models presents significant challenges, including training instability that severely impacts the performance of large models, and computing burdens that exceed the capabilities of most local machines. To address these challenges, we propose iRe-VLA framework, which iterates between Reinforcement Learning and Supervised Learning to effectively improve VLA models, leveraging the exploratory benefits of RL while maintaining the stability of supervised learning. Experiments in two simulated benchmarks and a real-world manipulation suite validate the effectiveness of our method.

Smart robotics has advanced significantly with the integration of reinforcement learning and sensor technologies, enabling robots to perform tasks with greater intelligence, adaptability, and autonomy. Reinforcement learning allows robots to learn optimal actions through interaction with the environment, while sensors provide real-time information required for perception, navigation, object detection, and decision-making. The combination of these technologies improves the ability of robots to operate efficiently in dynamic and uncertain environments across various application domains.This chapter presents the fundamental concepts of smart robotics, reinforcement learning, and robotic sensor technologies, followed by an overview of sensor data processing, sensor fusion, and deep reinforcement learning techniques. The discussion explains how reinforcement learning utilizes sensor information to support autonomous navigation, adaptive control, and real-time decision-making. Simulation environments, digital twins, sim-to-real transfer learning, edge intelligence, cloud robotics, and performance evaluation methods are also discussed to highlight recent developments in intelligent robotic systems. Practical applications in industrial automation, healthcare, agriculture, logistics, search-and-rescue, and service robotics demonstrate the effectiveness of integrating reinforcement learning with sensor data. The chapter concludes by discussing current challenges and future research directions for developing reliable, efficient, and intelligent autonomous robotic systems.

Probability matching occurs when an action is chosen with a frequency equivalent to the probability of that action being the best choice. This sub-optimal behavior has been reported repeatedly by psychologist and experimental economist. We provide an evolutionary foundation for this phenomenon by showing that learning by reinforcement can lead to probability matching and, if learning occurs suffciently slowly, probability matching does not only occur in choice frequencies but also in choice probabilities. Our results are completed by proving that there exists no quasi-linear reinforcement learning specification such that behavior is optimal for all environments where counterfactuals are observed.Probability Matching; Reinforcement Learning

In this paper, we propose a multi-objective camera ISP framework that utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox that consist of network-based and conventional ISP tools. The proposed DRL-based camera ISP framework iteratively selects a proper tool from the toolbox and applies it to the image to maximize a given vision task-specific reward function. For this purpose, we implement total 51 ISP tools that include exposure correction, color-and-tone correction, white balance, sharpening, denoising, and the others. We also propose an efficient DRL network architecture that can extract the various aspects of an image and make a rigid mapping relationship between images and a large number of actions. Our proposed DRL-based ISP framework effectively improves the image quality according to each vision task such as RAW-to-RGB image restoration, 2D object detection, and monocular depth estimation.

Real-world robotic manipulation in homes and factories demands reliability, efficiency, and robustness that approach or surpass skilled human operators. We present a real-world reinforcement learning (RL) framework, RL-100, for achieving complete task success under a predefined evaluation protocol built on diffusion visuomotor policies. RL-100 unifies imitation and RL under a single clipped proximal policy optimization surrogate objective applied in the denoising process, yielding conservative, stable improvements across offline and online stages. To meet deployment latency, a lightweight consistency distillation compresses multistep diffusion into a one-step controller for high-frequency control. The framework is task, embodiment, and representation agnostic and supports both single-action and action-chunking control. We evaluated RL-100 on eight diverse real-robot tasks, from pushing and bowling to pouring, cloth folding, unscrewing, multistage juicing, and long-horizon box folding. Under our predefined protocol, RL-100 achieved 100% success in the evaluated trials (1000 of 1000 episodes), including up to 250 of 250 consecutive trials on one task. It matched or surpassed expert teleoperators in time to completion. Without retraining, a single policy attained ∼90% zero-shot success under environmental and dynamics shifts, adapted in a few-shot regime to substantial task variations (86.7%), and remained robust to human perturbations (about 96%). Our juicing robot served customers continuously for about 7 hours without failure when deployed zero-shot in a shopping mall. These results suggest a potential path to deployable robot learning by starting from human priors, aligning training objectives with human-grounded metrics, and reliably extending performance beyond human demonstrations.

This paper studies the analytical properties of the reinforcement learning model proposed in Erev and Roth (1998), also termed cumulative reinforcement learning in Laslier et al (2001). This stochastic model of learning in games accounts for two main elements: the law of effect (positive reinforcement of actions that perform well) and the law of practice (the magnitude of the reinforcement effect decreases with players' experience). The main results of the paper show that, if the solution trajectories of the underlying replicator equation converge exponentially fast, then, with probability arbitrarily close to one, all the realizations of the reinforcement learning process will, from some time on, lie within an " band of that solution. The paper improves upon results currently available in the literature by showing that a reinforcement learning process that has been running for some time and is found suffciently close to a strict Nash equilibrium, will reach it with probability one.Strict Nash Equilibrium, Reinforcement Learning

Robotic manipulation policies are commonly initialized through imitation learning, but their performance is limited by the scarcity and narrow coverage of expert data. Reinforcement learning can refine policies to alleviate this limitation, yet real-robot training is costly and unsafe, while training in simulators suffers from the sim-to-real gap. Recent advances in generative models have demonstrated remarkable capabilities in real-world simulation, with diffusion models in particular excelling at generation. This raises the question of how diffusion model-based world models can be combined to enhance pre-trained policies in robotic manipulation. In this work, we propose World4RL, a two-stage system-level framework for robotic manipulation that uses a frozen diffusion visual world model as an imagined environment for reinforcement-learning-based policy refinement. Unlike prior works that primarily employ world models for planning, our framework enables direct end-to-end policy optimization. World4RL is designed around two principles: pre-training a diffusion world model that captures diverse dynamics on multi-task datasets and refining policies entirely within a frozen world model to avoid online real-world interactions. Extensive simulation and real-world experiments demonstrate that World4RL provides high-fidelity environment modeling and enables consistent policy refinement, yielding significantly higher success rates compared to imitation learning and other baselines.

Reinforcement Learning (RL) has emerged as a transformative branch of artificial intelligence that enables autonomous systems to learn optimal decision-making strategies through continuous interaction with dynamic environments. Unlike conventional machine learning approaches that depend primarily on labeled datasets or predefined rules, reinforcement learning develops adaptive control policies by maximizing cumulative rewards, making it highly suitable for complex robotic and industrial applications. Recent advances in deep reinforcement learning, computational intelligence, and high-performance computing have significantly expanded the capability of autonomous systems to perform intelligent navigation, robotic manipulation, industrial process optimization, predictive maintenance, production scheduling, warehouse automation, and collaborative human–robot interaction. Simultaneously, the convergence of reinforcement learning with Industrial Internet of Things (IIoT), cyber-physical systems, digital twins, edge computing, and Industry 5.0 has accelerated the development of intelligent manufacturing ecosystems capable of autonomous adaptation, real-time optimization, and data-driven decision-making. This chapter presents a comprehensive foundation of reinforcement learning by integrating its theoretical principles, mathematical formulations, core learning components, value functions, policy optimization methods, exploration strategies, and classical reinforcement learning algorithms with modern deep reinforcement learning techniques. The discussion further examines practical implementation frameworks, simulation platforms, and representative applications in autonomous robotics and industrial systems while addressing critical challenges including sample inefficiency, reward engineering, safety-aware learning, computational complexity, explainability, and simulation-to-real-world deployment. Emerging research directions involving hierarchical reinforcement learning, multi-agent systems, federated learning, explainable artificial intelligence, and foundation models are also highlighted to demonstrate the future evolution of intelligent autonomous systems.

This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot’s I/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.

Autonomous robotic systems increasingly operate in dynamic, uncertain, and safety-critical environments where intelligent real-time decision making plays a decisive role in ensuring reliable and efficient task execution. Conventional control approaches often exhibit limited adaptability when environmental conditions change rapidly, creating the need for learning-based frameworks capable of continuous improvement through experience. Reinforcement Learning (RL) has emerged as a powerful paradigm for enabling autonomous robots to learn optimal decision policies by interacting with their surroundings and maximizing long-term cumulative rewards. Recent advances in Deep Reinforcement Learning (DRL) have further strengthened robotic perception, motion planning, autonomous navigation, manipulation, and cooperative multi-agent coordination by integrating deep neural networks with sequential decision-making models.The chapter presents a comprehensive review of real-time decision making using reinforcement learning in autonomous robotics, covering fundamental RL concepts, policy optimization strategies, deep reinforcement learning algorithms, robot perception, environment understanding, motion planning, dynamic obstacle detection, continuous learning, and multi-agent reinforcement learning. Critical aspects of safety-aware learning, simulation-to-real-world transfer, policy adaptation, and scalable deployment are examined to provide a holistic understanding of intelligent robotic systems. Practical applications across industrial automation, healthcare, warehouse logistics, autonomous vehicles, unmanned aerial systems, agriculture, and swarm robotics demonstrate the growing impact of reinforcement learning in solving complex real-world problems. Emerging research directions, including explainable reinforcement learning, edge artificial intelligence, digital twins, federated learning, and foundation models, are also discussed to highlight future opportunities for developing robust, adaptive, and trustworthy autonomous robotic systems. The chapter provides a structured reference for researchers, academicians, and practitioners seeking to understand recent developments and future trends in reinforcement learning-driven autonomous robotics.

In recent years, significant progress has been made in the field of robotic reinforcement learning (RL), enabling methods that handle complex image observations, train in the real world, and incorporate auxiliary data, such as demonstrations and prior experience. However, despite these advances, robotic RL remains hard to use. It is acknowledged among practitioners that the particular implementation details of these algorithms are often just as important (if not more so) for performance as the choice of algorithm. We posit that a significant challenge to the widespread adoption of robotic RL, as well as the further development of robotic RL methods, is the comparative inaccessibility of such methods. To address this challenge, we developed a carefully implemented library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment, a high-quality controller for a widely adopted robot, and a number of challenging example tasks. We provide this library as a resource for the community, describe its design choices, and present experimental results. Perhaps surprisingly, we find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation between 25 to 50 minutes of training per policy on average, improving over state-of-the-art results reported for similar tasks in the literature. These policies achieve perfect or near-perfect success rates, extreme robustness even under perturbations, and exhibit emergent recovery and correction behaviors. We hope these promising results and our high-quality open-source implementation will provide a tool for the robotics community to facilitate further developments in robotic RL. Our code, documentation, and videos can be found at https://serl-robot.github.io/

Smart robotics has emerged as a transformative field that integrates artificial intelligence, adaptive control, sensing technologies, and intelligent decision-making to develop autonomous systems capable of operating in complex and dynamic environments. Conventional robotic controllers encounter significant limitations when addressing nonlinear system dynamics, environmental uncertainty, changing payloads, sensor noise, and unpredictable external disturbances, creating a strong demand for advanced learning-driven control methodologies. This chapter presents a comprehensive examination of adaptive robotic control with particular emphasis on model-based reinforcement learning as a robust framework for achieving intelligent, data-efficient, and reliable robotic autonomy. Fundamental concepts of smart robotics, robotic dynamics modeling, uncertainty representation, predictive system modeling, and adaptive control architectures are systematically discussed to establish the theoretical foundation required for modern autonomous robotic systems. Critical challenges associated with perception, motion planning, computational complexity, communication reliability, energy efficiency, and real-time decision-making are analyzed to highlight existing research limitations and engineering constraints. The discussion extends toward hybrid learning-based controllers that integrate physics-based dynamic models with machine learning algorithms, enabling continuous adaptation, predictive control, enhanced robustness, and improved generalization across diverse operating conditions. Comparative insights into model-based and model-free reinforcement learning demonstrate the advantages of predictive environmental modeling in terms of sample efficiency, operational safety, learning speed, planning capability, and long-term policy optimization. Emerging developments involving probabilistic uncertainty estimation, digital twins, edge intelligence, and physics-informed artificial intelligence are examined as enabling technologies that strengthen adaptive robotic performance in real-world applications. The chapter establishes an integrated perspective connecting intelligent perception, predictive dynamics, adaptive learning, and autonomous control, providing a unified framework for designing next-generation robotic systems capable of achieving high precision, resilience, computational efficiency, and dependable operation across industrial automation, healthcare, agriculture, transportation, logistics, and collaborative human–robot environments.

Quadrupedal robots resemble the physical ability of legged animals to walk through unstructured terrains. However, designing a controller for quadrupedal robots poses a significant challenge due to their functional complexity and requires adaptation to various terrains. Recently, deep reinforcement learning, inspired by how legged animals learn to walk from their experiences, has been utilized to synthesize natural quadrupedal locomotion. However, state-of-the-art methods strongly depend on a complex and reliable sensing framework. Furthermore, prior works that rely only on proprioception have shown a limited demonstration for overcoming challenging terrains, especially for a long distance. This work proposes a novel quadrupedal locomotion learning framework that allows quadrupedal robots to walk through challenging terrains, even with limited sensing modalities. The proposed framework was validated in real-world outdoor environments with varying conditions within a single run for a long distance.

In this work, we propose a deep reinforcement learning (DRL) based reactive planner to solve large-scale Lidar-based autonomous robot exploration problems in 2D action space. Our DRL-based planner allows the agent to reactively plan its exploration path by making implicit predictions about unknown areas, based on a learned estimation of the underlying transition model of the environment. To this end, our approach relies on learned attention mechanisms for their powerful ability to capture long-term dependencies at different spatial scales to reason about the robot's entire belief over known areas. Our approach relies on ground truth information (i.e., privileged learning) to guide the environment estimation during training, as well as on a graph rarefaction algorithm, which allows models trained in small-scale environments to scale to large-scale ones. Simulation results show that our model exhibits better exploration efficiency (12% in path length, 6% in makespan) and lower planning time (60%) than the state-of-the-art planners in a $\text{130}\;\text{m}\times \text{100}\,\text{m}$ benchmark scenario. We also validate our learned model on hardware.

Reinforcement learning has become a key enabling technology for developing autonomous robotic systems capable of learning intelligent behaviors through continuous interaction with dynamic and uncertain environments. A critical challenge in this paradigm is achieving an effective balance between exploration and exploitation, where exploration discovers new knowledge while exploitation utilizes learned experience to maximize long-term performance. Efficient balancing of these strategies directly influences learning speed, policy stability, sample efficiency, and decision-making accuracy in robotic applications. This chapter presents a comprehensive overview of exploration and exploitation strategies in reinforcement learning, covering fundamental concepts, classical exploration methods, advanced techniques such as curiosity-driven and Bayesian exploration, exploitation through policy optimization, and deep reinforcement learning approaches. The discussion further examines applications in autonomous navigation, industrial, medical, agricultural, and service robotics, highlighting current challenges including sparse rewards, computational complexity, safe exploration, and simulation-to-real transfer. Comparative performance analysis of existing exploration strategies provides insights into their effectiveness across diverse robotic environments. The chapter concludes by discussing emerging research directions involving explainable reinforcement learning, foundation models, lifelong learning, and uncertainty-aware decision-making, providing a comprehensive foundation for designing robust, adaptive, and intelligent robotic systems.

This work contributes a novel deep navigation policy that enables collision-free flight of aerial robots based on a modular approach exploiting deep collision encoding and reinforcement learning. The proposed solution builds upon a deep collision encoder that is trained on both simulated and real depth images using supervised learning such that it compresses the high-dimensional depth data to a low-dimensional latent space encoding collision information while accounting for the robot size. This compressed encoding is combined with an estimate of the robot’s odometry and the desired target location to train a deep reinforcement learning navigation policy that offers low-latency computation and robust sim2real performance. A set of simulation and experimental studies in diverse environments are conducted and demonstrate the efficiency of the emerged behavior and its resilience in real-life deployments.