Data from: Heavy metal acrobatics: Learning dynamic control for skid-steer loaders
Data files
Oct 22, 2026 version files 1.23 MB
-
figures_scripts_and_data.zip
1.22 MB
-
README.md
7.49 KB
Abstract
Automation of heavy machinery offers the opportunity to improve productivity in mechanized industries such as construction, mining, and material handling. Yet automation of such machines is challenged by their complex dynamics, due in part to hydraulic actuation that involves fluids, flexible elements, and nonlinear delays. A concomitant challenge is the size and weight of these machines, which can make experiments on the physical systems exceedingly dangerous. Here, we show that Reinforcement Learning (RL) in simulation, followed by transfer of the learned policies from simulation to reality, can successfully automate acrobatic maneuvers that push a heavy construction machine to the limits of its actuation and have only previously been accomplished by a small number of elite human operators in official world-record performances. We train policies that perform autonomous acrobatic maneuvers on a John Deere 332G, an operational 5-tonne skid-steer loader (SSL). The policies can autonomously swing the machine up onto two wheels and, while balancing on two wheels, drive over rough terrain, trace out specified trajectories, spin around, and swing the bucket back and forth before landing it safely back onto four wheels. This work demonstrates that RL and sim-to-real transfer enable dynamic autonomous control of heavy machinery, directly contributing to greater automation in heavy equipment sectors.
Dataset DOI: 10.5061/dryad.d7wm37qhh
Usage
Requirements
Extract figures_scripts_and_data.zip
unzip figures_scripts_and_data.zip
cd figures_scripts_and_data
Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
Generate Figures
uv sync
bash generate_all_figures.sh
This generates all the figures under outputs
Content
figure_3.pkl
Overview
Figure 3 contains 4 graphs showing the deployment of the controller on real hardware. It compares the RL controller in reality to its behavior in simulation and to a Model Predictive Control (MPC) baseline when available:
- figure_3_swing_up: machine sensors during swing up maneuver in simulation and in reality. Only RL controller, no MPC.
- figure_3_soft_landing: machine sensors during soft landing maneuver in simulation and in reality. Only RL controller, no MPC. Compared to a sequence where the machine is let fall back on the ground with 0 control.
- figure_3_balance_in_place: machine sensors during balancing in place in simulation and in reality as well as compared to the MPC baseline.
- figure_3_balance_vel_tracking: machine sensors during tracking target velocities in simulation and in reality as well as compared to the MPC baseline.
It contains time series of all or a subset of the following machine sensors:
- Commands [percent]
- pitch [degree]
- wheel speed [radian/second]
- pitch rate [degree/second]
Pkl file content
The pkl file is structured as follows:
| maneuver type (swing_up, soft_landing, ...)
| | controller (rl: RL in reality, sim: RL in sim, mpc: MPC, ...)
| | | sensor type (Command, Pitch, ...)
| | | | x
| | | | y
| | text (contains text annotations like the roman numbers)
| | vertical lines (contains vertical lines of the transitions between controllers)
where x and y describe each point in the graph.
figure_4.pkl
Overview
Figure 4 contains 3 graphs showing the evaluation of the robustness of the controller and comparing it to simulation and the Model Predictive Control (MPC) baseline.
It contains time series of all or a subset of the following machine sensors:
- Spin rate [Rotation Per minute]
- Arm angle [degree]
- Commands [percent]
- pitch [degree]
- wheel speed [radian/second]
- pitch rate [degree/second]
Each graph corresponds to a specific test:
- figure_4_arm_movements: Moving the machine's arm while balancing. This shows the arm angle, the wheel command and the machine pitch.
- figure_4_spinning: Spinning the machine while balancing. This shows the spin rate, the wheel command and the machine pitch.
- figure_4_rough_terrain: Driving the machine over bumps while balancing. This shows the wheel speed for 3 different cases (RL in reality, RL in simulation and MPC baseline).
Pkl file content
In general the pkl file is structured as follows:
| maneuver type (arm_movements, spinning, ...)
| | controller (rl: RL in reality, sim: RL in sim, mpc: MPC, ...)
| | | sensor type (command, pitch, ...)
| | | | x
| | | | y
| | text (contains text annotations like the roman numbers)
| | vertical lines (contains vertical lines of the transitions between controllers)
where x and y describe each point in the graph.
Notice some minor inconsistencies for the rough terrain where an additional layer is added:
| rough_terrain
| | rl
| | | data
| | | | command (command, not plotted)
| | | | | x
| | | | | y
| | | target
| | | | command (target)
| | | | | x
| | | | | y
figure_5.pkl
Overview
Figure 5 contains 2 graphs about the model of the actuators:
- figure_5_response_to_different_loads: shows the wheel speed under different load conditions (on the ground moving the machine and off the ground rotating freely in the air).
- figure_5_actuator_model_predictions: shows the predictions of the learned actuator model against the measured wheel speed for different sets of commands.
It contains time series of the following machine sensors:
- command [percent]
- wheel speed [rad/second]
- Wheel speed on and off the ground and difference.
- Target wheel speed, predicted wheel speed and difference.
Pkl file content
The pkl file is structured as follows:
| figure (response_to_different_loads or actuator_model_predictions)
| | category (off_ground, on_ground or type of commands for actuator model)
| | | sensor type (command, wheel_speed)
| | | | x
| | | | y
where x and y describe each point in the graph.
figure_6.pkl
Overview
Figure 6 contains 7 figures about the Reinforcement Learning training in simulation:
- figure_6_episode_initialization_state_distribution: histogram of the initial pitch, pitch rate and wheel speed for all 4 maneuvers. The variables displayed are:
- pitch [degree]
- pitch rate [degree/second]
- wheel speed [radian/second]
- figure_6_reward_functions: Reward functions used for swing up and soft landing.
- figure_6_swing_up_learning: Curriculum and average discounted reward during swing up learning.
- figure_6_swing_up_final_performance: histogram of the final pitch angle [degree] after running the swing up controller in simulation on 1000 environments.
- figure_6_stabilize_and_balance_learning: termination probability and mean rewards for the stabilize and balance controller.
- figure_6_soft_landing_learning: termination probability and mean rewards for the soft landing controller.
- figure_6_balance_and_track_learning: termination probability and mean rewards for the balance and track velocities controller.
The rewards, termination probabilities and curriculum don't have units.
Pkl file content
For histograms, the pkl file is structured as follows:
| figure (response_to_different_loads or actuator_model_predictions)
| | maneuver (swing up, ...)
| | | sensor type (pitch, pitch rate, wheel_speed)
| | | | x
| | | | y
| | | | bar_width
where x is the abscissa, y is the height of the bar and bar_width is the width of the bar.
For the graphs, the pkl file is structured as follows:
| figure (response_to_different_loads or actuator_model_predictions)
| | maneuver (swing up, ...)
| | | metric (reward, curriculum, ...)
| | | | x
| | | | y
where x and y describe each point in the graph.
figure_s2.pkl
Overview
Figure S2 contains 2 graphs about the preliminary work done on another hardware platform (SuperMegaBot):
- figure_s2_velocity_tracking: Time series of machine sensors while tracking linear and angular velocities. The variables displayed are:
- Command (left and right) [percent]
- Base linear velocity [meter/second]
- Base yaw rate [radian/second]
- figure_s2_actuator_model: shows the predictions of the learned actuator model against the measured wheel speed:
- Command [percent]
- Base linear velocity (prediction, measured, error) [meter/second]
Pkl file content
The structure of this file is different than the previous ones:
| figure (velocity_tracking, actuator_model_predictions)
| | x (x values for actuator model predictions)
| | t (x values for velocity tracking in reality)
| | sim_t (x values for velocity tracking in simulation)
| | y values (base_vel_model, base_vel_real, ...)
