Field testing Crusher. The DARPA UPI program conducted over a dozen field tests of the Crusher unmanned ground vehicle (UGV). Each one cost around $100K and consumed thousands of man-hours. Using a virtual proving ground could have reduced this to a few targeted validation tests, saving millions of dollars and thousands of man-hours of time.
The simulated test environment will incorporate sensor data like this three-dimensional LADAR scan.
||Field Testing Goes Virtual
NREC researchers are pioneering a new way to test autonomous unmanned systems: a virtual proving ground that uses recorded sensor data to simulate conditions in the field. A $2.5 million award from the Department of Defense’s Test Resource Management Center (TRMC) underwrites this work.
“Field tests help us to develop robust autonomous systems,” said Dr. Alonzo Kelly, the project’s leader. “But they also burn time and money and lack rigor and repeatability.” A team can spend weeks in the field running, testing and debugging autonomous systems – not to mention fixing hardware problems! Even if everything goes right, it’s impossible to test a system under all operating conditions and control every environmental factor that affects its performance.
The result? Higher costs, longer development times, and less reliable systems.
To change this, Kelly is taking field tests virtual. The virtual proving ground is a data-driven simulation of a robot’s operating environment. It feeds simulated perception, positioning, and vehicle dynamics into the robot’s autonomy system and responds to the system’s actions. In effect, it fools the autonomy system into thinking that the robot is moving around the test site. The development team can then see how the system reacts under test conditions.
“The real world is too rich to model from scratch,” said Kelly. “So we model it with data.” Sensor data that was recorded at the test site is the heart of the virtual proving ground. Using recorded data eliminates the need to construct complex, physics-based models, allowing high fidelity simulations to be created quickly and inexpensively.
The project’s goal is to make it faster, easier, safer, and less expensive to develop reliable, high-performance unmanned systems. The virtual proving ground would not totally eliminate field testing, but it would dramatically reduce it. Development teams could use it to test earlier and more often during the process. They could see how systems perform under conditions that might not occur during an actual field test – for example, under different weather conditions. They could also perform experiments that are too dangerous or expensive to do in the field, such as testing a system to destruction.
“We believe that the virtual proving ground will revolutionize the development of all unmanned autonomous systems, whether they move over the ground, in the sea, or through the air,” said Kelly.
Note: This project is funded by the Test Resource Management Center (TRMC) Test and Evaluation/Science & Technology (T&E/S&T) Program through the U.S. Army Program Executive Office for Simulation, Training and Instrumentation (PEO STRI) under Contract No. W900KK-11-C-0026.