Carnegie Mellon University

Autonomous John Deere tractor in an orchard by the National Robotics Engineering Center (NREC).

April 10, 2015

NREC to research Human Detection for farm safety

Researchers from NREC are studying how to detect and track people in orchards, fields and other agricultural settings. Their goal is to enable humans to safely work alongside robots on farms and in other non-urban environments. Eventually, their research could be applied to any situation where humans interact with robots. Funding for this two-year, $550K program comes from the US Department of Agriculture via the National Science Foundation. 

“Robotics offers a great opportunity to improve efficiency while also improving safety,” said Dr. Carl Wellington, the project’s principal investigator. “There has been a lot of interest recently in pedestrian detection for automotive applications, and we believe a similar effort is required in agricultural applications.” 

Detecting human beings is a challenging problem in any setting. But it’s particularly challenging to locate people on a farm or orchard, as opposed to a city street. Agricultural workers do not always walk upright like pedestrians in a crosswalk. Instead, they may bend, squat, climb ladders, or even lie on the ground if they’ve fallen or been injured. Branches, stalks, tree trunks, and leaves may cover part or all of a person’s body, which makes distinguishing workers from their surroundings much harder.


On the other hand, farms and orchards generally have a predictable, consistent appearance. Agricultural tasks are usually performed many times over in the same area (for instance, repeatedly mowing the rows of an orchard), which gives a robot a chance to familiarize itself with its surroundings. These characteristics can be used by the system to help distinguish workers from trees, crops, machinery, and other features of the farm environment. 

NREC’s approach combines several complementary methods for detecting pedestrians. Researchers will use data from low-cost stereo cameras to visually locate humans using shape and appearance. The cameras will also be used to estimate motion so GPS is not required. Because the background appearance in agricultural domains is generally consistent and static , people can be identified as anomalies in the environment as well as by tracking their movement over time. All of these indicators will be combined to improve the reliability of human detections. The system will use data-driven learning techniques to adapt itself to new locations, speeding this process through automated data collection. 

NREC researchers plan to test this suite of human detection and tracking techniques in a variety of farm environments. Their findings will be publicized via web site, papers, and a conference workshop. They will also generate and distribute benchmark labeled data sets for use by other researchers to help jump start additional work in this important problem domain. 

“We’d like these techniques to be extended to mines and construction sites,” said Dr. Wellington. “They could feed back into urban pedestrian detection, too. We think they can be used to improve safety wherever people and mobile robots work together.”