IARPA's HAYSTAC Program: Predicting People's Movements with AI and IoT

The US Intelligence Advanced Research Projects Activity (IARPA) is looking “to develop systems capable of modeling population movement patterns around the globe” using AI and sensors connected to the Internet of Things (IoT) and smart cities.

According to the Office of the Director of National Intelligence (ODNI), IARPA’s “Hidden Activity Signal and Trajectory Anomaly Characterization (HAYSTAC) program aims to establish ‘normal’ movement models across times, locations, and populations and determine what makes an activity atypical.

“Expansive data from the Internet of Things and Smart City infrastructures provides opportunities to build new models that understand human dynamics at unprecedented resolution and creates the responsibility to understand privacy expectations for those moving through this sensor-rich world.”

“This an unprecedented opportunity to understand how humans move, and HAYSTAC’s goal will be to build an understanding of what normal movement looks like at any given time and place”

Dr. Jack Cooper, IARPA

Leading the four-year HAYSTAC research program is Dr. Jack Cooper, who joined IARPA in 2020 after a stint at the National Geospatial-Intelligence Agency (NGA) in the Research Directorate, where he was a senior staff scientist for predictive analytics.

For the program manager, HAYSTAC represents “an unprecedented opportunity to understand how humans move, and HAYSTAC’s goal will be to build an understanding of what normal movement looks like at any given time and place.”

“With HAYSTAC, we have the opportunity to leverage machine learning and advances in artificial intelligence to understand mobility patterns with exceptional clarity,” said Dr. Cooper in a statement to the ODNI.

“The more robustly we can model normal movements, the more sharply we can identify what is out of the ordinary and foresee a possible emergency,” he added.

According to IARPA, “Current human mobility modeling techniques can provide high-level insight into human movement for the study of disease spread or population migration.”

However, “They don’t provide the complex, fine-grained modeling the Intelligence Community (IC) needs to identify more subtle anomalies with confidence.”

That’s where HAYSTAC and Dr. Cooper come in.

“With HAYSTAC, we have the opportunity to leverage machine learning and advances in artificial intelligence to understand mobility patterns with exceptional clarity”

Dr. Jack Cooper, IARPA

Dr. Cooper is also the program manager for at least two other IARPA research programs focused on detecting and characterizing human activities, which include:

• Space-Based Machine Automated Recognition Technique (SMART), which is using satellite imagery to detect, monitor, and characterize human construction projects, as well as natural processes like crop growth.
• Deep Intermodal Video Analytics (DIVA), which is creating automatic activity detectors that can watch hours of video and highlight the few seconds when a person or vehicle does a specific activity (e.g., carry something heavy, load it into a vehicle, then drive away).

“Internet of Thing devices are a growing source of data that can be collected to learn intent”

Dr. Catherine Marsh, IARPA

Speaking at the Department of Defense Intelligence Information System (DoDIIS) Worldwide Conference back in December, 2021, IARPA director Dr. Catherine Marsh foreshadowed the coming HAYSTAC program when she said:

“Internet of Thing devices are a growing source of data that can be collected to learn intent.

“Developing these new sensors and detectors, as well as thinking about clever ways to collect multi-modal data to reveal what our adversaries are attempting to hide from us, is at the very core of what our collection programs are aimed at doing.”

For its HAYSTAC program, IARPA has already awarded several contracts to big defense contractors and consulting firms with ties to academia, NGOs, and tech companies.

These contracts went to:

• Raytheon Technologies Research Center
• L3Harris Technologies, Inc.
• STR
• Kitware, Inc.
• Leidos, Inc.
• Novateur Research Solutions
• Deloitte Consulting LLP
• Raytheon BBN

“As the HAYSTAC systems mature, they will be evaluated based on probability of detection and false alarm performance in creating relevant alerts, ultimately seeking to identify 80% of anomalous activity while generating normal activity that is only 10% detectable,” according to the program description.