Wildlife conservation enters new era: Drones with AI now monitor birds, classify habitats and report findings

Researchers at the University of Missouri and the University of Illinois have developed a pioneering drone-based AI system that features a highly accurate and scalable method for waterfowl population monitoring and habitat analysis. The study, titled “New Methods for Waterfowl and Habitat Survey Using AI and Drone Imagery” and published in the journal Drones, outlines a robust end-to-end pipeline that integrates object detection, advanced image segmentation, spatial overlap correction, and automated flight reporting.

The system leverages cutting-edge deep learning algorithms to process aerial images captured by drones, enabling precise detection of birds like mallards and teal, classification of diverse wetland habitats, and elimination of redundant data across overlapping image sequences. A built-in large language model (LLM) component, based on GPT-4o, transforms raw detection data into human-readable survey reports, facilitating efficient communication for wildlife scientists and conservation agencies.

How does the AI-drone system detect birds and classify habitats?

The innovative system features a multi-module architecture combining bird detection, semantic habitat segmentation, and overlap region detection. The team evaluated several detection models, including YOLOv5, RetinaNet, and Faster R-CNN, before selecting the latter as the system’s core due to its superior performance in complex, vegetated environments and variable lighting conditions. Faster R-CNN achieved a mean average precision (mAP50) of 86.57% and showed particular robustness across varied drone altitudes and challenging habitats such as flooded crop fields.

To classify habitats, researchers developed a novel segmentation method by fusing Meta’s Segment Anything Model (SAM) with a ResNet50 classifier. SAM delineates pixel-level regions within each image, while the classifier assigns one of six habitat categories: open water, cropland, harvested crop, herbaceous, wooded, and other. The combined method delivered an overall accuracy of 85.1% and an average F1 score of 81.8%, outperforming traditional deep learning models like UNet, FCN, and PSPNet. The system particularly excelled in identifying open water, wooded areas, and croplands, although it showed lower accuracy for harvested crops due to visual ambiguity.

The dataset used for training and evaluation consisted of 572 drone images from ten Missouri conservation sites, collected during the nonbreeding season between October and March. These images captured representative habitat variability and bird behaviors under realistic survey conditions.

How are overlapping images handled to prevent double-counting?

Consecutive drone images often contain overlapping regions, raising the risk of double-counting birds or habitat features. To address this, the researchers embedded four different overlap detection methods in the pipeline:

1. GPS-Based Method – Utilizes drone metadata to geometrically estimate image overlaps. It is fast but susceptible to inaccuracy due to environmental noise.
2. SIFT-Based Method – Employs feature matching to detect common visual keypoints across images, but is computationally intensive.
3. RANSAC-Based Method – Refines SIFT outputs by eliminating outliers and optimizing homography transformations. It delivers high accuracy but is the slowest.
4. Bird-Location-Based Method – The most effective, this method compares spatial patterns of bird detections across images to identify overlaps, achieving high precision and speed simultaneously.

Performance tests on images captured at 30 m and 60 m altitudes showed that the bird-location-based method consistently offered the best trade-off between accuracy and efficiency, with prediction errors under 5% and execution times averaging just over one second.

What role does generative AI play in report generation?

One of the most innovative aspects of the system is the integration of OpenAI’s GPT-4o model for automated report writing. After processing a drone flight, the system generates a comprehensive report summarizing the number and location of birds, habitat classification results, flight metadata (altitude, duration, coordinates), and even real-time weather conditions using OpenWeatherMap API data.

Researchers pre-filled a data template and used structured prompts to feed validated detection outputs into GPT-4o, avoiding hallucinated content and ensuring factual consistency. The resulting text reports averaged 179 words in length and were generated in under six seconds per flight. Manual checks confirmed their accuracy, enabling seamless communication of complex ecological data to both technical teams and broader stakeholder groups.

This automation significantly reduces the time and expertise required to interpret large drone datasets, offering an accessible tool for conservation practitioners managing wetland ecosystems and migratory bird populations.

Broader implications and future directions

While this study focused on wetland bird species in Missouri, the framework is highly adaptable. With minimal retraining, it can be applied to other ecological monitoring scenarios such as seabird nesting, forest wildlife tracking, or even underwater habitat mapping. Researchers plan to publicly release the trained models and codebase, accelerating adoption in conservation biology and environmental management.