Hybrid Learning for Event-Based Visual Motion Detection and Tracking of Pedestrians

The Vision Zero Program seeks to eliminate traffic-related fatalities and serious injuries while promoting equitable, safe mobility. In this session, Dr. Cristian Axenie breaks down a low-power, neuromorphic edge solution built to detect and track pedestrians and bicyclists day and night. Developed for the TinyML Vision Zero San Jose Competition, the project relies on asynchronous event-based cameras paired with a highly efficient hybrid processing pipeline.

Key Takeaways

• A dual-pipeline edge solution combines SNNs for detection and event-based Expectation Maximization for tracking.
• The system uses Edge Impulse for rapid SNN deployment on the BrainChip Akida neural processor.
• Achieves robust pedestrian and bicyclist tracking in day and night conditions using sparse event data.
• Total system power draw is approximately 6 watts, allowing for scalable, city-level traffic safety infrastructure.

Workshop Format & Takeaways

The presentation walks through a complete end-to-end deployment lifecycle for an urban monitoring system. It covers data acquisition using custom DVS sensors mounted on urban intersections in Germany, the model design using Edge Impulse to generate a quantized Spiking Neural Network (a modified MobileNet architecture), and the tracking mechanism utilizing an event-based Expectation Maximization algorithm.

Crucially, the architecture distributes the workload hybrid-style: the detection model operates locally on the BrainChip Akida neural processor, while the continuous tracking algorithm executes via a Python flask server on a Raspberry Pi host. Finally, Axenie provided a deployment-ready hardware evaluation, detailing thermal resilience and robust tracking operations under 65°C oven stress tests while maintaining a stable, low 6-watt power footprint.

What This Means for Neuromorphic Computing

This implementation bridges the gap between experimental neuromorphic concepts and deployable civic infrastructure. By running a Spiking Neural Network (for detection) concurrently with a statistical tracker (for continuity) on embedded edge hardware, it demonstrates that neuromorphic pipelines can already meet rigorous real-world constraints—operating reliably under extreme California heat and dark nighttime conditions where conventional frame-based systems often fail.

The speaker noted that proving physical deployment metrics—like minimizing the energy footprint to a few watts—is critical for securing civic adoption. It highlights a viable path forward for integrating neuromorphic sensors into broad smart-city and traffic-control architectures. Most importantly, it proves that developers don’t have to wait for “perfect” fully-spiking toolchains; combining the energy efficiency of a neuromorphic accelerator with the reliable logic of a standard embedded processor yields a highly effective, market-ready hybrid solution today.

Resources

• Research group on Antifragility
• Code on EdgeImpulse
• Slides
• Paper: OBELISC: Oscillator-Based Modelling and Control using Efficient Neural Learning for Intelligent Road Traffic Signal Calculation, Axenie et al.