System architecture
IPDS block diagram
End to end, from physical sensing through to stored data. Seven layers, top to bottom.
01SensorsPhysical inputs
Vision
OV2640 Camera
320×240 RGB CMOS
MJPEG · ~15 FPS
MJPEG · ~15 FPS
Inertial
MPU6050 IMU
3-axis MEMS accel
±2 g · 16-bit · I²C
±2 g · 16-bit · I²C
Location
NEO-6M GPS
NMEA 0183 · UART
$GPGGA / $GPRMC
$GPGGA / $GPRMC
Time
DS3231 RTC
TCXO · I²C · CR2032
ISO-8601 timestamp
ISO-8601 timestamp
Parallel / DMAI²C · 0x69UART2 · 9600 baudI²C · shared bus
02FirmwareTwo ESP32 boards
Vision node
ESP32-CAM
AI-Thinker · 240 MHz · 4 MB PSRAM
esp_32_cam_final.ino
MJPEG HTTP server :81
esp_32_cam_final.ino
MJPEG HTTP server :81
Sensor node
ESP32 dev board
DevKit V1 · 240 MHz · I²C master
esp_32_final.ino
REST API :80 · GPS · RTC
esp_32_final.ino
REST API :80 · GPS · RTC
MJPEG stream · 2.4 GHz WiFiHTTP GET /query · WiFi
03TransportLocal WiFi network
Video
MJPEG over HTTP
multipart/x-mixed-replace
continuous stream, port 81
continuous stream, port 81
Sensor data
JSON over HTTP
GET /query?pothole_id=N
on demand, port 80
on demand, port 80
cv2.VideoCapture()urllib.request.urlopen()
04Computer visionPython processing hub
Inference
detector.py
Ultralytics YOLOv8m
conf threshold 0.25
conf threshold 0.25
Kalman
sort.py
7D Kalman filter
Hungarian matching · IoU
Hungarian matching · IoU
Tracking
tracker.py
Persistent IDs
unique-ID counting
unique-ID counting
Filters
filters.py
area ratio ≤ 0.25
aspect ratio ≤ 3.0
stationary ≤ 10 frames
aspect ratio ≤ 3.0
stationary ≤ 10 frames
centre crosses 0.75 × frame height → trigger
05Sensor fusionSeen and felt
Orchestrator
main.py
frame loop · ~15 FPS
one query per track
one query per track
IMU poll
Sensor burst
get_sensor_burst(): 5 reads
peak jerk (m/s³)
gate: jerk ≥ 1.5
peak jerk (m/s³)
gate: jerk ≥ 1.5
Fusion
calculate_severity()
jerk_norm = min(J / 20, 1)
S = 0.7·conf + 0.3·jerk_norm
S = 0.7·conf + 0.3·jerk_norm
confirmed eventseverityGPSRTC timestamp
06StorageOutputs on disk
Log
pothole_log.csv
outputs/logs/
time · ID · conf · jerk · severity · GPS
time · ID · conf · jerk · severity · GPS
Video
Annotated MP4
outputs/videos/
boxes, IDs, severity overlay
boxes, IDs, severity overlay
structured data for analysis and maps
07DownstreamPlanned, not built yet
Future
GIS / spatial database
PostGIS · GPS heatmap
severity clustering
severity clustering
Future
Navigation alerts
OpenStreetMap · V2X
driver warnings
driver warnings
Future
Municipal dashboard
repair prioritisation
work orders
work orders
Timeline of one pothole event
- 0 msOV2640 capture
- 10 msJPEG via DMA
- 25 msWiFi transmit
- 60 msYOLOv8 inference
- 65 msSORT assigns ID
- 150 msHTTP GET /query
- 170 msJSON reply
- 185 msCSV row written
Sensor fusion formula
jerk_norm = min(peak_jerk / 20, 1)
severity = 0.7 × confidence + 0.3 × jerk_norm
logged only if peak_jerk ≥ 1.5 m/s³
severity = 0.7 × confidence + 0.3 × jerk_norm
logged only if peak_jerk ≥ 1.5 m/s³