nebuleair_pro_4g/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

NebuleAir Pro 4G is an environmental monitoring system running on Raspberry Pi 4/CM4. It collects air quality and environmental data from multiple sensors and transmits it via 4G cellular modem. The system includes a self-hosted web interface for configuration and monitoring.

Architecture

Data Flow

1. Data Collection: Sensors are polled by individual Python scripts triggered by systemd timers
2. Local Storage: All sensor data is stored in SQLite database (sqlite/sensors.db)
3. Data Transmission: Main loop script reads aggregated data from SQLite and transmits via SARA R4/R5 4G modem
4. Web Interface: Apache serves PHP pages that interact with SQLite and execute Python scripts

Key Components

Sensors & Hardware:

• NextPM (NPM): Particulate matter sensor via Modbus (ttyAMA0)
• Envea CAIRSENS: Gas sensors (NO2, H2S, NH3, CO, O3) via serial (ttyAMA2-5)
• BME280: Temperature, humidity, pressure via I2C (0x76)
• NSRT MK4: Noise sensor via I2C (0x48)
• SARA R4/R5: 4G cellular modem (ttyAMA2)
• Senseair S88: CO2 sensor via Modbus RTU (any free ttyAMA — port configurable, see admin.html)
• CCS811: air-quality MOX sensor (TVOC + eCO2) via I2C (0x5A or 0x5B, configurable). Note: eCO2 is derived from VOCs, not a true NDIR CO2 measurement like the S88.
• Wind meter: via ADS1115 ADC
• MPPT: Solar charger monitoring

Custom PCB connector → UART port mapping:

PCB silkscreen	Linux device
NPM1	/dev/ttyAMA5
NPM2	/dev/ttyAMA4
NPM3	/dev/ttyAMA3
SARA	/dev/ttyAMA2

ttyAMA0 is the Pi's primary UART (header pins), not exposed on the custom PCB. When adding a new UART sensor (e.g. S88), it goes on one of the free NPM connectors.

Software Stack:

• OS: Raspberry Pi OS (Linux)
• Web server: Apache2
• Database: SQLite3
• Languages: Python 3, PHP, Bash
• Network: NetworkManager (nmcli)

Directory Structure

• NPM/: NextPM sensor scripts
• envea/: Envea sensor scripts
• BME280/: BME280 sensor scripts
• CCS811/: CCS811 air-quality sensor scripts (TVOC/eCO2, I2C)
• sound_meter/: Noise sensor code (C program)
• SARA/: 4G modem communication (AT commands)
• windMeter/: Wind sensor scripts
• MPPT/: Solar charger scripts
• RTC/: DS3231 real-time clock module scripts
• sqlite/: Database scripts (create, read, write, config)
• loop/: Main data transmission script
• html/: Web interface files
• services/: Systemd service/timer configuration
• logs/: Application logs

Common Development Commands

Database Operations

# Initialize database
/usr/bin/python3 /var/www/nebuleair_pro_4g/sqlite/create_db.py

# Set configuration
/usr/bin/python3 /var/www/nebuleair_pro_4g/sqlite/set_config.py

# Read data from specific table
/usr/bin/python3 /var/www/nebuleair_pro_4g/sqlite/read.py <table_name> <limit>

Systemd Services

The system uses systemd timers for scheduling sensor data collection:

# View all NebuleAir timers
systemctl list-timers | grep nebuleair

# Check specific service status
systemctl status nebuleair-npm-data.service
systemctl status nebuleair-sara-data.service

# View service logs
journalctl -u nebuleair-npm-data.service
journalctl -u nebuleair-sara-data.service -f  # follow

# Restart services
systemctl restart nebuleair-npm-data.timer
systemctl restart nebuleair-sara-data.timer

# Setup all services
sudo /var/www/nebuleair_pro_4g/services/setup_services.sh

Service Schedule:

• nebuleair-npm-data.timer: Every 10 seconds (NextPM sensor)
• nebuleair-envea-data.timer: Every 10 seconds (Envea sensors)
• nebuleair-sara-data.timer: Every 60 seconds (4G data transmission)
• nebuleair-bme280-data.timer: Every 120 seconds (BME280 sensor)
• nebuleair-ccs811-data.service: Daemon, continuous (CCS811 TVOC/eCO2 sensor — init once, reads every 10s; not a oneshot timer because the CCS811 must run continuously)
• nebuleair-mppt-data.timer: Every 120 seconds (MPPT charger)
• nebuleair-noise-data.timer: Every 60 seconds (Noise sensor)
• nebuleair-db-cleanup-data.timer: Daily (database cleanup)

Sensor Testing

# Test NextPM sensor
/usr/bin/python3 /var/www/nebuleair_pro_4g/NPM/get_data_modbus_v3.py

# Test Envea sensors (read reference/ID)
/usr/bin/python3 /var/www/nebuleair_pro_4g/envea/read_ref_v2.py

# Test Envea sensor values
/usr/bin/python3 /var/www/nebuleair_pro_4g/envea/read_value_v2.py

# Test BME280
/usr/bin/python3 /var/www/nebuleair_pro_4g/BME280/get_data_v2.py

# Test noise sensor
/usr/bin/python3 /var/www/nebuleair_pro_4g/sound_meter/NSRT_mk4_get_data.py

# Test RTC module
/usr/bin/python3 /var/www/nebuleair_pro_4g/RTC/read.py

# Test MPPT charger
/usr/bin/python3 /var/www/nebuleair_pro_4g/MPPT/read.py

4G Modem (SARA R4/R5)

# Send AT command
/usr/bin/python3 /var/www/nebuleair_pro_4g/SARA/sara.py ttyAMA2 "AT+CSQ" 5

# Check network connection
/usr/bin/python3 /var/www/nebuleair_pro_4g/SARA/sara_connectNetwork.py ttyAMA2 <networkID> 120

# Set server URL (HTTP profile 0 for AirCarto)
/usr/bin/python3 /var/www/nebuleair_pro_4g/SARA/sara_setURL.py ttyAMA2 data.nebuleair.fr 0

# Check modem status
/usr/bin/python3 /var/www/nebuleair_pro_4g/SARA/check_running.py

Network Configuration

# Scan WiFi networks (saved to wifi_list.csv)
nmcli device wifi list ifname wlan0

# Connect to WiFi
sudo nmcli device wifi connect "SSID" password "PASSWORD"

# Create hotspot (done automatically at boot if no connection)
sudo nmcli device wifi hotspot ifname wlan0 ssid nebuleair_pro password nebuleaircfg

# Check connection status
nmcli device show wlan0
nmcli device show eth0

Permissions & Serial Ports

# Grant serial port access (run at boot via cron)
chmod 777 /dev/ttyAMA* /dev/i2c-1

# Check I2C devices
sudo i2cdetect -y 1

Logs

# View main application log
tail -f /var/www/nebuleair_pro_4g/logs/app.log

# View SARA transmission log
tail -f /var/www/nebuleair_pro_4g/logs/sara_service.log

# View service-specific logs
tail -f /var/www/nebuleair_pro_4g/logs/npm_service.log
tail -f /var/www/nebuleair_pro_4g/logs/envea_service.log

# Clear logs (done daily via cron)
find /var/www/nebuleair_pro_4g/logs -name "*.log" -type f -exec truncate -s 0 {} \;

Configuration System

Configuration is stored in SQLite database (sqlite/sensors.db) in the config_table:

Key Configuration Parameters:

• deviceID: Unique device identifier (string)
• modem_config_mode: When true, disables data transmission loop (bool)
• modem_version: SARA R4 or R5 (string)
• SaraR4_baudrate: Modem baudrate, usually 115200 or 9600 (int)
• SARA_R4_neworkID: Cellular network ID for connection (int)
• send_miotiq: Enable UDP transmission to Miotiq server (bool)
• send_aircarto: Enable HTTP transmission to AirCarto server (bool)
• send_uSpot: Enable HTTPS transmission to uSpot server (bool)
• npm_5channel: Enable 5-channel particle size distribution (bool)
• envea: Enable Envea gas sensors (bool)
• windMeter: Enable wind meter (bool)
• BME280: Enable BME280 sensor (bool)
• MPPT: Enable MPPT charger monitoring (bool)
• NOISE: Enable noise sensor (bool)
• latitude_raw, longitude_raw: GPS coordinates (float)

Configuration can be updated via web interface (launcher.php) or Python scripts in sqlite/.

Data Transmission Protocols

1. UDP to Miotiq (Binary Protocol)

• 100-byte fixed binary payload via UDP socket
• Server: 192.168.0.20:4242
• Format: Device ID (8 bytes) + signal quality + sensor data (all packed as binary)

2. HTTP POST to AirCarto

• CSV payload sent as file attachment
• URL: data.nebuleair.fr/pro_4G/data.php?sensor_id={id}&datetime={timestamp}
• Uses SARA HTTP client (AT+UHTTPC profile 0)

3. HTTPS POST to uSpot

• JSON payload with SSL/TLS
• URL: api-prod.uspot.probesys.net/nebuleair?token=2AFF6dQk68daFZ (port 443)
• Uses SARA HTTPS client with certificate validation (AT+UHTTPC profile 1)

Important Implementation Notes

SARA R4/R5 Modem Communication

• The main transmission script (loop/SARA_send_data_v2.py) handles complex error recovery
• Error codes from AT+UHTTPER command trigger specific recovery actions:
  • Error 4: Invalid hostname → Reset HTTP profile URL
  • Error 11: Server connection error → Hardware modem reboot
  • Error 22: PDP connection issue → Reset PSD/CSD connection (R5 only)
  • Error 26/44: Timeout/connection lost → Send WiFi notification
  • Error 73: SSL error → Re-import certificate and reset HTTPS profile
• Modem hardware reboot uses GPIO 16 (transistor control to cut power)
• Script waits 2 minutes after system boot before executing

Serial Communication

• All UART ports must be enabled in /boot/firmware/config.txt
• Permissions reset after reboot, must be re-granted via cron @reboot
• Read functions use custom timeout logic to handle slow modem responses
• Special handling for multi-line AT command responses using wait_for_lines parameter

Time Synchronization

• DS3231 RTC module maintains time when no network available
• RTC timestamp stored in SQLite (timestamp_table)
• Script compares server datetime (from HTTP headers) with RTC
• Auto-updates RTC if difference > 60 seconds
• Handles RTC reset condition (year 2000) and disconnection

LED Indicators

• GPIO 23 (blue LED): Successful data transmission
• GPIO 24 (red LED): Transmission errors
• Uses thread locking to prevent GPIO conflicts

Web Interface

• Apache DocumentRoot set to /var/www/nebuleair_pro_4g
• html/launcher.php provides REST-like API for all operations
• Uses shell_exec() to run Python scripts with proper sudo permissions
• Configuration updates modify SQLite database, not JSON files

Security Considerations

• www-data user has sudo access to specific commands (defined in /etc/sudoers)
• Terminal command execution in launcher.php has whitelist of allowed commands
• No sensitive credentials should be committed (all in database/config files)

Boot Sequence

1. Grant serial/I2C permissions (cron @reboot)
2. Check WiFi connection, start hotspot if needed (boot_hotspot.sh)
3. Start SARA modem initialization (SARA/reboot/start.py)
4. Systemd timers begin sensor data collection
5. SARA transmission loop begins after 2-minute delay

Cron Jobs

Located in /var/www/nebuleair_pro_4g/cron_jobs:

• @reboot: Permissions setup, hotspot check, SARA initialization
• Daily 00:00: Truncate log files