nebuleair_pro_4g/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

NebuleAir Pro 4G is an environmental monitoring system running on Raspberry Pi 4/CM4. It collects air quality and environmental data from multiple sensors and transmits it via 4G cellular modem. The system includes a self-hosted web interface for configuration and monitoring.

## Architecture

### Data Flow
1. **Data Collection**: Sensors are polled by individual Python scripts triggered by systemd timers
2. **Local Storage**: All sensor data is stored in SQLite database (`sqlite/sensors.db`)
3. **Data Transmission**: Main loop script reads aggregated data from SQLite and transmits via SARA R4/R5 4G modem
4. **Web Interface**: Apache serves PHP pages that interact with SQLite and execute Python scripts

### Key Components

**Sensors & Hardware:**
- NextPM (NPM): Particulate matter sensor via Modbus (ttyAMA0)
- Envea CAIRSENS: Gas sensors (NO2, H2S, NH3, CO, O3) via serial (ttyAMA2-5)
- BME280: Temperature, humidity, pressure via I2C (0x76)
- NSRT MK4: Noise sensor via I2C (0x48)
- SARA R4/R5: 4G cellular modem (ttyAMA2)
- Wind meter: via ADS1115 ADC
- MPPT: Solar charger monitoring

**Software Stack:**
- OS: Raspberry Pi OS (Linux)
- Web server: Apache2
- Database: SQLite3
- Languages: Python 3, PHP, Bash
- Network: NetworkManager (nmcli)

### Directory Structure
- `NPM/`: NextPM sensor scripts
- `envea/`: Envea sensor scripts
- `BME280/`: BME280 sensor scripts
- `sound_meter/`: Noise sensor code (C program)
- `SARA/`: 4G modem communication (AT commands)
- `windMeter/`: Wind sensor scripts
- `MPPT/`: Solar charger scripts
- `RTC/`: DS3231 real-time clock module scripts
- `sqlite/`: Database scripts (create, read, write, config)
- `loop/`: Main data transmission script
- `html/`: Web interface files
- `services/`: Systemd service/timer configuration
- `logs/`: Application logs

## Common Development Commands

### Database Operations
```bash
# Initialize database
/usr/bin/python3 /var/www/nebuleair_pro_4g/sqlite/create_db.py

# Set configuration
/usr/bin/python3 /var/www/nebuleair_pro_4g/sqlite/set_config.py

# Read data from specific table
/usr/bin/python3 /var/www/nebuleair_pro_4g/sqlite/read.py <table_name> <limit>
```

### Systemd Services
The system uses systemd timers for scheduling sensor data collection:

```bash
# View all NebuleAir timers
systemctl list-timers | grep nebuleair

# Check specific service status
systemctl status nebuleair-npm-data.service
systemctl status nebuleair-sara-data.service

# View service logs
journalctl -u nebuleair-npm-data.service
journalctl -u nebuleair-sara-data.service -f  # follow

# Restart services
systemctl restart nebuleair-npm-data.timer
systemctl restart nebuleair-sara-data.timer

# Setup all services
sudo /var/www/nebuleair_pro_4g/services/setup_services.sh
```

**Service Schedule:**
- `nebuleair-npm-data.timer`: Every 10 seconds (NextPM sensor)
- `nebuleair-envea-data.timer`: Every 10 seconds (Envea sensors)
- `nebuleair-sara-data.timer`: Every 60 seconds (4G data transmission)
- `nebuleair-bme280-data.timer`: Every 120 seconds (BME280 sensor)
- `nebuleair-mppt-data.timer`: Every 120 seconds (MPPT charger)
- `nebuleair-noise-data.timer`: Every 60 seconds (Noise sensor)
- `nebuleair-db-cleanup-data.timer`: Daily (database cleanup)

### Sensor Testing
```bash
# Test NextPM sensor
/usr/bin/python3 /var/www/nebuleair_pro_4g/NPM/get_data_modbus_v3.py

# Test Envea sensors (read reference/ID)
/usr/bin/python3 /var/www/nebuleair_pro_4g/envea/read_ref_v2.py

# Test Envea sensor values
/usr/bin/python3 /var/www/nebuleair_pro_4g/envea/read_value_v2.py

# Test BME280
/usr/bin/python3 /var/www/nebuleair_pro_4g/BME280/get_data_v2.py

# Test noise sensor
/usr/bin/python3 /var/www/nebuleair_pro_4g/sound_meter/NSRT_mk4_get_data.py

# Test RTC module
/usr/bin/python3 /var/www/nebuleair_pro_4g/RTC/read.py

# Test MPPT charger
/usr/bin/python3 /var/www/nebuleair_pro_4g/MPPT/read.py
```

### 4G Modem (SARA R4/R5)
```bash
# Send AT command
/usr/bin/python3 /var/www/nebuleair_pro_4g/SARA/sara.py ttyAMA2 "AT+CSQ" 5

# Check network connection
/usr/bin/python3 /var/www/nebuleair_pro_4g/SARA/sara_connectNetwork.py ttyAMA2 <networkID> 120

# Set server URL (HTTP profile 0 for AirCarto)
/usr/bin/python3 /var/www/nebuleair_pro_4g/SARA/sara_setURL.py ttyAMA2 data.nebuleair.fr 0

# Check modem status
/usr/bin/python3 /var/www/nebuleair_pro_4g/SARA/check_running.py
```

### Network Configuration
```bash
# Scan WiFi networks (saved to wifi_list.csv)
nmcli device wifi list ifname wlan0

# Connect to WiFi
sudo nmcli device wifi connect "SSID" password "PASSWORD"

# Create hotspot (done automatically at boot if no connection)
sudo nmcli device wifi hotspot ifname wlan0 ssid nebuleair_pro password nebuleaircfg

# Check connection status
nmcli device show wlan0
nmcli device show eth0
```

### Permissions & Serial Ports
```bash
# Grant serial port access (run at boot via cron)
chmod 777 /dev/ttyAMA* /dev/i2c-1

# Check I2C devices
sudo i2cdetect -y 1
```

### Logs
```bash
# View main application log
tail -f /var/www/nebuleair_pro_4g/logs/app.log

# View SARA transmission log
tail -f /var/www/nebuleair_pro_4g/logs/sara_service.log

# View service-specific logs
tail -f /var/www/nebuleair_pro_4g/logs/npm_service.log
tail -f /var/www/nebuleair_pro_4g/logs/envea_service.log

# Clear logs (done daily via cron)
find /var/www/nebuleair_pro_4g/logs -name "*.log" -type f -exec truncate -s 0 {} \;
```

## Configuration System

Configuration is stored in SQLite database (`sqlite/sensors.db`) in the `config_table`:

**Key Configuration Parameters:**
- `deviceID`: Unique device identifier (string)
- `modem_config_mode`: When true, disables data transmission loop (bool)
- `modem_version`: SARA R4 or R5 (string)
- `SaraR4_baudrate`: Modem baudrate, usually 115200 or 9600 (int)
- `SARA_R4_neworkID`: Cellular network ID for connection (int)
- `send_miotiq`: Enable UDP transmission to Miotiq server (bool)
- `send_aircarto`: Enable HTTP transmission to AirCarto server (bool)
- `send_uSpot`: Enable HTTPS transmission to uSpot server (bool)
- `npm_5channel`: Enable 5-channel particle size distribution (bool)
- `envea`: Enable Envea gas sensors (bool)
- `windMeter`: Enable wind meter (bool)
- `BME280`: Enable BME280 sensor (bool)
- `MPPT`: Enable MPPT charger monitoring (bool)
- `NOISE`: Enable noise sensor (bool)
- `latitude_raw`, `longitude_raw`: GPS coordinates (float)

Configuration can be updated via web interface (launcher.php) or Python scripts in `sqlite/`.

## Data Transmission Protocols

### 1. UDP to Miotiq (Binary Protocol)
- 100-byte fixed binary payload via UDP socket
- Server: 192.168.0.20:4242
- Format: Device ID (8 bytes) + signal quality + sensor data (all packed as binary)

### 2. HTTP POST to AirCarto
- CSV payload sent as file attachment
- URL: `data.nebuleair.fr/pro_4G/data.php?sensor_id={id}&datetime={timestamp}`
- Uses SARA HTTP client (AT+UHTTPC profile 0)

### 3. HTTPS POST to uSpot
- JSON payload with SSL/TLS
- URL: `api-prod.uspot.probesys.net/nebuleair?token=2AFF6dQk68daFZ` (port 443)
- Uses SARA HTTPS client with certificate validation (AT+UHTTPC profile 1)

## Important Implementation Notes

### SARA R4/R5 Modem Communication
- The main transmission script (`loop/SARA_send_data_v2.py`) handles complex error recovery
- Error codes from AT+UHTTPER command trigger specific recovery actions:
  - Error 4: Invalid hostname → Reset HTTP profile URL
  - Error 11: Server connection error → Hardware modem reboot
  - Error 22: PDP connection issue → Reset PSD/CSD connection (R5 only)
  - Error 26/44: Timeout/connection lost → Send WiFi notification
  - Error 73: SSL error → Re-import certificate and reset HTTPS profile
- Modem hardware reboot uses GPIO 16 (transistor control to cut power)
- Script waits 2 minutes after system boot before executing

### Serial Communication
- All UART ports must be enabled in `/boot/firmware/config.txt`
- Permissions reset after reboot, must be re-granted via cron @reboot
- Read functions use custom timeout logic to handle slow modem responses
- Special handling for multi-line AT command responses using `wait_for_lines` parameter

### Time Synchronization
- DS3231 RTC module maintains time when no network available
- RTC timestamp stored in SQLite (`timestamp_table`)
- Script compares server datetime (from HTTP headers) with RTC
- Auto-updates RTC if difference > 60 seconds
- Handles RTC reset condition (year 2000) and disconnection

### LED Indicators
- GPIO 23 (blue LED): Successful data transmission
- GPIO 24 (red LED): Transmission errors
- Uses thread locking to prevent GPIO conflicts

### Web Interface
- Apache DocumentRoot set to `/var/www/nebuleair_pro_4g`
- `html/launcher.php` provides REST-like API for all operations
- Uses shell_exec() to run Python scripts with proper sudo permissions
- Configuration updates modify SQLite database, not JSON files

### Security Considerations
- www-data user has sudo access to specific commands (defined in /etc/sudoers)
- Terminal command execution in launcher.php has whitelist of allowed commands
- No sensitive credentials should be committed (all in database/config files)

## Boot Sequence

1. Grant serial/I2C permissions (cron @reboot)
2. Check WiFi connection, start hotspot if needed (`boot_hotspot.sh`)
3. Start SARA modem initialization (`SARA/reboot/start.py`)
4. Systemd timers begin sensor data collection
5. SARA transmission loop begins after 2-minute delay

## Cron Jobs

Located in `/var/www/nebuleair_pro_4g/cron_jobs`:
- @reboot: Permissions setup, hotspot check, SARA initialization
- Daily 00:00: Truncate log files