Skip to content

Commit 599ea61

Browse files
committed
docs: update README and user guide for v0.7.0 camera-supervised training
- Add v0.7.0 section with 92.9% PCK@20 result and new scripts - Add camera-supervised training section to user guide with step-by-step - Update release table (v0.7.0 as latest) - Update ADR count (62 → 79) - Update beta notice with camera ground-truth link Co-Authored-By: claude-flow <ruv@ruv.net>
1 parent 8dddbf9 commit 599ea61

2 files changed

Lines changed: 107 additions & 3 deletions

File tree

README.md

Lines changed: 48 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
> **Beta Software** — Under active development. APIs and firmware may change. Known limitations:
1010
> - ESP32-C3 and original ESP32 are not supported (single-core, insufficient for CSI DSP)
1111
> - Single ESP32 deployments have limited spatial resolution — use 2+ nodes or add a [Cognitum Seed](https://cognitum.one) for best results
12-
> - Camera-free pose accuracy is limited (2.5% PCK@20) — camera-labeled data significantly improves accuracy
12+
> - Camera-free pose accuracy is limited — use [camera ground-truth training](docs/adr/ADR-079-camera-ground-truth-training.md) for 92.9% PCK@20
1313
>
1414
> Contributions and bug reports welcome at [Issues](https://github.com/ruvnet/RuView/issues).
1515
@@ -56,6 +56,7 @@ RuView also supports pose estimation (17 COCO keypoints via the WiFlow architect
5656
> | **Through-wall** | Fresnel zone geometry + multipath modeling | Up to 5m depth |
5757
> | **Edge intelligence** | 8-dim feature vectors + RVF store on Cognitum Seed | $140 total BOM |
5858
> | **Camera-free training** | 10 sensor signals, no labels needed | 84s on M4 Pro |
59+
> | **Camera-supervised training** | MediaPipe + ESP32 CSI → 92.9% PCK@20 | 19 min on laptop |
5960
> | **Multi-frequency mesh** | Channel hopping across 6 bands, neighbor APs as illuminators | 3x sensing bandwidth |
6061
6162
```bash
@@ -174,6 +175,49 @@ All scripts support `--replay data/recordings/*.csi.jsonl` for offline analysis
174175

175176
</details>
176177

178+
### What's New in v0.7.0
179+
180+
<details open>
181+
<summary><strong>Camera Ground-Truth Training — 92.9% PCK@20</strong></summary>
182+
183+
**v0.7.0 adds camera-supervised pose training** using MediaPipe + real ESP32 CSI data:
184+
185+
| Capability | What it does | ADR |
186+
|-----------|-------------|-----|
187+
| **Camera ground-truth collection** | MediaPipe PoseLandmarker captures 17 COCO keypoints at 30fps, synced with ESP32 CSI | [ADR-079](docs/adr/ADR-079-camera-ground-truth-training.md) |
188+
| **ruvector subcarrier selection** | Variance-based top-K reduces input by 50% (70→35 subcarriers) | ADR-079 O6 |
189+
| **Stoer-Wagner min-cut** | Person-specific subcarrier cluster separation for multi-person training | ADR-079 O8 |
190+
| **Scalable WiFlow model** | 4 presets: lite (189K) → small (474K) → medium (800K) → full (7.7M params) | ADR-079 |
191+
192+
```bash
193+
# Collect ground truth (camera + ESP32 simultaneously)
194+
python scripts/collect-ground-truth.py --duration 300 --preview
195+
python scripts/record-csi-udp.py --duration 300
196+
197+
# Align CSI windows with camera keypoints
198+
node scripts/align-ground-truth.js --gt data/ground-truth/*.jsonl --csi data/recordings/*.csi.jsonl
199+
200+
# Train WiFlow model (start lite, scale up as data grows)
201+
node scripts/train-wiflow-supervised.js --data data/paired/*.jsonl --scale lite
202+
203+
# Evaluate
204+
node scripts/eval-wiflow.js --model models/wiflow-real/wiflow-v1.json --data data/paired/*.jsonl
205+
```
206+
207+
**Result: 92.9% PCK@20** from a 5-minute data collection session with one ESP32-S3 and one webcam.
208+
209+
| Metric | Before (proxy) | After (camera-supervised) |
210+
|--------|----------------|--------------------------|
211+
| PCK@20 | 0% | **92.9%** |
212+
| Eval loss | 0.700 | **0.082** |
213+
| Bone constraint | N/A | **0.008** |
214+
| Training time | N/A | **19 minutes** |
215+
| Model size | N/A | **974 KB** |
216+
217+
Pre-trained model: [HuggingFace ruv/ruview/wiflow-v1](https://huggingface.co/ruv/ruview)
218+
219+
</details>
220+
177221
### What's New in v0.5.5
178222

179223
<details>
@@ -294,7 +338,7 @@ See [ADR-069](docs/adr/ADR-069-cognitum-seed-csi-pipeline.md), [ADR-071](docs/ad
294338
|----------|-------------|
295339
| [User Guide](docs/user-guide.md) | Step-by-step guide: installation, first run, API usage, hardware setup, training |
296340
| [Build Guide](docs/build-guide.md) | Building from source (Rust and Python) |
297-
| [Architecture Decisions](docs/adr/README.md) | 62 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
341+
| [Architecture Decisions](docs/adr/README.md) | 79 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
298342
| [Domain Models](docs/ddd/README.md) | 7 DDD models (RuvSense, Signal Processing, Training Pipeline, Hardware Platform, Sensing Server, WiFi-Mat, CHCI) — bounded contexts, aggregates, domain events, and ubiquitous language |
299343
| [Desktop App](rust-port/wifi-densepose-rs/crates/wifi-densepose-desktop/README.md) | **WIP** — Tauri v2 desktop app for node management, OTA updates, WASM deployment, and mesh visualization |
300344
| [Medical Examples](examples/medical/README.md) | Contactless blood pressure, heart rate, breathing rate via 60 GHz mmWave radar — $15 hardware, no wearable |
@@ -1267,7 +1311,8 @@ Download a pre-built binary — no build toolchain needed:
12671311

12681312
| Release | What's included | Tag |
12691313
|---------|-----------------|-----|
1270-
| [v0.6.0](https://github.com/ruvnet/RuView/releases/tag/v0.6.0-esp32) | **Latest**[Pre-trained models on HuggingFace](https://huggingface.co/ruv/ruview), 17 sensing apps, 51.6% contrastive improvement, 0.008ms inference | `v0.6.0-esp32` |
1314+
| [v0.7.0](https://github.com/ruvnet/RuView/releases/tag/v0.7.0) | **Latest** — Camera-supervised WiFlow model (92.9% PCK@20), ground-truth training pipeline, ruvector optimizations | `v0.7.0` |
1315+
| [v0.6.0](https://github.com/ruvnet/RuView/releases/tag/v0.6.0-esp32) | [Pre-trained models on HuggingFace](https://huggingface.co/ruv/ruview), 17 sensing apps, 51.6% contrastive improvement, 0.008ms inference | `v0.6.0-esp32` |
12711316
| [v0.5.5](https://github.com/ruvnet/RuView/releases/tag/v0.5.5-esp32) | SNN + MinCut (#348 fix) + CNN spectrogram + WiFlow + multi-freq mesh + graph transformer | `v0.5.5-esp32` |
12721317
| [v0.5.4](https://github.com/ruvnet/RuView/releases/tag/v0.5.4-esp32) | Cognitum Seed integration ([ADR-069](docs/adr/ADR-069-cognitum-seed-csi-pipeline.md)), 8-dim feature vectors, RVF store, witness chain, security hardening | `v0.5.4-esp32` |
12731318
| [v0.5.0](https://github.com/ruvnet/RuView/releases/tag/v0.5.0-esp32) | mmWave sensor fusion ([ADR-063](docs/adr/ADR-063-mmwave-sensor-fusion.md)), auto-detect MR60BHA2/LD2410, 48-byte fused vitals, all v0.4.3.1 fixes | `v0.5.0-esp32` |

docs/user-guide.md

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1055,6 +1055,65 @@ See [ADR-071](adr/ADR-071-ruvllm-training-pipeline.md) and the [pretraining tuto
10551055
10561056
---
10571057
1058+
## Camera-Supervised Pose Training (v0.7.0)
1059+
1060+
For significantly higher accuracy, use a webcam as a **temporary teacher** during training. The camera captures real 17-keypoint poses via MediaPipe, paired with simultaneous ESP32 CSI data. After training, the camera is no longer needed — the model runs on CSI only.
1061+
1062+
**Result: 92.9% PCK@20** from a 5-minute collection session.
1063+
1064+
### Requirements
1065+
1066+
- Python 3.9+ with `mediapipe` and `opencv-python` (`pip install mediapipe opencv-python`)
1067+
- ESP32-S3 node streaming CSI over UDP (port 5005)
1068+
- A webcam (laptop, USB, or Mac camera via Tailscale)
1069+
1070+
### Step 1: Capture Camera + CSI Simultaneously
1071+
1072+
Run both scripts at the same time (in separate terminals):
1073+
1074+
```bash
1075+
# Terminal 1: Record ESP32 CSI
1076+
python scripts/record-csi-udp.py --duration 300
1077+
1078+
# Terminal 2: Capture camera keypoints
1079+
python scripts/collect-ground-truth.py --duration 300 --preview
1080+
```
1081+
1082+
Move around naturally in front of the camera for 5 minutes. The `--preview` flag shows a live skeleton overlay.
1083+
1084+
### Step 2: Align and Train
1085+
1086+
```bash
1087+
# Align camera keypoints with CSI windows
1088+
node scripts/align-ground-truth.js \
1089+
--gt data/ground-truth/*.jsonl \
1090+
--csi data/recordings/csi-*.csi.jsonl
1091+
1092+
# Train (start with lite, scale up as you collect more data)
1093+
node scripts/train-wiflow-supervised.js \
1094+
--data data/paired/*.jsonl \
1095+
--scale lite \
1096+
--epochs 50
1097+
1098+
# Evaluate
1099+
node scripts/eval-wiflow.js \
1100+
--model models/wiflow-supervised/wiflow-v1.json \
1101+
--data data/paired/*.jsonl
1102+
```
1103+
1104+
### Scale Presets
1105+
1106+
| Preset | Params | Training Time | Best For |
1107+
|--------|--------|---------------|----------|
1108+
| `--scale lite` | 189K | ~19 min | < 1,000 samples (5 min capture) |
1109+
| `--scale small` | 474K | ~1 hr | 1K-10K samples |
1110+
| `--scale medium` | 800K | ~2 hrs | 10K-50K samples |
1111+
| `--scale full` | 7.7M | ~8 hrs | 50K+ samples (GPU recommended) |
1112+
1113+
See [ADR-079](adr/ADR-079-camera-ground-truth-training.md) for the full design and optimization details.
1114+
1115+
---
1116+
10581117
## Pre-Trained Models (No Training Required)
10591118
10601119
Pre-trained models are available on HuggingFace: **https://huggingface.co/ruvnet/wifi-densepose-pretrained**

0 commit comments

Comments
 (0)