fix(enphase): settle the power buckets and derive load from the balance - #4430
Merged
Conversation
The instantaneous power sensors read the 15-minute energy bucket that had just closed, on the assumption it was complete. It is not - the cloud keeps back-filling a bucket for several minutes after it ends, so the first read returns roughly a third of the eventual figure and later corrects upward. Every rollover therefore produced a ~3x saw-tooth on pv_power, grid_power and battery_power alike; in a 13-hour log 23 buckets were re-read with a changed value, every one an increase. Reading a bucket that has settled removes it, at the cost of up to one extra bucket of lag. load_power was published from /app-api/<site>/get_latest_power, which returns site PRODUCTION, not consumption - it read 0-1 W all night and tracked the PV ramp by day. That fed PV-shaped data into load_today's gap filling. It is now the energy-balance residual of the other three channels, taken from the same settled bucket so the four sensors agree and a power-flow display balances. This is precisely how the cloud derives its own consumption channel - verified equal to within 1 Wh on 80 of 96 buckets - so reading that channel instead would gain nothing. Being the residual, load absorbs the timing and rounding skew between the micros, the CT clamps and the battery telemetry. On a site cycling 30 kWh a day through the battery to serve a 5 kWh house load, those terms dwarf the house term and the residual can go unphysical, so it is clamped at zero rather than rendering a negative house load. Aggregation does not rescue it (a 2-4 hour window is needed before it is reliably plausible) and neither does deltaing the daily totals, which update per channel on independent schedules and are noisier still. load_today is unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes two EnphaseCloud monitoring issues in Predbat by (1) deriving instantaneous power from a settled intra-day /today energy bucket to avoid saw-toothing, and (2) publishing load_power as the energy-balance residual of PV/grid/battery so it reflects house usage and keeps the power-flow model consistent.
Changes:
- Adjust power derivation to step back from the just-closed (still back-filled) bucket and use a settled bucket instead.
- Replace
load_powerpublishing fromget_latest_powerwith a derived residual (pv + grid + battery) clamped at zero. - Update docs and extend Enphase API tests to cover settled-bucket selection, start-of-day clamping, derived load, and negative residual clamping.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| docs/components.md | Documents the added 15–30 minute lag for power sensors and the residual-based load_power limitations. |
| apps/predbat/tests/test_enphase_api.py | Adds/updates tests for settled-bucket power selection and derived/clamped load behavior. |
| apps/predbat/predbat.py | Bumps Predbat version to v8.47.6. |
| apps/predbat/enphase.py | Implements settled-bucket selection and derives/publishes load_power from the bucket-based balance. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
springfall2008
added a commit
that referenced
this pull request
Aug 3, 2026
The cloud reports a schedule family as scheduleStatus "pending" while a change settles on the gateway - the normal state straight after any write Predbat makes. Only "active"/"enabled"/"supported"/"available" counted as supported, so a site with a perfectly good charge-from-grid family was judged incapable of it, automatic_config raised and run() returned False: Warn: Automatic configuration skipped - Charge-from-grid (CFG) scheduling not supported on this site, cannot configure seen on a site whose cfg family held an active schedule and whose profile reported scheduleSupported true for both cfg and dtg. "pending" now counts as supported, and so does any family that actually holds a schedule, whatever the status string says. "not_supported" still reports unsupported. Also aligns the livestream fallback test with the settled-bucket selection that landed on main in #4430: the helper froze time one bucket too early, and load now falls back to the energy-balance residual rather than being left empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
springfall2008
added a commit
that referenced
this pull request
Aug 4, 2026
* feat(enphase): read measured power from the Enlighten livestream The instantaneous power sensors were derived from the /today 15-minute energy buckets, which cannot produce a usable house load: consumption is the residual of much larger terms, so on a site cycling 30 kWh a day through the battery to serve a 5 kWh house load it is unphysical about a fifth of the time. load_power was instead published from get_latest_power, which reports PRODUCTION, not consumption - it reads 0-1 W all night and tracks the PV ramp by day. The Enlighten app streams a protobuf DataMsg once a second over MQTT-on-WebSockets from AWS IoT, carrying separately METERED pv, storage, grid and load channels plus SOC. Predbat now takes one reading per cycle - connect, first message, disconnect, the same lifecycle the web app uses - rather than holding the stream open and re-authorising every 900s. Credentials are bootstrapped from /pv/aws_sigv4/livestream.json using the gateway serial, which /today already carries, so no extra discovery call is needed. AWS IoT's custom authorizer is fed through the MQTT CONNECT username: the WebSocket takes no query string and no password, because a browser cannot set custom headers on a WebSocket. Verified against 379 frames captured from a real session: the channels satisfy load = pv + grid + battery to 0.0 W on every frame, load reads 154-1979 W where PV reads 4452-4933 W, and the signs already match Predbat's convention. One of those frames is committed as a test fixture. The bucket-derived values remain the fallback for pv/grid/battery when the stream is unavailable, so a failure degrades rather than blanking the sensors; load is left empty in that case rather than published wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [pre-commit.ci lite] apply automatic fixes * fix(enphase): keep livestream credentials out of the log The livestream bootstrap response carries aws_token_value and aws_digest, the live credentials for the account's AWS IoT stream. Debug API logging redacted only token/auth_token/access_token, so both were written out in full - and Predbat logs are routinely shared for debugging. The endpoint and topic are still logged so the call stays diagnosable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(enphase): treat a pending schedule family as supported The cloud reports a schedule family as scheduleStatus "pending" while a change settles on the gateway - the normal state straight after any write Predbat makes. Only "active"/"enabled"/"supported"/"available" counted as supported, so a site with a perfectly good charge-from-grid family was judged incapable of it, automatic_config raised and run() returned False: Warn: Automatic configuration skipped - Charge-from-grid (CFG) scheduling not supported on this site, cannot configure seen on a site whose cfg family held an active schedule and whose profile reported scheduleSupported true for both cfg and dtg. "pending" now counts as supported, and so does any family that actually holds a schedule, whatever the status string says. "not_supported" still reports unsupported. Also aligns the livestream fallback test with the settled-bucket selection that landed on main in #4430: the helper froze time one bucket too early, and load now falls back to the energy-balance residual rather than being left empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(enphase): publish grid power positive when exporting Predbat's convention is grid positive on EXPORT - web.py's power flow reads `grid_power >= 10` as exporting, and sigenergy.py documents the same - but enphase.py published `import - export`, so every Enphase user's grid power has been inverted, showing import where the flow diagram expects export. The livestream's grid channel is negative while exporting too, so both the measured and the bucket-derived paths needed flipping. The residual load derivation follows from the sign change: in Predbat's signs (grid +export, battery +discharge) the balance is pv + battery - grid, not pv + grid + battery. Load values are unchanged by this; only the grid sensor's sign moves. Battery power is left alone: `discharge - charge` (positive on discharge) already matches the core convention in inverter.py, which detects charging as `power < -threshold`, and sigenergy's documented mapping. Note that web.py's power flow reads battery the opposite way (`>= 10` as charging), which looks like a display bug affecting every integration rather than something to correct here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(enphase): never republish a stale livestream reading Livestream readings are instantaneous and carry no usable timestamp of their own (DataMsg.timestamp is a constant), so keeping one around past its moment presents an old measurement as current. Two ways that happened: - Caching. _load_cache restores each key's storage age into data_age, so a restart within ENPHASE_REFRESH_POWER left the refresh gate satisfied, get_live_power unrun and the restored reading published as live. live_power is now in-memory only. - A failed read left the previous reading in place, so a stream outage mid-run republished the last good measurement indefinitely, no restart required. A failure now drops it. Either way the sensors fall back to the bucket values, which lag but are genuinely current. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(enphase): age livestream readings out instead of dropping them Clearing the reading on a failed read made a single missed connection flip all four sensors onto the bucket fallback, which lags 15-30 minutes - a bigger visible step than simply holding the last measurement a little longer. A reading now stays in use for ENPHASE_LIVE_MAX_AGE_MINUTES and is ignored after that, so a blip is absorbed while genuinely old data still stops being presented as current. Readings are stamped on arrival because DataMsg.timestamp is a constant and the payload cannot date itself. They remain in-memory only, so nothing survives a restart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes two sensor bugs spotted in a deployed system: a saw-toothing
pv_power, and aload_powerthat tracked PV instead of house usage. Independent of #4429.1. Saw-tooth on the power sensors
interval_powerread the 15-minute bucket that had just closed, assuming it was complete. It isn't — the cloud keeps back-filling a bucket for several minutes after it ends, so the first read returns roughly a third of the eventual value and later corrects upward:In a 13-hour log, 23 buckets were re-read with a changed value — every one an increase. This affected
pv_power,grid_powerandbattery_powerequally, since all three come from the same call; PV was just the most visible.Fixed by stepping back to a bucket that has stopped changing. Observed revisions only ever touched the just-closed bucket, so one extra step suffices. Cost is up to one extra bucket of lag.
2. load_power was PV
It was published from
/app-api/<site>/get_latest_power, which returns production, not consumption. The night-time data is conclusive — it reads 0-1 W from midnight to 04:00, and tracks the PV ramp by day:automatic_configwires this to Predbat'sload_powerarg, which gap-fillsload_today— so PV-shaped data was being fed into house-load history.Load is now the energy-balance residual of the other three channels, from the same settled bucket, so the four sensors agree and a power-flow card balances. That is exactly how the cloud derives its own consumption channel — verified equal to within 1 Wh on 80 of 96 buckets — so reading that channel instead would gain nothing.
Known limitation, deliberately accepted
Being the residual, load absorbs all the timing and rounding skew between the micros, the CT clamps and the battery telemetry. On a site cycling 30 kWh/day through the battery to serve a 5 kWh house load, those terms dwarf the house term and the residual goes unphysical while the battery works hard — around 21% of buckets. It is clamped at zero so it can never render a negative house load, and it is accurate when the battery is calm (1 bad bucket in 8 with the battery idle; a steady 220-260 W through the morning).
Alternatives were measured and rejected:
load_todayis unaffected and remains accurate (buckets sum exactly to the daily total). Documented indocs/components.md, including the 15-30 minute lag now carried by all four power sensors.The real fix is the Enlighten live MQTT feed (
MeterSummaryDataover AWS IoT), which carries measuredpv/storage/grid/loadchannels directly. That is being investigated separately; this change is the best available from the/todayendpoint in the meantime.Testing
Five new tests written test-first, covering the settled-bucket selection, the start-of-day clamp, the derived load, the negative-residual clamp, and the consistency invariant (
load == pv + grid + battery). One existing test updated — it asserted the oldget_latest_powervalue for load../run_pre_commitpasses: all hooks green, full quick suite green.Version bumped v8.47.4 → v8.47.6.
Note:
get_latest_poweris still fetched and cached but no longer published. It is retained deliberately as the only genuinely live PV figure available — it will be needed to validate the MQTT feed'sagg_p_mwscaling.🤖 Generated with Claude Code