Skip to content

Divide commercial kWh on survey electricity or other inputs, not on bills (#899, #705) - #1305

Merged
WesIngwersen merged 8 commits into
mainfrom
commercial_kwh_follow_inputs
Nov 13, 2026
Merged

WesIngwersen merged 8 commits into
mainfrom
commercial_kwh_follow_inputs

Conversation

@WesIngwersen

@WesIngwersen WesIngwersen commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Part of #899 and #705, stacked on #1306 (#1304). Results for the stack are in this comment.

B driver tracker

The B driver tracker on this branch shows the stack's test build (v1.0.0_54dbcea with this flag on), including the N_d volatility column from #1308, which this branch merges until #1308 lands on main. Across the 132 commercial-class commodities the median N_d volatility is 0.100 against 0.106 on fd271dc, lower for 78 and higher for 23. Cable, broadcasting, elementary and secondary schools and securities brokerage drop off the high-volatility table.

What this does

In the generation and delivery split (electricity_delivery.py), each buyer's kWh are bill / class price, scaled so the class equals EIA Table 2.2. Every commercial buyer pays one price, so its share of commercial generation, and of the emissions that come with it, is its share of commercial bills. Balancing sets those bills without regard to kWh: other real estate's fell from 20.5% to 13.5% of EIA commercial revenue over 2017-2024 while its inputs grew 47%.

With the flag commercial_kwh_follow_intermediate_inputs (off), commercial kWh are divided on a basis instead of on the bill (commercial_kwh_basis):

The class's kWh stay on EIA, and only the split between commercial buyers changes. Bills, value added, every other class (manufacturing and utilities included) and the tables are untouched. The base years come from the model's own 2017 and 2022 tables (commercial_electricity_base.csv, written by write_commercial_electricity_base), so 2017 is unchanged and 2022 has no step.

Checks, flag off vs on, 2017-2024 (fd271dc)

  • Outside the commercial class every buyer's bill, kWh, generation and delivery are identical, every year. Commercial bills and kWh are unchanged, and total emissions are identical.
  • 2017 is identical.
  • One buyer is floored in 2023 and three in 2024 ($0.25bn and $0.42bn of bills).
  • Other real estate's share of commercial generation in 2024 is 14.6% (14.0% off), and its N rises 1%.

Delivery margins. The implied price of most buyers stays close to the class price: kWh-weighted p10-p90 of 10.1-13.2 ¢/kWh in 2024 against a class price of 12.3. CBECS 2018 has 9.19-11.22 across building activities. The exceptions are buyers whose bill carries the AIES 2024 step (#1304): scientific R&D at 22.5 ¢/kWh and trucking at 20.0 in 2024, and warehousing, whose 2020-2023 bill drifts down while its survey kWh rise, at 8.2-8.6.

Tests

split_bills with a basis moves kWh between commercial buyers only and keeps buyers outside it exact; the basis follows the survey through 2022 and other inputs from 2022; a survey withheld in any year gives no path.

🤖 Generated with Claude Code

@WesIngwersen

Copy link
Copy Markdown
Member Author

Commercial electricity by sector, before and after

Flag off (before) and on (after), on the v1.0.0_fd271dc tables. Commercial bills don't change, the class total stays on EIA's kWh, and only the split between commercial buyers moves. The same table on #1306's tables follows when its rebuild finishes.

2024

sector group TWh, before → after generation $bn delivery $bn ¢ per kWh
retail trade 304.0 → 306.9 16.87 → 17.03 20.53 → 20.37 12.3 → 12.2
accommodation and food 225.5 → 223.7 12.51 → 12.42 15.23 → 15.33 12.3 → 12.4
real estate (other real estate) 202.9 → 210.6 11.26 → 11.69 13.71 → 13.28 12.3 → 11.9
health care 123.3 → 133.2 6.84 → 7.39 8.33 → 7.77 12.3 → 11.4
wholesale trade 101.5 → 98.8 5.63 → 5.48 6.86 → 7.00 12.3 → 12.6
information 96.7 → 100.1 5.37 → 5.37 6.53 → 6.53 12.3 → 11.9
transportation and warehousing 89.3 → 59.7 4.96 → 3.31 6.03 → 7.68 12.3 → 18.4
government 67.6 → 65.6 3.75 → 3.64 4.56 → 4.67 12.3 → 12.7
professional services 47.3 → 46.0 2.62 → 2.37 3.19 → 3.45 12.3 → 12.7
management of companies 42.6 → 44.7 2.36 → 2.48 2.88 → 2.76 12.3 → 11.7
other services 42.0 → 48.4 2.33 → 2.68 2.84 → 2.49 12.3 → 10.7
arts and recreation 34.8 → 41.5 1.93 → 2.30 2.35 → 1.98 12.3 → 10.3
finance and insurance 26.7 → 29.1 1.48 → 1.61 1.81 → 1.68 12.3 → 11.3
administrative and waste 21.9 → 25.4 1.21 → 1.41 1.48 → 1.28 12.3 → 10.6
rental and leasing 17.7 → 8.6 0.98 → 0.48 1.19 → 1.70 12.3 → 25.3
education 4.8 → 6.2 0.27 → 0.34 0.33 → 0.25 12.3 → 9.7
all commercial 1,450.9 → 1,450.9 80.52 → 80.16 98.01 → 98.36 12.3

Other years. 2017 is identical, by construction. In 2022 the largest shifts are health care (100.6 to 113.4 TWh), arts and recreation (27.5 to 35.1) and real estate (210.1 to 203.7). All groups stay between 8.6 and 13.8 ¢/kWh, except "other", a few unmapped codes on 4 TWh. In 2020 the range is 9.4 to 12.6 ¢.

What's wrong in 2024. Transportation and warehousing (18.4 ¢) and rental and leasing (25.3 ¢) are the AIES 2024 step in their bills (#1304): their kWh follow their survey path to 2022 and then their inputs, but their bills don't, so the excess becomes delivery. CBECS 2018 prices these buildings at 9.9-11.4 ¢/kWh. #1306 removes the step from the bills, so these two should come back near the class price on its tables.

🤖 Generated with Claude Code

WesIngwersen added a commit that referenced this pull request Nov 13, 2026
… --vintage (#1305)

The trucking-to-warehousing electricity mapping in #1306 applies in every
survey year, so 2022's tables differ slightly from fd271dc's. The base file
holds the 2022 inputs the basis moves on from, so it comes from the build
the stack will be pinned to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WesIngwersen

Copy link
Copy Markdown
Member Author

Results for the stack (#1306 + #1305)

Test rebuild run 43461231130, build v1.0.0_54dbcea: #1306 with hold_services_seed_at_last_sas_year on in the nine v1.0 configs. All 16 jobs passed, and nothing was uploaded. "Hold" below is that build, and "hold + #1305" adds commercial_kwh_follow_intermediate_inputs, which runs after the balance. Each is compared with v1.0.0_fd271dc.

Balance: passes in every year

q = L_dom y passes for 2017-2024 with no failing commodity, both as built and with #1305 on.

Bills: the AIES 2024 step is gone

Electricity bill as a share of output:

2023 2024, fd271dc 2024, hold
warehousing 0.90% 1.41% 0.83%
truck transportation 0.77% 1.09% 0.78%
scientific R&D 0.39% 0.59% 0.38%
equipment rental 0.39% 0.79% 0.40%
legal services 0.11% 0.04% 0.11%
management consulting 0.27% 0.14% 0.29%
software publishers 0.05% 0.02% 0.05%
other real estate 1.37% 1.31% 1.37%

Commercial electricity by sector, before and after #1305 (hold tables, 2024)

Bills don't change, and commercial kWh stay on EIA's 1,451 TWh. Outside the commercial class every buyer's bill, kWh, generation and delivery are identical, in every year. 2017 is identical.

sector group TWh, before → after generation $bn delivery $bn ¢ per kWh
retail trade 303.3 → 307.1 16.83 → 17.04 20.66 → 20.44 12.4 → 12.2
accommodation and food 240.6 → 223.9 13.35 → 12.42 16.38 → 17.31 12.4 → 13.3
real estate (other real estate) 210.7 → 210.6 11.69 → 11.69 14.35 → 14.35 12.4 → 12.4
health care 122.8 → 133.4 6.81 → 7.40 8.36 → 7.77 12.4 → 11.4
wholesale trade 100.9 → 99.0 5.60 → 5.49 6.87 → 6.97 12.4 → 12.6
information 99.5 → 100.1 5.52 → 5.54 6.77 → 6.76 12.4 → 12.3
government 66.4 → 65.7 3.68 → 3.64 4.52 → 4.56 12.4 → 12.5
transportation and warehousing 64.9 → 58.4 3.60 → 3.24 4.42 → 4.77 12.4 → 13.7
other services 50.5 → 48.2 2.80 → 2.67 3.44 → 3.56 12.4 → 13.0
professional services 45.0 → 46.1 2.49 → 2.56 3.06 → 3.00 12.4 → 12.1
management of companies 42.0 → 44.8 2.33 → 2.48 2.86 → 2.71 12.4 → 11.6
arts and recreation 33.8 → 41.8 1.88 → 2.32 2.30 → 1.86 12.4 → 10.0
finance and insurance 29.0 → 29.1 1.61 → 1.62 1.97 → 1.96 12.4 → 12.3
administrative and waste 25.3 → 25.4 1.40 → 1.41 1.72 → 1.71 12.4 → 12.3
rental and leasing 9.7 → 8.7 0.54 → 0.48 0.66 → 0.72 12.4 → 13.8
education 4.4 → 6.1 0.24 → 0.34 0.30 → 0.20 12.4 → 8.8
all commercial 1,450.9 → 1,450.9 80.52 → 80.51 98.80 → 98.81 12.4

On fd271dc tables, transportation and warehousing had come out at 18.4 ¢ and rental and leasing at 25.3 ¢, because the AIES step was still in their bills. With the hold, every group sits between 8.8 and 14.2 ¢ in every year, and the kWh-weighted 10th-90th percentile of buyers' prices in 2024 is 10.6-14.2 ¢. CBECS 2018 prices commercial building types at 9.2-11.4 ¢. One buyer is floored in each of 2023 and 2024 ($0.25bn of bills).

N

Change against fd271dc, 404 sectors:

2020 2023 2024
hold: output-weighted change -0.00% -0.01% +1.16%
hold: sectors moving >1% / >5% 0 / 0 0 / 0 112 / 35
hold + #1305: output-weighted change -0.16% +0.26% +1.53%
hold + #1305: sectors moving >1% / >5% 68 / 12 50 / 12 119 / 42

The hold moves only 2024, as it should. There it undoes the AIES step: sectors whose 2024 bills it inflated fall (consumer goods rental -13%, equipment rental -10%, warehousing -8%, trucking -5%, couriers -4%), and those it deflated rise (cable +23%, broadcasting +19%, spectator sports +16%, legal +12%). The change in N from 2023 to 2024 shrinks: its median across all 404 sectors falls from 4.6% to 4.2%. Spectator sports goes from -14.3% to -0.9%, consumer goods rental from +11.7% to -2.9%, legal from -15.7% to -5.3%, cable from -34.3% to -19.5%. Electricity generation's N doesn't move.

Volatility

The IV score (spend weights) sums to 7.90 under hold and under hold + #1305, against 7.97 on fd271dc, with the same top 30. A service sector's B is its own fuel and refrigerants, so neither change reaches its volatility score. They show up in the volatility of N_d (the column #1308 adds), for the 132 commercial-class commodities:

N_d volatility fd271dc hold #1305 alone hold + #1305
median 0.106 0.105 0.101 0.100
mean 0.117 0.114 0.111 0.109
lower / higher than fd271dc 39 / 27 74 / 22 78 / 23
automotive repair 0.168 0.119 0.097 0.091
lessors of intangible assets 0.171 0.121 0.167 0.121
cable programming 0.180 0.132 0.176 0.132
warehousing 0.135 0.124 0.114 0.115
equipment rental 0.135 0.105 0.107 0.109
software publishers 0.204 0.208 0.202 0.188
scientific R&D 0.068 0.067 0.071 0.061
hospitals 0.071 0.069 0.066 0.064
other real estate 0.082 0.075 0.080 0.083

Still to do

Turning the flags on is a re-pin, as in #1302. v1.0.0_54dbcea has #1306's hold on but not #1303's service trade flags (built separately as v1.0.0_826ea66), so the re-pin needs one combined rebuild with both, plus commercial_electricity_base.csv rewritten from it (write_commercial_electricity_base --vintage). This PR's base file is from 54dbcea.

🤖 Generated with Claude Code

@WesIngwersen
WesIngwersen marked this pull request as ready for review November 13, 2026 17:50
WesIngwersen added a commit that referenced this pull request Nov 13, 2026
…bcea with #1305

IV score, volatility, N_d volatility, driving factor and emissions after the
arrow now read the services seed hold (#1306) with commercial kWh divided on
survey electricity or other inputs (#1305). Cable, broadcasting, elementary
and secondary schools and securities brokerage drop to 0.200 or below and
leave the high-volatility table; none had an issue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@bl-young bl-young left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Problem framing looks right, and the two PRs belong together once the flags are on.

#1306 / #1304. AIES 2024 against AIES 2023 is not a growth step for many seeded cells: 20/48 electricity reporters move past their own SAS-era yearly range, and other items jump 40-80% in one year, so the relative index divides by a bill that is changing for survey reasons. Holding every AIES year on the SAS 2022 index (Step 5 still sets levels) is the conservative fix the issue asked for. Trucking -> warehousing for electricity is a separate, reasonable claim (terminals/depots, not fleet); CBECS kWh-per-worker vs BEA 2017 for those two supports not understating the 2017 levels.

#1305 / #899+#705. With one commercial price, kWh shares are bill shares, and the balance moves bills without watching physical use. Re-splitting commercial kWh on SAS electricity through 2022 (then other intermediate inputs), while leaving bills, class EIA totals, and other classes alone, is a clear improvement over that: the split follows something that at least moves with electricity spending or activity, instead of with whatever residual the balance left in the bill. This is not the commercial bill seed (#899 still open for that).

Coupling. #1305 alone on fd271dc tables is what produces the 18.4 / 25.3 c/kWh delivery outliers; the stack rebuild (v1.0.0_54dbcea + #1305) brings every group to 8.8-14.2 c against CBECS building-type prices of ~9.2-11.4. That is better-behaved than bill-proportional kWh under the AIES step. Approving both together for a flags-on read is correct; landing them flag-off first (as written) matches the #1260 -> #1302 pattern.

Evidence is enough to approve as an improvement over what we had, with a few watch-outs:

  • Balance: q = L_dom y clean 2017-2024 with and without #1305 -- good.
  • Hold does what it claims on bills: warehousing / trucking / R&D / rental return to ~2023 shares of output; the inflated/deflated 2024 N movers line up with the AIES step undoing.
  • Delivery margins with both on are in a believable band; the #1305-alone failure mode is documented and fixed by #1306.
  • N_d volatility (commercial class) improves under the stack; IV barely moves, which is expected if B is own fuel/refrigerants. Soft corroboration that the reassignment is steadier for users than bill share was.
  • Outside commercial / 2017 identical / class kWh conserved -- the right invariants for #1305.

Ask / clarify before treating flags-on as done:

  1. Hold is blanket. #1304's write-up still says hold, then test AIES item-by-item. Evidence here is strongest on electricity; the 2.4% / $163bn "all items" freeze is argued from the same YoY chaos but not item-graded. Fine as a v1.0 stopgap if you say so explicitly; otherwise call out the follow-up. Holding on 2022 also means every later AIES year inherits that freeze until the exit path is chosen -- not only a 2024 patch.
  2. Trucking proxy is gated by the hold flag in the seed, but ELECTRICITY_PROXY_INDUSTRIES is always used in the #1305 kWh path when that flag is on. Intentional coupling for now is fine -- just don't lose the proxy if AIES is later re-enabled without revisiting trucking.
  3. Who is floored? One buyer / $0.25bn in 2023 and 2024 -- name them if easy.

Minor: usa_config comment on commercial_kwh_follow_intermediate_inputs still says the basis is "chained at the 2023 survey change"; the code and PR body correctly stop at SAS 2022 and skip AIES. Worth aligning the comment so the next reader doesn't think AIES is in the kWh path.

Default CI won't re-prove the all-years balance; taking rebuild 43461231130 as the check is the same bargain as #1302.

Verdict: Approve #1306 and #1305 as the flag-off stack. With both flags on, the AIES step is gone, delivery margins behave, and commercial kWh allocation is an improvement over bill share (conserved class total, steadier N_d, no absurd implied prices). Hold the flags-on / pin / snapshot work for the combined re-pin with #1303, and leave item-by-item AIES re-entry and #899 bill seeding as explicit follow-ups.

WesIngwersen added a commit that referenced this pull request Nov 13, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@WesIngwersen

Copy link
Copy Markdown
Member Author

Thanks @bl-young. Answers in order:

  1. The hold is blanket. Yes. Holding every AIES item on the SAS 2022 index is a v1.0 stopgap. It covers every AIES year from 2023 on, not only 2024, until we choose how each item re-enters. The evidence is strongest for electricity; the other items are held because their year-to-year changes look the same, but none of them has been graded item by item yet. Item-by-item AIES re-entry stays open as a follow-up on Services seed moves on AIES 2024 against AIES 2023, and that step is beyond the survey's range for many industries #1304's write-up.
  2. Trucking proxy. The coupling is intentional for now. If AIES is turned back on later, trucking's electricity has to be looked at again at the same time, in both the seed and the Divide commercial kWh on survey electricity or other inputs, not on bills (#899, #705) #1305 kWh path.
  3. Who is floored. In the commercial class it's software publishers (511200), at $0.25bn of bills in both 2023 and 2024. Its survey electricity path rises while its bill falls, so its kWh at the generation price come to more than the bill (5.7 ¢/kWh in 2023 and 5.3 in 2024). The other floored buyers in those years are industrial or exports (331110, 331410, 311221, and F04000 in 2024), and those are the same with the flag off.

Minor. I fixed the usa_config comment in 3c18839. It now says the basis follows SAS through 2022 and other intermediate inputs after that, and that AIES years aren't read.

The flags stay off until the combined re-pin with #1303. #899's bill seed is still open.

🤖 Generated with Claude Code

Base automatically changed from services_seed_hold_aies to main November 13, 2026 20:09
WesIngwersen and others added 8 commits November 13, 2026 15:09
…the split (#899, #705)

Flag commercial_kwh_follow_intermediate_inputs, off. Within the commercial
class of the generation and delivery split, each industry's kWh follow its
2017 bill moved on its survey electricity bill (chained at the 2023 survey
change) or else on its other intermediate inputs. Class kWh stay on EIA;
bills, other classes and the tables don't change. Buyers pinned on their own
series (data processing on LBNL) keep their kWh exactly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#899, #1304)

AIES 2024 against AIES 2023 moves purchased electricity beyond any SAS-era
year in 20 of 48 survey industries, so the basis reads the SAS through 2022
and moves each buyer's 2022 value on its other inputs after that. A buyer
whose survey electricity is withheld in any SAS year uses the inputs rule in
every year. Base years come from the model's own 2017 and 2022 tables
(commercial_electricity_base.csv), so 2017 is unchanged and 2022 has no step.
Buyers left out of the basis keep their kWh exactly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… --vintage (#1305)

The trucking-to-warehousing electricity mapping in #1306 applies in every
survey year, so 2022's tables differ slightly from fd271dc's. The base file
holds the 2022 inputs the basis moves on from, so it comes from the build
the stack will be pinned to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rs get

IV_score.py now reports V_Nd, the same volatility formula on each
commodity's real N_d. The tracker page gets an N_d volatility column in all
three tables. V only sees a commodity's own emissions, so fixes to what
service sectors buy, electricity above all, show up in N_d and not in V.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bcea with #1305

IV score, volatility, N_d volatility, driving factor and emissions after the
arrow now read the services seed hold (#1306) with commercial kWh divided on
survey electricity or other inputs (#1305). Cable, broadcasting, elementary
and secondary schools and securities brokerage drop to 0.200 or below and
leave the high-volatility table; none had an issue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… from the 5514c3b run

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bl-young added a commit that referenced this pull request Nov 14, 2026
…1164f (#1314)

* feat(config): turn on v1.0 holds for services seed, summary trade, commercial kWh

Enable hold_services_seed_at_last_sas_year, hold_imports_on_summary_groups,
condition_service_exports_on_summary, and commercial_kwh_follow_intermediate_inputs
on the nine v1.0 configs, and require the three Step 5-7 holds in the rebuild flag check.

* feat(config): pin v1.0 to CI rebuild v1.0.0_2b1164f

Retarget the nine v1.0 configs, 2024 balanced SUT and Cornerstone GHG
pins, and upstream FBS inputs to the rebuild from Actions run 43526965633.

* docs(analysis): move IV and B tracker currents to v1.0.0_2b1164f

Equal- and spend-weight IV on the flags-on pin rebuild; landfill drops
below 0.2 after calendar-2017 waste weights.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants