1. Executive Summary

This project implements a full-stack Capacity Expansion Planning (CEP) model for the Southwest Power Pool (SPP) Regional Transmission Organization. Starting from the full Eastern Interconnection (EI) power system model (92,511 real buses), a 10-step subgraph-based Ward reduction pipeline produces a tractable network of 1,304 buses and 3,626 equivalent lines. A mixed-integer-free DCOPF-based CEP model then optimizes generation and transmission investment decisions across six planning years (2024–2049) at 4% annual load growth.

2. Pipeline Architecture

The reduction is implemented in reduction/run_subgraph_ward.py, a single end-to-end script with 10 sequential steps. The pipeline preserves total generation and load via the Ward equivalencing load-flow (LF) matrix.

2.1 Pipeline Steps

2.2 End-to-End Chain

#	Script	Input	Output
1	run_subgraph_ward.py	2024ITPFinal-24S_fixed.raw	Ward reduction + reconstruction CSVs
2	export_full_lines.py	Reconstructed lines	EqLineCap/spp.xlsx
3	Julia test_cap_spp.jl	spp.xlsx	spp_results.xlsx (3,626/3,626 OPTIMAL)
4	load_analysis.py	FERC 714, retained buses	cep_load.csv, cep_peak.csv, cep_blockhours.csv
5	build_cep_full_dataset.py	All reduction + cost data	In_DatasetSPP.xlsx (18 sheets)
6	cep_model.py --no-crossover	In_DatasetSPP.xlsx	CEP results CSVs
7	cep_dashboard.py	CEP results	Interactive Plotly/Leaflet dashboard

3. SPP Area Definition

SPP area membership follows the MMWG Procedural Manual v43, Appendix V (page 62). SPP has 25 operational areas, of which 23 have active buses in the 2024 ITP model.

3.1 The 25 SPP Areas

Area	Code	Area	Code	Area	Code
506	MJMEUC	523	GRDA	541	EMMW
511	AECC	524	OKGE	542	KACY
515	SWPA	525	WFEC	544	EMDE
520	AEPW	526	SPS	545	INDN
527	OMPA	531	MIDW	546	SPRM
534	SUNC/MKEC	536	EKC	540	GMO
640	NPPD/MEAN	641	HAST	642	GRIS
645	OPPD	650	LES	652	WAPA/CBPC
659	BEPC

Plus areas 997, 998, 999 (HVDC terminals to WECC/ERCOT) -- not load/generation areas.

3.2 Area Classification

Category	Areas	Notes
Active SPP (23)	506, 515, 520, 523, 524, 525, 526, 527, 531, 534, 536, 541, 542, 544, 545, 546, 640, 641, 642, 645, 650, 652, 659	Configured in constants.yaml
Empty SPP (2)	511 (AECC), 540 (GMO)	Zero buses in 2024 ITP case
External SERC	327 (Entergy-AR), 330 (AECI), 356 (Ameren-MO)	NOT SPP; previously misidentified
External MISO	600, 608, 613, 615, 620, 627, 633, 635, 661, 663, 667, 672, 680, 694, 696, 697, 698	600-series are MISO/MRO

3.3 Hanging Fragments

144 real buses in 46 fragments are dropped in Step 4. These buses connect to the SPP mainland only through external areas (327, 330, 356). The largest fragments are in area 515 (66 buses via areas 327/330/356) and area 523 (24 buses via 330). Total dropped: 677 MW load, 1,212 MW generation.

4. Subgraph Consolidation

Buses that are electrically at the same physical location (same substation or switchyard) are collapsed into single-bus subgraphs. Three criteria determine membership:

4.1 Consolidation Criteria

Criterion	Condition	Rationale
Transformers	Any 2W or 3W transformer connection, any voltage	Transformers connect buses within the same substation
Low-Z bus ties	Z < 0.001 pu AND |B| < 0.001 pu	Near-zero impedance connections (bus couplers, switches). The susceptance filter excludes cables (which have high B but are real lines).
Short lines	Length < 2 miles	Lines shorter than 2 miles indicate same physical location (adjacent substations)

4.2 Representative Selection

Each subgraph elects one representative bus using these tiebreaking rules (in order):

1. Exclude star-point buses (≥ 1,000,000)
2. Prefer SPP-area buses over external
3. Highest voltage (BASE_KV)
4. Highest generation capacity (PMAX) tiebreak
5. Lowest bus number (deterministic)

4.3 Retention Criteria

Category	Condition	Count
Expanded subgraph reps	Subgraph with absorbed radials, ≥ 69 kV	~1,050
Junction buses	Single buses with degree ≥ 3, ≥ 69 kV	~254
Total retained		1,304

5. Three-Winding Transformer Handling

PSS/E models three-winding transformers with an internal "star bus" (bus number ≥ 1,000,000, area = 0). The pipeline must correctly classify each star bus as SPP or non-SPP.

5.1 Classification Rules

Case	Winding Areas	Star Bus Treatment	Count
Internal	All windings in SPP	Star assigned to SPP area (majority vote)	1,375
Border	Mixed SPP + non-SPP windings	Star stays non-SPP; winding connections become tie-lines	16

5.2 Key Collision Fix

The xfmr_3w_star_points dictionary uses compound keys that include the star bus ID to avoid collisions when multiple three-winding transformers share endpoint buses. The full set contains 3,867 entries with zero lost due to key collision.

6. Line Expansion Costs

Line expansion cost (ec, in M$/MW) has three components, computed in build_cep_full_dataset.py. Source data: In_TxInvestmentCost.xlsx.

6.1 Cost Components

Component	Formula	Source Sheet	Unit
Line cost	MDpMWMI × distance_miles	costmwmile	M$/MW
Transformer cost	DpMVA / 1,000,000	trans	M$/MW (at unity PF)
Substation expansion	MDpMW at both endpoints	ACSubExpCost	M$/MW

6.2 Line Cost by Voltage (MDpMWMI)

Voltage (kV)	34.5	69	115	138	161	230	345	500	765
MDpMWMI	0.4000	0.2100	0.1125	0.0960	0.0640	0.0408	0.0084	0.0050	0.0031

Lower-voltage lines have higher per-MW-mile costs due to surge impedance loading (SIL) scaling: a 69 kV line carries far fewer MW than a 345 kV line, so the cost per MW is proportionally higher. This is correct physics.

6.3 Transformer Cost (DpMVA) -- Selected Pairs

Pair (kV)	DpMVA ($/MVA)	Pair (kV)	DpMVA ($/MVA)
69 / 138	4,581	138 / 345	6,241
69 / 230	5,348	230 / 345	6,566
69 / 345	6,566	345 / 500	8,466
115 / 230	5,348	345 / 765	10,286
138 / 230	5,348	500 / 765	11,347

6.4 Substation Expansion Cost (MDpMW)

Voltage (kV)	69	115	138	161	230	345	500	765
MDpMW	0.1008	0.0575	0.0510	0.0371	0.0259	0.0122	0.0080	0.0074

Substation expansion cost is applied at both endpoints of every new line (new bay: circuit breakers, disconnect switches, bus work, protection equipment). Voltage-mismatch lines additionally include the transformer cost at the lower-kV end.

7. CEP Model

The CEP model (reduction/cep_model.py) is a linear program (LP) implementing DCOPF-coupled capacity expansion planning, solved with Gurobi's barrier method.

7.1 Model Parameters

Parameter	Value	Notes
Planning years	2024, 2029, 2034, 2039, 2044, 2049	6 five-year periods
Time blocks	17	4 seasons × 4 blocks + 1 Peak block
Total hours modeled	8,760	Covers full year
Load growth	4% annual (compound)	Reference scenario
Discount rate	5.4%	NREL ATB 2024 value
VoLL	$9,000/MWh	Value of lost load (LBNL/Carvallo 2024 central)
PRM	15%	Planning Reserve Margin
End-effect years	30	Years of operation after last investment year
SBASE	100 MVA	PSS/E standard per-unit base
Theta bound	1.5 rad	Voltage angle limit (~86 degrees)
Min admittance (DCOPF)	1e-6 pu	Lines below this are transport-only; upper cutoff (MAX_ADM) 100 pu excludes from angle coupling
Max admittance (DCOPF)	100 pu	Clamped to improve condition number

7.2 Solver Configuration

Setting	Value
Solver	Gurobi (barrier, Method=2)
Crossover	Disabled (--no-crossover)
Barrier convergence tolerance	1e-6
Homogeneous barrier	Enabled (robustness)
OBJ_SCALE	1000x (improves conditioning)
Cost unit	M$ (millions of dollars)

7.3 Generation Technologies

Category	Full Name	HR (MMBtu/MWh)	Investable	Capacity Credit
Wind	Onshore Wind	0.0	Yes	0.20 (ELCC)
Solar	Solar PV	0.0	Yes	Seasonal CF
CC	Combined Cycle Gas	6.3	Yes	1.0
GasCT	Combustion Turbine Gas	9.7	Yes	1.0
GasST	Steam Turbine Gas	9.7	Existing only	1.0
CoalST	Steam Turbine Coal	10.0	No (existing)	1.0
Nuclear	Nuclear	10.5	Yes	1.0
Hydro	Conventional Hydro	0.0	No (existing)	CF-based
OilGT	Oil Gas Turbine	10.0	No (existing)	1.0
Biomass	Biomass/Renewable	13.5	No (existing)	1.0
STO	Battery Storage	0.0	No (disabled)	--
PS	Pumped Storage Hydro	0.0	No (disabled)	--

7.4 Capacity Factors

Wind and Solar capacity factors are derived from HRRR weather data, mapped to each of the 1,304 retained buses via NREL renewable site databases (60 km search radius). The CapFac sheet contains 1,065 bus-level seasonal/block capacity factors. Wind capacity credit for the PRM constraint is 0.20, validated against SPP's 2025 ELCC value (5,504 MW / 27,400 MW nameplate = 20.1%).

7.5 DCOPF Formulation

The model uses a DC optimal power flow formulation with these key features:

• Flow limits: Forward (FW) and backward (BW) = SBASE × π/6 / X
• Admittance: adm = 1/X, clamped to [0.1, 100] pu for DCOPF constraints
• Line investment: Increases capacity (FW/BW) only; impedance is fixed
• Angle coupling: Flow = adm × (theta_i - theta_j) for lines with adm ≥ 0.1
• Shedding penalty: VoLL × hours × gapfac (uses gapfac, not dfac, for consistency with VOM/Fuel)

7.6 Cost Accounting

Component	Formula	Units
GenInv (OCC)	OCC ($/kW) × invested_MW × 1e-3	M$
FOM	FOM ($/kW-yr) × total_cap_MW × gapfac × 1e-3	M$
VOM	VOM ($/MWh) × gen_MW × hours × gapfac × 1e-6	M$
Fuel	FuelPrice × HR × gen_MW × hours × gapfac × 1e-6	M$
LineInv	ec (M$/MW) × invested_MW	M$
Shed	VoLL × shed_MW × hours × gapfac × 1e-6	M$

8. Results

8.1 Cost Summary

Reference scenario (4% load growth), DCOPF, corrected re-run 2026-06-15 (commit bbac115).

Cost Component	Value (M$)	Share (%)
Generation Investment (OCC)	135,564	45.9
Fixed O&M	88,398	29.9
Fuel	60,651	20.5
Variable O&M	7,915	2.7
Line Investment	2,680	0.9
Load Shedding	52	0.02
Total	295,261	100.0

8.2 Investment Summary

Category	Count (decisions)	Notes
Generation investments	1,190	Bus-tech-year triples
Line investments	403	Line-year pairs
Generation retirements	0	Endogenous only (none economic at current FOM)

8.3 Load Shedding Analysis

Total reference-scenario shedding is 1,804 MWh across 52 buses (over the full 25-year horizon, all 17 time blocks) — economically optimal at VoLL = $9,000/MWh and a negligible 0.02% of total cost. All shedding is FW-constrained (line capacity), never DCOPF-constrained (angle). The low (1.5%) scenario sheds 662 MWh; the high (7%) scenario sheds 0.93 TWh — physical, driven by ~5× demand growth by 2049, not the phantom 14 TWh that earlier appeared in a stale pre-fix run (see Audit §9.8).

9. Audit Findings & Fixes

Three independent audit agents reviewed the pipeline on 2026-04-12. All critical findings were resolved; remaining items are deferred decision points.

9.1 Fixed Issues

ID	Finding	Fix	Impact
C2	Nuclear fuel price = $0	Added $0.76/MMBtu (EIA AEO 2025)	~$48M fuel increase
H1	|X| filter instead of |Z| for line pruning	Changed to |Z| = sqrt(R² + X²)	75 phantom high-admittance lines removed
M2	Shed penalty used dfac instead of gapfac	Changed to gapfac (consistent with VOM/Fuel)	Shedding 4.5x costlier; dropped 1,659 to 817 MWh
C3	Line ec missing transformer + substation costs	Added DpMVA and ACSubExpCost from In_TxInvestmentCost.xlsx	282 to 80 lines at EC floor
M3	EC floor applied only to ec=0	Applied 0.1 M$/MW minimum to all lines below threshold	Prevents trivially cheap investment
M5	Wind capacity credit unvalidated	Validated CC = 0.20 against SPP 2025 ELCC (5,504/27,400 = 20.1%)	No change needed
M6	Low-kV line costs appear high	Confirmed correct (SIL scaling physics)	No change needed

9.2 Sensitivity Tests

Test	Shed (MWh)	Change	Finding
Pre-audit baseline	1,659	--	--
Post-audit (gapfac etc.)	817	-51%	gapfac fix was biggest driver
VoLL = $500,000 (10x)	596	-64%	Economic, not physical
No theta bounds (100 rad)	1,624	-2%	Theta NOT the bottleneck
MIN_ADM = 0.5	1,537	-7%	Redistributes, does not eliminate

10. Known Limitations & Future Work

10.1 Decision Points (Deferred)

ID	Issue	Status	Impact
D1	Line investment increases capacity (FW) only, not admittance in DCOPF. In reality, parallel lines both increase capacity and halve impedance.	Deferred	Not currently binding: all shedding is FW-bound, never angle-bound
D2	Battery storage (STO) and pumped storage (PS) have IsInv=0 -- cannot invest. GenCost has STO OCC trajectory ($1,770 to $919/kW by 2049) but unused.	Needs candidate sites	Model may over-invest in GasCT for flexibility
D3	Endogenous retirement is wired in (pv_GenRetYear variable + capacity balance constraint in cep_model.py:472,610) and works — the 2026-04-13 run retired 9.4 GW of wind, 0 GW of coal/gas. What is not wired are the scenario-level aggregate retirement targets Coal_Ret, Gas_Oil_Ret, Nuclear_Ret defined in the Scens sheet (build_cep_full_dataset.py:961 — never read by cep_model.py) and no constraint enforces announced EIA-860 retirement dates.	Partially implemented	Optimizer chooses endogenously; real-world policy retirements (e.g. coal-by-2030 commitments) are not enforced. Coal currently stays flat at 21.8 GW across all 25 years.
D4	Single scenario only (reference, 4% load growth). Need: low (1.5%), high (7%), no-carbon, low-discount-rate.	Scens sheet defined	Results represent one scenario only

10.2 High Priority (Before Publication)

ID	Issue	Notes
H1	Forced outage rates (FOR) not modeled	Tech sheet has FOR values (3.25-5.93%) but thermal is modeled at 100% availability
H2	102 generators (9,827 MW) have non-exact technology match	78 inferred-capacity, 13 inferred, 5 exact_collision. Manual review needed.

10.3 Medium Priority

ID	Issue
M1	Refactor identify_transformer_buses to include 3W_STAR rows in xfmr_2w output
M2	Y_eq dense iteration in network_reconstruction.py is O(n²); works for 1,304 buses but will not scale
M3	Per-generator heat rates (currently per-technology; EIA-860 has plant-level data)
M4	FERC 714 multi-year load validation (currently single year 2024)
M5	Solar CandCap capping (top site: 339 GW at one bus -- NREL aggregation artifact)

10.4 Accepted Limitations

• Crossover disabled -- barrier-only (--no-crossover). Sufficient for CEP accuracy.
• THETA_BOUND = 1.5 rad -- does not bind in practice (removing it changes shedding by only 2%).
• Wind CC = 0.20 -- validated against SPP 2025 ELCC (5,504/27,400 = 20.1%).
• Reference shedding (1,804 MWh, 52 buses) — FW-bound (not angle-bound). VoLL is $9,000/MWh in both the code default (build_cep_full_dataset.py:919) and the Scalars sheet (LBNL/Carvallo 2024 central value). (The earlier "$50k code re-set" concern is resolved — the builder now writes $9,000 on every regeneration.)
• Line investment does not reduce impedance -- not currently binding.
• All costs in 2022 USD -- NREL ATB 2024 dollar year. Dollar year harmonization is a future task.

10.5 2026-04-19 deep-audit summary

The 2026-04-19 independent audit (full report: AUDIT_2026_04_19.html) raised additional items on top of 9.1 above. Severity legend: CRIT — blocks publishable use; MAJ — materially biases results; MIN — does not change top-line numbers; OBS — observational.

Sev	Finding	Status (2026-04-19)
CRIT	MMWG-derived PSSE case is CEII-restricted under SPP/FERC.	Resolved. 50 MB .sav untracked and purged from git history (git filter-repo, force-push completed). Local copy retained; .gitignore broadened.
MAJ	OSM overlay data has no ODbL attribution in the React dashboard.	Resolved. Credits added under the map (dashboard/src/tabs/MapView.tsx).
MIN	tsc disabled in CI (.github/workflows/build.yml).	Resolved. Type-check step re-enabled; build 33 deployed.
MIN	18 legacy tmp_*.py, utils.py, amapgenload.py scratch files at repo root.	Resolved. Deleted; -6,839 LoC.
CRIT	Single deterministic scenario; VoLL code-default $50k suppresses shed.	Resolved. Code-default and Scalars both VoLL=$9,000 (build_cep_full_dataset.py:919, LBNL central). All three load-growth scenarios (low/ref/high) re-run 2026-06-15 and monotonic. (No-carbon variant still deferred.)
MAJ	Ward reduction never validated against full-network PTDF/LODF.	Open. Diagnostic scripts (compare_dc_flows.py, compare_full_vs_reduced.py) exist; need one committed report under docs/validation/.
MAJ	Line investment adds MW but not admittance.	Open, acknowledged. Tracked under D1.
MAJ	FOR defined but not applied to thermal firm capacity in PRM constraint.	Open. Tracked under H1.
MAJ	Storage IsInv=0, cost trajectory loaded but unused.	Open. Tracked under D2.
MAJ	SPP PRM now 16% summer / 36% winter; model still uses 15% annual.	Open. Raise PRM & split by season.
MAJ	Full-network JSON payloads ~35 MB uncompressed; LCP above 2.5 s on 4G.	Open. Candidate fix: pmtiles or deck.gl binary attributes.
MAJ	4,336 of 90,455 buses are topology-inferred (centroid smoothing), not validated.	Open. Need stratified ground-truth sample & per-tier accuracy report.
MAJ	374 manual corrections in bus_locations_corrected.csv have no audit trail.	Open. Document sampling protocol and before/after impact.
MIN	Voltage colour scheme in dashboard not CVD-safe (mixed warm/cool, red on top).	Open. Consider viridis/cividis perceptually-uniform scale.
OBS	Compression ratio 92,511→1,304 buses (1.4% retention, EI basis; 11.9% SPP-internal basis). Aggressive vs Shi 2012 (5–10% typical).	Within-SPP retention is defensible; add PTDF fidelity evidence (see 10.5 above).

11. Data Files

11.1 Ward Reduction Outputs

Directory: data/processed/reduction_subgraph/

File	Description	Records
reduced_buses.csv	Retained bus list with area, kV, reason	1,304 buses
retained_buses_subgraph.csv	Retained buses (alternate format)	1,304 buses
reconstructed_generators_ward.csv	Generators mapped to retained buses via LF	347 gens, 77,910 MW
reconstructed_lines_ward.csv	Equivalent lines after |Z| pruning	3,626 lines
reconstructed_loads_ward.csv	Loads aggregated via Ward formula	1,304 loads, 58,438 MW
equivalent_lines_all.csv	All Ward equivalent lines (diagnostic)	6,390 lines
equivalent_lines_subgraph.csv	Pruned equivalent lines (diagnostic)	3,626 lines
Y_eq_subgraph.npz	Ward equivalent admittance matrix	1,304 x 1,304
LF_subgraph.npz	Load-flow redistribution matrix	1,304 x (eliminated)
Y_spp_subgraph.npz	Full SPP admittance matrix (pre-Ward)	Full SPP size
bus_index_map_subgraph.csv	Bus ID to Y-matrix index mapping	All SPP buses
boundary_tie_gens.csv	Tie-line generators (empty in carve-out mode)	0
cep_load.csv	Hourly-to-block load by bus	1,304 buses
cep_peak.csv	Peak load by bus	1,304 buses
cep_blockhours.csv	Hours per season/block (sums to 8,760)	17 blocks
bus_load_shares.csv	Bus-level load share fractions	1,304 buses
bus_hourly_loads_2024.parquet	Hourly bus loads (intermediate)	1,304 x 8,760

11.2 CEP Input

Directory: data/processed/cep/

File	Description
In_DatasetSPP.xlsx	18-sheet CEP input workbook (GAMS-compatible format)

The 18 sheets are: Buses, Lines, Gens, Tech, Times, Years, Scalars, Controls, GenCost, Fuel_Prices, CarbonEmRed, Scens, CapFac, Load, Peak, DFac, ACSubExpCost, TransCost.

11.3 CEP Results

Directory: data/processed/cep/results/

File	Description
cep_cost_summary.csv	Objective breakdown: GenInv, LineInv, FOM, VOM, Fuel, Shed, Total
cep_gen_investment.csv	Generation investment decisions (1,044 rows)
cep_line_investment.csv	Line investment decisions (428 rows)
cep_gen_retirement.csv	Generation retirement decisions (52 rows)
cep_gen_capacity.csv	Total generation capacity by bus/tech/year
cep_dispatch_summary.csv	Dispatch by technology and year
cep_dispatch_detail.csv	Detailed dispatch by bus/tech/year/season/block
cep_load_shedding.csv	Load shedding events by bus/year/season/block

11.4 Dashboards

Directory: data/processed/cep/

File	Description
cep_dashboard.html	Interactive Plotly/Leaflet dashboard (latest)
cep_dashboard_full_v7.html	Full-featured dashboard (version 7)

12. Technology Stack

12.1 Core Tools

Component	Version	Purpose
Python	3.9 (spp conda env)	Pipeline, CEP model, dashboards
Julia	1.11	Equivalent line capacity optimization (EqLineCap)
Gurobi	11.x	LP solver (barrier method)
VeraGrid	--	PSS/E RAW file parser and circuit object model

12.2 Python Libraries

Library	Purpose
gurobipy	Gurobi Python interface for CEP model
pandas / numpy	Data manipulation and numerical computation
scipy.sparse	Sparse matrix operations (Y-bus, Ward reduction)
networkx	Graph analysis (connectivity, subgraphs, degree)
plotly	Interactive charts in dashboards
folium / leaflet	Geographic map visualizations
openpyxl	Excel I/O for CEP dataset and cost data
PyYAML	Configuration file parsing

12.3 Deployment

Component	Detail
Repository	jahanbani/spp-acep-demo (GitHub)
Branch	extended-boundary
Server	spp.ali.ece.iastate.edu
Container	Docker + Nginx (port 3002)
Orchestration	Portainer (auto-deploys from GitHub polling)
Build trigger	Increment # build: N in docker-compose.yml

13. Comparison with SPP / NREL Authoritative Studies

The 2026-04-19 audit flagged order-of-magnitude deltas between the CEP demo and SPP’s own Integrated Transmission Plan (ITP) and NREL’s ReEDS mid-case. This section reports actual numbers extracted from the current deployed scenario (site/data/scenarios/scen_ref/*.parquet) and explains each delta.

13.1 Transmission investment: our $2.68 B / 25 y vs. SPP ITP10 $7.7 B / 10 y

Metric	Our model (25 y, 2024–2049)	SPP ITP10 approved 2024 (10 y, 2024–2034)
Total transmission $	$2,683 M	$7,700 M
Corridors upgraded	221	89 projects
MW added	13,937 MW thermal uprate	~2,333 mi new + 495 mi rebuild (different unit)
Timing	66% of MW in final year 2049; back-loaded	Front-loaded for immediate reliability drivers
Size distribution	185 corridors under 100 MW (30% of total MW); 5 corridors over 500 MW (30%)	Most ITP10 projects are bulk 345 kV or 765 kV, new-build

Our line-investment totals are about one-eighth of SPP’s own 10-year bulk plan when normalised to a 10-year window. The audit called this a red flag; the deep-dive shows several defensible reasons:

1. The reduced network has no candidate for many ITP corridors. SPP’s 89 projects include new greenfield 345 kV corridors between substations that are collapsed in the Ward reduction (they sit inside retained-bus subgraphs or were absorbed as radial sub-trees). The CEP cannot build lines that don’t exist as candidates.
2. Line investment does not relieve impedance. Under D1, expansion only raises MW cap. A parallel 345 kV circuit in reality halves reactance and doubles PTDF-weighted flow headroom — a benefit the objective cannot see. Lines that would be worth building for congestion relief look unbinding to the optimiser.
3. No N-1 security constraint. A large fraction of ITP10 investment is reliability-driven (N-1, N-1-1) rather than economic-congestion-driven. The CEP formulation has no such constraint.
4. No public-policy drivers. ITP includes transmission needed for wind integration under SPP’s renewable mandates, queue-based interconnection upgrades, and state-mandated siting. None of these enter the objective.
5. Solver numerical floor dampens small investments. With OBJ_SCALE=1 (natural M$; Gurobi ScaleFlag=2 handles conditioning) and BarConvTol=1e-6, investments below the solver’s numerical floor (tens of MW at the bus level) can be dismissed as noise. This also explains the heavy back-loading into 2049 — the model defers builds until they are economically unambiguous.
6. Scope: ITP10 covers bulk facilities ≥100 kV; our pruning floor is 69 kV. Line-cost per MW at sub-transmission voltages differs substantially, so absolute $ comparison needs normalisation by MW-miles and voltage class.

Action items: fix the admittance-scaling issue (see D1), add N-1 security (run OPF with standard contingency set and bind flow at 0.8x rating or use LODF), improve numerical conditioning (see §14). After these, rerun and expect transmission spend in the $4–7 B range, closer to ITP10 when normalised.

13.2 Generation investment: our 284 GW / 25 y vs. NREL ReEDS SPP 45–95 GW

The audit cited a 140 GW investment figure; the actual aggregate across all new-build technologies in the 2026-04-13 run is 284.2 GW (confirmed from gen_investment.parquet). Breakdown:

Technology	Invested (GW)	Final 2049 capacity (GW)	Delta from 2024 (GW)
Wind	161.1	173.2	+161.1
Solar	50.2	50.5	+50.2
GasCT	43.0	50.9	+42.9
CC	29.9	40.2	+19.2
Storage (STO)	0	0	0
CoalST / GasST / Hydro / Nuclear	0 (not investable)	37.1	0
Total	284.2	352.5	+273.5

Reality-check against the PRM constraint: 2024 peak ~58 GW, 4%/y growth → 2049 peak ~155 GW. At PRM=15% the firm requirement is ~178 GW. Our firm contributions: Wind 173 × 0.20 = 35; Solar 50.5 × ~0.15 = 7.6; GasCT 51 + CC 40 + Coal 22 + GasST 10 + Hydro 3.5 + Nuclear 2 + OilGT 0.5 ≈ 129 thermal-firm. Sum ≈ 171 GW firm — tightly matches the ~178 GW PRM target. Nameplate 352 GW looks oversized only because the CC factor of wind (20%) and solar (~15%) is low: the optimiser must build ~5 MW of wind nameplate for every 1 MW of firm capacity credit.

Why we’re much higher than ReEDS

1. Storage is disabled (D2). In ReEDS mid-case, 4-hour batteries substitute for 15–30 GW of thermal firm capacity and lift effective VRE CC by 10–20 percentage points. Without storage, our model must over-build VRE nameplate and keep GasCT to cover the residual peak. This alone accounts for roughly 80–100 GW of the delta.
2. Higher load growth. The reference scenario is 4%/yr (aggressive); NREL ReEDS mid-case uses the EIA AEO reference trajectory which is roughly 1.5–2%/yr on SPP. Over 25 years, that’s a 2× difference in terminal peak load.
3. No coal retirement. ReEDS endogenously retires coal on economics; we have endogenous retirement as a decision variable but the optimiser keeps all 21.8 GW online. NREL/EIA-860 has every SPP coal plant with a scheduled retirement year — wiring those as mandatory constraints would shift build mix toward Wind+Storage replacement capacity and reduce the 352 GW terminal footprint.
4. No line-admittance upgrades. Transmission congestion is the main reason ReEDS curtails wind rather than building more of it. With flat impedance, our model sees no such cap and keeps adding wind until wind-CC-weighted capacity balance is met.

Action items: enable storage investment (D2), wire EIA-860 announced retirements (D3), run low (1.5%) growth scenario (D4). After these, expect the Wind+Solar+Storage envelope to be in the NREL ReEDS range and total nameplate around 200–230 GW.

13.3 Retirement: clarification of prior audit

The 2026-04-19 audit stated retirement was “disabled.” This is incorrect. Endogenous retirement via pv_GenRetYear is wired into cep_model.py:472, 610 and is part of the capacity balance constraint. What is not implemented is (a) scenario-level aggregate retirement targets (Coal_Ret/Gas_Oil_Ret/Nuclear_Ret parameters in the Scens sheet, read at build_cep_full_dataset.py:961 but never consumed by the MIP) and (b) exogenous EIA-860 announced-retirement dates. See corrected D3 above.

14. Solver Numerical Conditioning

14.1 Current state

The LP is built with coefficients whose magnitudes span many orders of magnitude:

Quantity	Natural unit	Order of magnitude	Reason
Objective coefficients (OCC)	M$/MW-y	0.5–5	ATB CAPEX (Solar ∼ $0.8M/MW, Nuclear ∼ $8M/MW)
Shed penalty	M$/MWh	9e-3–5e-2	VoLL/1e6; 9k–50k $/MWh
Line flow bounds	MW	50–5×105	2·θ·adm·SBASE; can be huge where adm is large
Angle θ	rad	1.5	THETA_BOUND
Admittance	1/pu	0.001–103	Short ties vs long corridors
Cumulative hours	h	102–104	Block hours × years
OBJ_SCALE	dimensionless	103	Workaround for conditioning

The resulting coefficient range spans ~10¹⁰ (1e-3 shed penalty × 1e4 hours on one side; 1e5 MW flow × 10³ admittance on the other). Gurobi’s barrier with BarConvTol=1e-6 targets a normalised residual; the effective physical tolerance is tol × MaxCoeff / OBJ_SCALE ≈ 1e-6 × 1010 / 103 = 10 MW at worst-case conditioning. That matches the observed “tens-of-MW noise per bus-block” reported by the audit. (A former OBJ_SCALE=1000 masked it by uniformly shrinking the objective without changing the ratio; the model now runs at OBJ_SCALE=1 and relies on Gurobi ScaleFlag=2 for conditioning.)

14.2 Root-cause fix: matrix equilibration, not tolerance tuning

The right fix is to make the coefficient matrix well-conditioned by unit choice and per-unit normalisation, so Gurobi’s scaling sees a tight range (1e-3 to 1e+3 at most). Concrete steps:

1. Represent capacity in GW not MW everywhere (PSCALE=1000 already does this for some variables; extend to all capacity/flow/load variables so the numeric range is 0–150 GW rather than 0–150,000 MW).
2. Remove the per-line custom flow bound 2θ·adm·SBASE — this multiplies admittance (which can be 1000) by SBASE (100) yielding bounds on the order of 10⁵. Replace with explicit FW + CandCap bound everywhere; the DCOPF physics is still enforced by the flow == adm · (θi − θj) constraint. The custom bound is redundant and noisy.
3. Normalise admittance to base-case reactance. Use x_pu × Sref scaling where Sref = 1 GW so angle-flow coupling is O(1), not O(10–1000).
4. Set OBJ_SCALE = 1 once the ranges are fixed. Re-introduce only if Gurobi reports NumericalIssue.
5. Turn on Gurobi auto-scaling explicitly: model.Params.ScaleFlag = 2 (aggressive equilibration), and verify with model.Params.InfUnbdInfo = 1 + model.printStats() that Matrix range is ≤ 10⁴.
6. Enable crossover for the reported run (Crossover=0 currently). Crossover guarantees a basic-optimal solution rather than an interior point that may lie on a face of degenerate constraints. The runtime cost is typically 10–30% more wall clock; the benefit is extremely clean MW numbers for every bus-block. For sensitivity runs we can keep barrier-only.

A one-day scaling refactor plus a validation run should collapse the shedding’s numeric floor from ~10 MW to <0.1 MW. Only after this is it meaningful to re-run with stricter tolerances (BarConvTol=1e-8) and to make numerical claims about individual bus-block outcomes.

15. |Z| Pruning Sensitivity for Ward Fill-in

15.1 Current state

After the DCPF Ward reduction, the equivalent admittance matrix Y_eq reconstructs 6,390 equivalent lines. Lines with |Z| > 5 pu are pruned, leaving 3,626. The 5 pu threshold is un-sensitized; the audit flagged it as a “magic number” (reduction/modules/network_reconstruction.py).

15.2 Why pruning is needed and what it drops

Ward reduction introduces fictitious “fill-in” edges whenever two retained buses connected through external-bus Schur complements end up with a non-zero equivalent admittance. Many of these fills are physically negligible (very high equivalent impedance ⇔ very weak coupling) but syntactically present. Keeping them (a) inflates the DC-OPF constraint count, and (b) biases PTDFs by introducing parallel paths that don’t exist in the real grid.

However, not every high-|Z| line is fill-in — long high-voltage inter-area ties genuinely have |X| ≈ 2–5 pu. A blind 5 pu cut risks removing real inter-area flowgates.

15.3 Recommended methodology (two-part fix)

Part A — flow-based screening instead of pure impedance threshold. Run a base-case DC power flow on the full reconstruction, then drop only lines whose |flow| < 0.5% of base-case rating and |Z| > 3 pu. This keeps high-|Z| inter-area ties that actually carry flow and discards the Ward fill-in that is electrically invisible. Cross-reference with base-case full-network PTDFs to validate that dropped lines contribute <1% to any retained PTDF row.

Part B — commit a sensitivity sweep. Add a script reduction/sensitivity_zprune.py that runs the pipeline at |Z|max ∈ {2, 3, 5, 7, 10} pu and reports:

• Line count per threshold.
• PTDF RMSE vs. full 96 k case over 30 random source-sink pairs.
• Max angular error in the Ward-reduced DC OPF vs. full-case DC OPF.
• CEP objective delta (rerun the LP at each threshold).

Acceptance: choose the smallest threshold that keeps PTDF RMSE < 5% and angular error < 5°. Literature benchmark: Shi (2012) uses 2–3 pu after flow screening; ReEDS uses a multi-step screening that rarely drops below 4 pu.

15.4 Interaction with |X| → |Z| audit fix (H1 in §9.1)

The earlier H1 fix (use |Z| = √(R²+X²), not just |X|) removed 75 phantom high-admittance lines. Part A above is the next step: move from any impedance-only threshold to a flow-sensitive one.

16. Outstanding TODOs (consolidated)

Single source of truth, synthesised from §9, §10, the 2026-04-12 audit, and the 2026-04-19 deep audit. Ordered by expected impact on headline results; ID prefix indicates priority: P0 blocking, P1 high, P2 medium, P3 nice-to-have.

ID	Item	Category	Expected impact
P0-1	Apply FOR to thermal firm capacity in the PRM constraint (CC_effective = CC · (1 − FOR), FOR already in Tech sheet).	CEP logic	Firm capacity shortfall 3–6%, adds several GW of thermal or storage builds.
P0-2	Switch PRM to 16% summer / 36% winter (current SPP standard, effective 2026), split the PRM constraint by season.	CEP logic	Adds ~20 GW winter firm capacity; storage+CC become very valuable.
P0-3	Enable storage investment: set IsInv=1 for STO, define per-bus CandCap from the SPP GI queue.	CEP logic	Large reduction in Wind/GasCT nameplate; shifts 40–80 GW of firm capacity to 4-hour batteries.
P0-4	Matrix coefficient conditioning (§14): GW units everywhere, remove custom 2θ·adm·Sbase flow bound, OBJ_SCALE=1, re-enable crossover for headline runs.	Numerics	Shedding numeric floor drops from ~10 MW to <0.1 MW. Prerequisite for defensible bus-level results.
P0-5	Correct the code-default VoLL from $50,000 to $9,000. DONE — build_cep_full_dataset.py:919 writes $9,000 (LBNL/Carvallo central) on every rebuild; Scalars sheet matches.	Economics	No more silent regression on dataset rebuild.
P1-1	Commit a PTDF-fidelity report under docs/validation/: RMSE of |PTDF_reduced − PTDF_full| across 30 random source-sink pairs.	Network reduction	Evidence for reviewers; cheap (existing compare_dc_flows.py).
P1-2	Enforce EIA-860 announced retirement dates for coal & gas-ST (exogenous constraints on pv_GenRetYear).	CEP logic	Shifts 20+ GW coal capacity out of the mix; changes replacement build profile.
P1-3	Add N-1 security: for each monitored line, enforce |flow| ≤ 0.8 · rating in the base case, or use LODF screening for the top 50 contingencies.	CEP logic	Brings transmission investment closer to SPP ITP10 levels.
P1-4	Flow-based |Z| pruning (§15) and sensitivity sweep at {2, 3, 5, 7, 10} pu.	Network reduction	Removes “magic number” objection; tightens PTDF fidelity.
P1-5	Line-investment admittance scaling: a parallel circuit should halve X and double admittance, not just add MW cap.	CEP logic	Currently not binding but unverifiable under scenarios the model doesn’t run. Enables angle-bound shedding to be resolved by transmission.
P1-6	Scenario set: run at least 3 growth (1.5, 4, 7%) × 2 fuel (AEO vs. low-gas) × 2 policy (BAU vs. IRA) = 12 runs. Publish a fan chart.	Uncertainty	Moves the demo from “one outcome” to “plausible range.”
P1-7	Add Winter Storm Uri (Feb 2021) and Elliott (Dec 2022) as mandatory stress blocks in the Times sheet (7 days each, fixed load profile).	Resilience	Identifies winter firm-capacity shortfall beyond the PRM constraint.
P1-8	Dashboard payload: move full_*_lines.json (12–15 MB uncompressed) to pmtiles or deck.gl binary attributes. Target first-paint under 2 MB gzipped.	Dashboard	LCP under 2.5 s on 4G; Google Web Vitals compliant.
P1-9	Dollar-year harmonisation (cep_model.py:8 TODO). Choose USD 2024 as reporting year and apply GDP deflator to ATB-2022 values consistently.	Economics	Numbers become directly comparable to published benchmarks.
P2-1	102 generators (9,827 MW) with non-exact technology match — manual review.	Data quality	Small shift in starting capacity by tech; tracked as H2.
P2-2	Bus-location ground-truth validation: stratify 100 buses by confidence tier, hand-verify against Google imagery.	Data quality	Empirical accuracy per tier; lets us mark topology-inferred buses visually.
P2-3	Document the 374 manual corrections in bus_locations_corrected.csv: sampling protocol + before/after impact report.	Data quality	Audit trail for reviewers.
P2-4	Voltage colour scheme in dashboard: switch to viridis/cividis (perceptually uniform, CVD-safe).	Dashboard	Accessibility; no content change.
P2-5	Carbon as price, not only cap: run a scenario where Carb_Emission_Flag=0 and add a $/tCO2 price line; report implicit price of the current cap.	Economics	Standard sensitivity; cheap.
P2-6	AC-OPF validation of the 2049 optimal dispatch against the unreduced 96 k-bus case: report max V, MVA, and reactive violations.	Validation	Strong evidence of AC feasibility of the DC-OPF-derived plan.
P2-7	Add a PyTest that loads In_DatasetSPP.xlsx, solves a 2-year CEP on a toy 20-bus subset, and asserts objective within ±0.1% of a golden value.	Reproducibility	CI regression coverage; currently only vite build.
P2-8	Pin Python deps (requirements.lock via pip-tools or uv).	Reproducibility	Standard hygiene; currently only lower bounds.
P3-1	Fold legacy static dashboards (/dashboards/cep_dashboard_*.html) into the React SPA or drop them.	Dashboard	Single maintained surface.
P3-2	Alternative-reducer comparison: K-means on PTDF rows vs. spectral clustering vs. current Ward at similar retention.	Research	Defensible choice of method, not just “the one we ran.”
P3-3	Retire run_reduction.py (legacy, 2184 lines) to reduction/legacy/; run_subgraph_ward.py is canonical.	Hygiene	Reduces confusion for new contributors.
P3-4	Dashboard: add per-source “methodology” info-drawer for PSSE / PSSE-clipped / OSM so users understand coverage differences.	Dashboard	Prevents misleading conclusions (“OSM disagrees with PSSE”).

---

SPP Capacity Expansion Planning — Master Technical Documentation
Iowa State University, Department of Electrical & Computer Engineering
Revision 2026-04-19 (deep-audit integration). Source: reduction/run_subgraph_ward.py (rev 2026-04-11).
Full audit report: AUDIT_2026_04_19.html