Visitar URL original
Alpha 4 Release by cbuahin · Pull Request #135 · HydroCouple/openswmm.engine · GitHub
Skip to content

Alpha 4 Release - #135

Open
cbuahin wants to merge 909 commits into
developfrom
swmm6_rel
Open

cbuahin wants to merge 909 commits into
developfrom
swmm6_rel

Conversation

@cbuahin

@cbuahin cbuahin commented Aug 17, 2026

Copy link
Copy Markdown
Member
  • Addressing legacy parity bugs
  • Change of license
  • Water quality (Eulerian and Langrangian), Heat transport, Multi-species Reactions

@dickinsonre

Copy link
Copy Markdown

Water quality (Eulerian and Langrangian), Heat transport, Multi-species Reactions Thanks, this is very welcome.

Allow only known AArch64 libgcc_s SME ABI helper routines and exact CNTD x0 vector-granule queries in the inspected unwind regions. Keep engine/plugin code, other runtime functions and AVX-512 checks strict. Report allowed support instructions separately and include the runtime policy in the content-cache key so identical bytes under an engine filename cannot bypass the gate.

Reproduced the same 45 flagged instructions using official Alpine GCC 14.2.0 runtime binaries and checked the GCC helper/unwind sources. The runtime passes after the fix; an identical engine/plugin copy still fails. All 9 regression tests and Ruff pass. No full Linux wheel rebuild was performed locally.
…and process models

Commit the subsets of the 2026-10-22..28 SoA/vectorization program that were
retained after their targeted performance screens and exact-output gates.
Numerical methods, GPU compatibility and node order are preserved: each change
keeps the serial arithmetic order, publication points and checkpoint/hotstart
wire formats. Rejected trials are not retained.

Instrumentation and coverage
- core/ProcessPerf.hpp: per-process timers and counters gated by OPENSWMM_PERF,
  registered per engine; disabled profiling accumulates nothing. Optional
  macOS allocation probe (process_malloc_probe.cpp).
- Four process-coverage decks (SIGMA column, GW species/heat, UEB mesh snow,
  Richards LID) added to tests/parity/MANIFEST, with the Phase-0 README,
  soa_review_baseline.py and prepare_soa_review_cases.py; the trackI G1 gate
  now uses the tracked deck census.

Avoided work, copies and allocations
- 2D plain-cell sources and timestep-reduction fast paths; runoff
  ordinary-cell extraction; hoisted decay table and tier-0 boundary guard.
- Common-source stage: consumed-credit move, shared context and private
  SourceSnow snapshots; selected opening pollutant queues in the source-water
  driver.
- MeshSnow: the common-source stage advances its private trial in place;
  held-window UEB predictions are reused.
- Reused workspaces: thermal contacts, porewater and SIGMA columns, local
  bed-species scratch, atmospheric midpoint, MSX capacity and four-mass
  buffers, OutfallExchange SPREAD receivers, groundwater LID buffers.
- Result-history append rolls back on late refusal instead of copying.

Layout and parallelism
- CPU-private 2D face packets (water sides, FULL_SWE momentum); checkpoints
  still stream the original logical vectors. Restrict-qualified face/cell
  views.
- Edge-major groundwater lateral packets and private layer storage; hotstart
  rows and public snapshot/HDF axes keep the legacy order.
- Ordered scheduler lists and parallel face publication; OpenMP batches for
  groundwater, Richards columns, ordinary runoff and 2D infiltration with
  serial reduction order.
- Typed callback for the groundwater ODE integration (OdeSolverDetail.hpp);
  the type-erased ode::integrate interface is unchanged for other callers.
- All-thin degree-day snow batch.

Measured gains apply to the targeted workloads each change was screened on
(e.g. 2-48% on its own fixtures); no whole-model or I3 speedup is claimed,
and the program's phase-wide gates remain open. New and extended unit tests
cover the reuse, refusal/rollback and thread-count behaviour of most retained
paths.
Use conserved cell momentum for FULL_SWE initialization, output and
velocity envelopes. Expose the same physical SI velocity through a bulk
C API so live clients do not reconstruct it from Riemann mass flux.

Preserve hydraulic fluxes and wet/dry controls. Cover native and dry-cell
publication, wetting fronts, fast flow, API lifecycle and output metadata.

Refs #197
Correct the aggregate-volume/per-barrel-rate mismatch in 5.3.0 and 6.0.0
(dynamic wave and the 6.0 finite-volume loss precomputation).

Limit each dynamic-wave conduit's natural losses by the water available
at both end nodes, aggregating shared-node demands before scaling. Exclude
the numerical wetting floor from removable water and retain accepted rates
for hydraulics, quality, and continuity accounting. Preserve aquifer gains
and the existing authoritative host-seepage forcing contract.

Add 15 regression cases covering the reported hydrograph, low constant
inflow, timestep/barrel/outfall/thread variations, evaporation, and FV caps.
The reported continuity error improves from -304.589% to +1.021% at 1 s
and +0.724% at 0.1 s; document residual dry/wet discretization error (up to
3% in the added edge-case gates). Six related CTest suites also pass.

Refs #2
Refs USEPA#216
Related: USEPA#234
Replace mean-depth discharge reconstruction with an area-weighted fit to
wetted-edge normal velocities for local-inertial and diffusive-wave output.
Respect stage-boundary depths and interior porosity; preserve native
FULL_SWE momentum velocity and all hydraulic updates and controls.

Represent unresolved wet-cell vectors as paired NaNs and omit unavailable
samples from velocity envelopes. Preserve those diagnostics across saved
output and checkpoints, with explicit metadata and API documentation.

Validation: 218 engine checks passed, 864 manufactured fields and 48 paired
controlled cases. Bellinge comparisons preserve sampled hydraulics and
arrival times. Synthetic reconstruction scales linearly to one million cells.

Refs #197
…sure

Add EDDY_VISCOSITY_BACKGROUND, a constant horizontal eddy viscosity added
to the selected closure, and TURBULENCE_MODEL MIXING_LENGTH with
MIXING_LENGTH_COEFFICIENT (default 0.5, the depth-averaged value of
Stansby 2003/2006). Mixing length reuses the conservative stress operator,
diffusion timestep bound and energy checks of Smagorinsky, with
nu = nu0 + l^2 |omega| and l = min(c h, 0.41 d), d the distance to the
nearest mesh boundary edge. The length follows the flow depth rather than
the cell, so the viscosity does not collapse under refinement, and the
vorticity form leaves irrotational contraction flow unmixed.

The options round-trip through .inp and GeoPackage, enter the checkpoint
identity, and are recorded in the 2D output attributes and eddy-viscosity
field. Both are inactive under TURBULENCE_MODEL NONE.

Validation: 2D momentum-physics suite including exact l^2|omega| on affine
fields (depth- and wall-limited branches), zero viscosity in irrotational
strain, refinement convergence where Smagorinsky collapses, analytical
viscous decay for nu0 and conservation in the full solver; 2D surface,
reconstruction, output options, CPU correctness, momentum modes, timestep
reduction, plain sources, LTS equivalence, quad mesh, edge conveyance,
.inp round-trip and hotstart suites pass. UK EA Test 6A: G2 jump arrival
18.3 / 18.6 / 18.7 s on 0.1 / 0.05 / 0.035 m meshes, where Smagorinsky
Cs = 0.42 loses the jump on refinement; inactive runs stay bit-identical.
Honor per-junction StorageArea in 5.3.0 and 6.0.0, balance chamber and conduit inventory, preserve it in model formats and Python APIs, and reject unsupported solver/export combinations. Update engine, format and Python manuals with regression coverage.

Refs #10
Refs USEPA#200
Expose swmm_2d_get_velocities_bulk through Surface2D.get_velocities, returning independent float64 X/Y cell arrays in physical SI m/s. Use the native reconstruction and preserve paired NaNs, with the same operation lock and GIL release as other bulk getters. Add the public typing signature and regenerate the catalog.

Reproduce the missing declaration from all four CI logs, then run the exact contract workflow under Python 3.10 through 3.13. Each passes the strict API audit, catalog freshness and 72 tests with 5 runtime-only skips. Add native runtime regressions for physical velocity/sign and SI units, dry cells, NaNs, lifecycle, independent memory and non-advancing reads. All 94 selected compiled contract and runtime tests pass on the isolated native/Python build, with no new Ruff diagnostics.
Classic MSVC /openmp rejects the OpenMP 4.0 SIMD directive in meltThinPack with C7660. Guard the optional hint by the advertised OpenMP version so Windows retains the same loop and arithmetic without requiring experimental compiler flags. OpenMP 4.0 and newer retain SIMD.

Validate preprocessing of the actual translation unit with OpenMP disabled and versions 2.0, 4.0 and 5.1. Rebuild the native library and run all 75 snow tests with both the normal modern OpenMP path and the 2.0 fallback profile. Native MSVC compilation remains for Windows CI.
… classic FV hotstarts

Deliver the explicit-FV conservation, reporting and restart work qualified
by the 2026-10-26..27 FV programs. Legacy dynamic-wave numerics, the
retained implicit backend and the explicit wave rule are unchanged.

Explicit FV solver
- Fused closure area/width/integral and friction-radius lookup for the
  post-update cell closure (bit-identical to the separate evaluations);
  centred physical faces read their stored beds directly.
- Per-window accepted-water receipts: face forward/reverse, lateral,
  spill, structure, cell loss and node deficit volumes.
- Gross external withdrawals: the unavailable share is refused and
  refunded to its inflow categories instead of creating negative water.
- Node balance: stage boundaries act as external reservoirs, zero-storage
  splices pass the same accepted volume in both directions, and a DUMMY
  outlet drains until its capacity binds.
- Positive-volume scalar inventory survives hydraulic-dry thresholds in
  Euler, RK2 averaging and FCT prediction; RK2 averages inventory, not
  concentration. LTS scratch is safe to reuse across solver instances.

Routing and mass balance
- FV totals integrate accepted-window volumes once over their own
  interval; virtual junctions report the spliced face's receipts, less the
  half of a diverted lateral the face already carries, so a fed virtual
  junction's inflow and outflow balance (lateral + through-flow).
- A reversing stage outfall books both its discharge and its accepted
  incoming water.
- Native FV conduit seepage/evaporation rates are scaled to the volume
  actually removed.
- OPENSWMM_FV_RECEIPT_TRACE=<csv> appends per-cell/node receipt balances
  for diagnostics; unset, nothing is recorded.

Conservative quality transport
- FvQualityTransport and ConservativeMixing route pollutants, MSX species,
  water age and temperature over the accepted FV water graph when the
  legacy quality solver runs with native CPU FV routing.

Restart
- Hotstart V18 extension stores named, unit-checked disconnected FV
  pollutant/species/age/temperature inventories, validates the whole batch
  before mutation and seeds the opening dry ledgers. A classic FV restore
  projects saved conduit volumes onto the existing mesh and resets
  integration history once. This is an aggregate restart: no hydraulic
  cell profile or coupled thermal history is restored.

Routing.hpp and ConservativeMixing.hpp include their FV/quality headers
file-relatively, so targets without the engine source include root
(bench_fv_closure) build.

Tests: six new FV suites (conservative mixing, transport allocations,
accepted water, conservative quality, scalar dry inventory, dry restart)
and extended engine-integration, network, LTS, section-geometry and TPA
signed-head cases. Treatment/LID/booster conservative ports, LTS timing
and larger-step/second-order field accuracy remain open; no hydraulic
speedup is claimed.
The committed engine and legacy-solver export headers carry a Doxygen
file block, but generate_export_header did not emit it, so every clean
build rewrote them without it. Supply the block through
CUSTOM_CONTENT_FROM_VARIABLE for the engine, legacy solver and legacy
output export headers.
Balance physical chamber and adjacent conduit inventories at dry starts, surcharge and terminal junctions. Use bounded local volume inversion for pipe-dominated chambers and refuse unresolved water budgets with actionable timestep guidance. Document supported routing settings and extend cold-start and small-chamber regression coverage.

Validation: all 20 junction storage regressions pass in the refactored and 5.3.0 engines; zero-area outputs remain bit-exact in both engines.

Refs #10
Refs USEPA#200
Extend the section underline to match the 29-character title. Sphinx treats the short underline as a warning, failing the documentation workflow under -W.

Reproduce the reported failure with Sphinx 9.1.1 against a clean HEAD snapshot, then rebuild all Python HTML documentation with -W --keep-going after the correction. The complete build succeeds with zero warnings. Check the remaining reStructuredText heading underlines for the same defect. Local validation dependencies and generated documentation remain uncommitted.
set_node_type now clears a non-junction's StorageArea, but contexts built
column by column (the GeoPackage test fixture) never size storage_area, so
GeoPackageTest.OptionsRoundTrip wrote through an empty vector and crashed.
Skip the write when the array is unsized; unsized already means no
chamber, and the GeoPackage writer reads it through safe_dbl.
The bit-exact legacy parity gate (XSectBitExact) needs -ffp-contract=off
in C as well as C++, because legacy xsect.c is C. Only CMakePresets.json
set it, so a configure without a preset compiled the legacy kernels with
clang's default contraction and they drifted by 1 ulp at small depths.

Append -ffp-contract=off to CMAKE_C_FLAGS and CMAKE_CXX_FLAGS for Clang
and GCC when it is missing. Preset builds already carry it and compile
with unchanged flags.
Retain all GeoPackage polygon rings and reconstruct exterior/hole nesting when saving. Use ring-aware containment for 2D subcatchment ownership and document the closed-ring coordinate convention.

Validation: multipart geometry decoding and save/reopen regressions, plus the existing single-polygon round-trip test (3 tests passed).
NODE_ENROLMENT AUTO connected every node inside the mesh to the aquifer,
outfalls included. At an outfall that discharges onto the mesh, the
outfall exchange books the node's net flow as surface water, so the
aquifer's draw on the outfall's inflow (applied a step later) was taken
as a withdrawal from a dry cell and stopped the run with "1D outfall
exceeded its reserved surface water budget". The Bellinge 2D model with
any [2D_AQUIFER] stopped at step 4.

Automatic connection now skips outfalls; an explicit [2D_AQUIFER_NODE]
row still connects one. Adds a reproducer with a falling inflow trickle
into a mesh-coupled outfall.
With [2D_MESH_FILE], the writer sent [GW_*], [2D_COVERAGES],
[2D_LOADINGS], [2D_CURB_LENGTH] and [2D_AQUIFER*] to the .2dm sidecar,
because they were emitted at the end of the mesh-section routine. The
reader takes those sections from the .inp only, so the next open
dropped them and a second save lost them (the Bellinge groundwater
deck lost its aquifer, well and land-use coverage). A mesh that failed
to load lost them on the first save.

Emit them from a separate routine that always writes the main .inp.
Saves through the GUI were not affected: it inlines the mesh first.
XY sources are resolved when the model opens, before MeshBuilder fills
the cell centroids, so every centroid was still zero and every XY
source landed in the first cell.

Add MeshData::cell_centroid_xy, which computes the area centroid from
the vertices, and use it both for the XY lookup and in MeshBuilder so
the two cannot disagree.
swmm_pollutant_rename updated LID rules and inflows only, and
swmm_landuse_rename only the name registry. [2D_LOADINGS],
[2D_INITIAL_QUALITY], [2D_INITIAL_BED_QUALITY], [2D_BOUNDARY_QUALITY],
the [GW_*] species rows, [GW_SOURCES] terms, REPORT_2D_SPECIES and the
[2D_COVERAGES] land uses kept the old names, so a renamed model saved
a file that refused to open.

Add SurfaceRouter2D::renameSpecies and renameLandUse and call them from
both rename functions.
…nction

On the mesh, a land use with no buildup function washes off without
limit. When a step washed off more than the cell held, the whole step
was booked as buildup, so the store already counted as initial buildup
was counted twice and the 2D Surface Washoff Summary did not balance
(38 kg on the Bellinge test model).

Book only the excess over the store. The subcatchment kernel keeps the
legacy landuse.c rule for parity with SWMM 5.
…he API

swmm_gw_source_get reports an XY-authored [GW_SOURCES] row as CELL with
its resolved cell, and swmm_gw_source_set clears the XY form, so an
editor could neither show nor keep it: saving through the GUI turned XY
rows into CELL rows.

Add swmm_gw_source_xy_get (is the row XY, and its coordinates) and
swmm_gw_source_xy_set (locate a row by XY; it is placed in the nearest
cell exactly as an authored XY row is, and written back as XY). The
nearest-centroid rule moves into nearestCell, shared by the open-time
resolver and the setter.
…acy does

The Link Pollutant Load Summary always converted to pounds and labelled
the column "lbs", so SI runs printed pound values; a count pollutant was
printed as a raw count under "#" without the liters-per-cubic-foot
factor. The Subcatchment Washoff Summary printed kg values under "lbs"
and raw counts under "#".

Follow legacy statsrpt.c: totalLoad x LperFT3 x the pollutant's mass
factor (lbs for US, kg for SI, 1 for counts), LOG10 for a count
pollutant under "LogN", and %14.3e for link loads of 10000 or more.
US values move by the legacy factor (2.203e-6 lb/mg instead of
1/453592).
A junction's spill to the 2D surface is netted into its lateral flow, and
the quality ledger booked a negative lateral flow's mass only when the NET
flow was negative. A junction that spilled while its runoff, dry weather
flow or external inflow was still larger lost the spilled mass from the
ledger: 21.7 % quality continuity error on the coupled-pond fixture, whose
flow ledger closes to -0.75 %.

The spill never enters the quality mix's inflow volume, so it now leaves the
ledger in full at the mixed concentration; the rest of the lateral flow
keeps legacy's net rule. The floored-net denominator correction takes the
coupling back out of lat_flow, as its comment already intended, so a
withdrawal beside a spill shrinks the mix volume once and the spill is
booked once.

Both terms are exact no-ops when no 2D coupling flows, so legacy-comparable
decks are unchanged. On Bellinge without the aquifer the 1D SurfPoll error
moves 4.880 % -> 4.748 %; the rest is legacy node mixing at wetting
junctions, not this booking.

Tests: CouplingPonds.SpillUnderALargerInflowLeavesTheQualityLedger and
CouplingPonds.SpillBesideAWithdrawalIsBookedOnce (quality continuity tracks
flow continuity; each fails without its half of the change).
Add absolute dyadic scheduling, dated transfer ownership, native restart validation and clock statistics, including Python accessors. Preserve SHARED as the default. Correct independent thin-column storage and round-trip the surface-step binding option safely.

Validation: native groundwater and clock gates pass; Python groundwater tests: 237 passed, 1 skipped. Full-event performance remains unqualified.
Add storm, node-fill and stiff-column controls plus producer-refinement, accuracy and paired performance runners. Support the approved matched-physics node reference while retaining legacy compatibility results. Keep full-event performance separate from startup slices.
Expose swmm_gw_source_xy_get/set through GroundwaterTransport.source_xy and set_source_xy. Return authored project-map coordinates, or None for cell/tag sources, and retain native nearest-cell resolution, editability and error handling. Add public typing signatures and regenerate the catalog.

Reproduce the two missing native declarations in the supplied Python 3.10-3.13 contract logs. The strict audit, catalog freshness and all four source contract suites now pass (72 tests and 5 runtime-only skips per version). A clean compiled native/Python build passes 93 selected contract and runtime tests, including CMS/CFS XY round-trip, preserved flow/scale/species, invalid inputs/indices and lifecycle/stale-view checks. The strict typing consumer and regression-test Ruff check pass. The supplied minimal-feature and downstream jobs already passed. Preserve unrelated staged and working changes; exclude plans and local build artifacts.
The new optional groundwater clock state contains map/set owners whose move construction allocates sentinel nodes on MSVC. That makes optional move assignment potentially throwing and breaks the existing transactional publication static assertions in LidNodeAquifer and other native callers.

Keep the clock in a dedicated unique-pointer owner with deep-copy value semantics. Moves transfer ownership without relocating the contained clock, while private snapshots retain independent schedules, receipts and histograms. Preserve all publication safety assertions.

Rebuild the isolated native library. The common-source suite passes 123 tests with 3 opt-in performance skips; all 106 LID-node tests and the independent groundwater clock gate pass. Add copy/move/empty-state and no-relocation regressions, including the solver no-throw move-assignment assertion. Native Windows compilation remains for CI. Preserve unrelated staged work and exclude plans and local test artifacts.
Correct reaction coupling, transport conservation and temperature handling;
preserve authored groundwater source coordinates and validate snow fractions.

Refuse incomplete restart saves, checkpoint LID layer scalars, publish
hotstart/HDF5 writes atomically, and verify dynamic I/O plugin ABI tags.

Add focused regression coverage and optional comparisons against locally
supplied stock EPA 5.2.4 outputs. Keep generated references, audit receipts,
plans and build products outside version control.

Validation: 24 targeted C++ suites, 215 checkpoint cases and 111 Python tests
passed in the working checkout. EPA comparisons also pass with local
references; their empty-path skip behavior is verified.

This branch was successfully deployed

1 active deployment
github-pages — 7bcb4f48 Deployed Nov 2, 2026 by cbuahin via Deploy to GitHub Pages #514
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants