Skip to content

qbmm: hoist s_mom_inv's internal procedures to module scope - #1688

Open
sbryngelson wants to merge 1 commit into
MFlowCode:masterfrom
sbryngelson:qbmm-hoist
Open

qbmm: hoist s_mom_inv's internal procedures to module scope#1688
sbryngelson wants to merge 1 commit into
MFlowCode:masterfrom
sbryngelson:qbmm-hoist

Conversation

@sbryngelson

Copy link
Copy Markdown
Member

Problem

The five procedures contained in s_mom_invs_coeff_selector, s_chyqmom, s_hyqmom,
f_quad, f_quad2D — are internal procedures called from device code. On CCE 21.0.2 every
QBMM test aborts at runtime:

OpenMP Execution Error: src/simulation/m_qbmm.fpp:984 -
  unsupported access to Fortran host-associated variable in GPU internal procedure

Line 984 is call s_hyqmom(myrho, up, M1) inside s_chyqmom. Both are internal procedures of
s_mom_inv, so s_chyqmom reaches its sibling through the host frame chain, which CCE 21
does not support on device. (cray_inline=True inlines s_hyqmom into s_chyqmom, which is
why the abort surfaces at the call site rather than at the callee.)

Why hoisting is safe

None of the five references anything from its host scope. Every name that appears to be
shared is shadowed by the procedure's own dummy or local:

name where it actually comes from
wght, abscX, abscY, momin dummies of s_chyqmom
pres, rho, c, coeff dummies of s_coeff_selector
q, r, s, wght_in dummies of f_quad
moms, M1, M3, myrho, up, i1 locals
sgm_eps, nmom, nnode parameters (compile-time constants)
weight, R0, nb module variables, use-associated either way

Containment was organizational, not functional. Module scope removes the construct entirely
rather than working around the compiler.

The change is move-only

Apart from the removed contains, all 922 lines are byte-identical after dedent:

$ diff <(git show master:src/simulation/m_qbmm.fpp | grep -v '^\s*$' | sed 's/^[[:space:]]*//' | sort) \
       <(git show HEAD:src/simulation/m_qbmm.fpp   | grep -v '^\s*$' | sed 's/^[[:space:]]*//' | sort)
172d171
< contains

No logic, no expressions, no declarations changed.

Verification

Frontier (MI250X), CCE 21.0.2 / ROCm 7.2.0, MPI, 6/6:

test --gpu mp --gpu acc
0501B3DA 1D Viscosity → Bubbles → QBMM → bubble_model=3 → cfl_adap_dt=T pass pass
6784C02E 1D Viscosity → Bubbles → QBMM pass pass
83291843 2D Viscosity → Bubbles → QBMM → bubble_model=3 pass pass

Confirmed independently in a second session on two further QBMM tests (284E0EF5,
48B9D0C4). A full-suite run with this change shows zero unsupported access to Fortran host-associated variable aborts across 627 tests, on both backends.

./mfc.sh precheck passes (7/7).

This is a correctness fix worth taking regardless of compiler: calling a sibling internal
procedure from device code is not portable, and no other file in src/simulation or
src/common still does it.

Context: #1684 (moving Frontier off cce/19.0.0).

The five procedures contained in s_mom_inv (s_coeff_selector, s_chyqmom,
s_hyqmom, f_quad, f_quad2D) reference nothing from their host scope: every
apparently-shared name is shadowed by the procedure's own dummy or local.
Containment was organizational only.

Calling a sibling internal procedure from device code needs the host frame
chain, which CCE 21 rejects at runtime:

  OpenMP Execution Error: m_qbmm.fpp:984 - unsupported access to Fortran
  host-associated variable in GPU internal procedure

(cray_inline=True inlines s_hyqmom into s_chyqmom, so the abort surfaces at
the call site.) Module scope removes the construct entirely.

Change is move-only: apart from the removed 'contains', all 922 lines are
byte-identical after dedent.

Verified on Frontier, CCE 21.0.2, 6/6: tests 0501B3DA, 6784C02E, 83291843
pass under both --gpu mp and --gpu acc.
Copilot AI review requested due to automatic review settings July 27, 2026 22:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR resolves a CCE 21.0.x OpenMP offload runtime abort in QBMM by eliminating calls between sibling internal procedures from device code, hoisting the five procedures previously contained within s_mom_inv to module scope in m_qbmm.

Changes:

  • Removed the internal contains block inside s_mom_inv and ended s_mom_inv before the helper procedures.
  • Hoisted s_coeff_selector, s_chyqmom, s_hyqmom, f_quad, and f_quad2D to module scope while keeping their bodies unchanged.
  • Preserved module encapsulation (module is private by default; only the existing public symbols remain exported).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants