Investigate: standardize the make -n dry-run-parsing pattern (and check for an existing BATS/Makefile-testing tool)

[jnasbyupgrade]

Context

make-test.bats's ordering test ("check-stale-expected depends on installcheck, so it runs after pg_regress, not before", pgxntool #61 / pgxntool-test #31) runs make -n test, then greps the dry-run output for two recipe lines and compares their line numbers to prove one runs before the other. This issue is to investigate whether this "parse make -n output to check wiring/ordering/argument-passing" pattern is duplicated enough elsewhere to be worth standardizing, and whether an existing tool already solves this. This is an investigation, not a mandate -- framed the same way as #44 (bats-assert research): here's what's known, here's what needs more digging, no recommendation.

Where this pattern already appears in this suite

Grepped test/standard/*.bats and test/sequential/*.bats for make -n. Found in at least 6 files besides make-test.bats itself:

• test/standard/dist-clean.bats: run make -n tag/run make -n rmtag (with/without PGXN_REMOTE=upstream), asserting the dry-run output contains the right git push/git fetch remote.
• test/standard/pgxntool-sync.bats: run make -n pgxntool-sync[-master|-local|-local-master], extracting the exact pgxntool-sync.sh invocation line via grep and comparing it to an expected literal command string.
• test/standard/test-test-build.bats: run make -n test (with/without PGXNTOOL_ENABLE_TEST_BUILD=no), grepping for presence/absence of test-build-related recipe text.
• test/standard/test-test-install.bats: run make -n test, grepping for "schedule".
• test/standard/test-verify-results.bats: run make -n verify-results / run make -n results PGXNTOOL_ENABLE_VERIFY_RESULTS=no, checking presence/absence of installcheck/verify-results recipe text.

(Checked test/standard/base-mk-include-guard.bats specifically since it was raised as a candidate -- it does NOT use this pattern, so it's not included above.)

Every instance re-implements the same basic shape by hand: run make -n <target> [VAR=value], then grep/assert_contains/line-number arithmetic against the raw text output. The ordering test in make-test.bats is the most complex instance (needs relative line-number comparison, not just presence/absence), but the underlying technique -- treat make -n output as a parseable command trace -- is the same across all of them.

Web research: does a BATS/Makefile-testing module already exist for this?

Searched for existing tools. Findings:

• No BATS-ecosystem module specifically for parsing make -n/dry-run output was found. The official bats-core-supported library list (bats-assert, bats-support, bats-file, bats-detik) covers generic assertions, filesystem checks, and Kubernetes e2e testing -- nothing Makefile-specific.
• MakeUnit (christianhujer/makeunit) is the closest adjacent tool found: a GNU-Make-based test framework/DSL, but it's for using Make as a test runner for shell scripts (given/when/then-style targets with assertion helpers like assertExistsAndIsEmpty), not for testing the wiring/correctness of another Makefile via dry-run parsing. Different problem.
• No dedicated "parse make --dry-run output and assert on recipe steps" tool (a static/dry analyzer of Makefile recipes) turned up in general web search either.

Conclusion of the research so far: this appears to be a genuinely homegrown pattern in this codebase, without off-the-shelf BATS/Makefile tooling to standardize on -- unlike bats-mock (issue #44's PATH-shadowing precedent) or bats-assert. If this is worth solidifying, it would likely mean writing our own small helper (e.g. a assert_dry_run_order/assert_dry_run_contains pair in helpers.bash or assertions.bash), not adopting something external.

Open questions for further research

1. Is a shared helper (e.g. dry_run_line_for <target> <pattern> returning the matching line number, or a full assert_recipe_order <target> <pattern_before> <pattern_after>) worth extracting given 6+ call sites already do variations of this by hand? What would its interface need to cover (presence/absence checks like most of the 6 sites above, vs. the ordering test's relative-line-number comparison)?
2. Would extracting a shared helper meaningfully reduce duplication, or would the per-site variations (different exclusion filters like test-test-build.bats's test-build string-matching caveat, or pgxntool-sync.bats's exact-line-extraction-and-compare) make a one-size-fits-all helper awkward?
3. If a shared helper is written, does it belong in assertions.bash (as a first-class assertion) or as a more helpers.bash-style utility function tests build assertions on top of?
4. Is it worth revisiting whether make -n dry-run parsing is even the right tool for some of these cases, versus e.g. make print-VAR (the existing print-% introspection target already used by a couple of tests, e.g. print-EXTRA_CLEAN, print-REGRESS_DBNAME) where the thing under test is really "what does this variable resolve to" rather than "in what order do these recipe lines execute."

References

• test/standard/make-test.bats (the ordering test that prompted this)
• test/standard/dist-clean.bats, test/standard/pgxntool-sync.bats, test/standard/test-test-build.bats, test/standard/test-test-install.bats, test/standard/test-verify-results.bats (other instances of the pattern)
• christianhujer/makeunit (adjacent but different-purpose tool found during research)