Skip to content

Let SBD choose the SQD carryover (qiskit-addon-sqd#369) - #49

Merged
Sophia Wen (hfwen0502) merged 15 commits into
mainfrom
sbd-carryover-369
Oct 9, 2026
Merged

Sophia Wen (hfwen0502) merged 15 commits into
mainfrom
sbd-carryover-369

Conversation

@hfwen0502

@hfwen0502 Sophia Wen (hfwen0502) commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Summary

qiskit-addon-sqd#369 (merged, 0.14.0) lets an sci_solver report its own carryover through SCIResult.carryover, and lets SCIResult.sci_state be None. This PR uses both, so SBD can choose which determinants seed the next SQD iteration. With carryover_type=1, the wavefunction never leaves SBD.

  • Report SBD's carryover. When carryover_type != 0 in sbd_config, solve_sci/solve_sci_batch return SBD's selection in SCIResult.carryover. The gate checks the installed addon's fields, so on older addons nothing changes.
  • Skip the wavefunction dump for type 1. Type 1 ranks half-determinants by sum_beta |c|^2, exactly the marginal weight the loop orders carryover by. Nothing else needs the eigenvector, so the |adet| x |bdet| dump-and-reload is skipped and sci_state is None.
  • Order the carryover the way the merged loop requires. _select_carryover returns a solver's carryover untouched, and a later max_dim truncation keeps the leading entries, so the order must be descending marginal weight (cf. #382). Types 2 and 3 end in sort_bitarray, which re-sorts into canonical order, so for those types the dump is kept and the carryover is re-ranked from the amplitudes: strings that were in the subspace first, by weight, then the generated ones.
  • run_sqd_sbd.py: adds --sbd_carryover_type/_ratio/_threshold, and passes the SQD threshold inside a StandardPolicy, since the merged loop rejects policy= together with carryover_threshold= (it falls back to the direct kwarg on addons below 0.14.0). It warns when a setting cannot take effect: --sqd_carryover_threshold under SBD carryover, and SBD's ratio/threshold exclusivity per type.
  • Docs: a new "SBD-selected carryover" section in examples/tpb/README.md, and a Python recipe for running Trim SQD (TrimPolicy) with SBD as the solver. Trim is not exposed as driver flags.

Default behaviour (carryover_type=0) is unchanged. GDB is untouched: neither GDB driver uses the SQD loop.

Testing

Against qiskit-addon-sqd main at b4ba0cc:

  • Local CPU: 8 serial / 15 MPI tests pass. Every carryover test has a standalone and an MPI variant: type 1 returns sci_state=None and writes no dump; types 2/3 return descending-weight carryover; and the diagonalize_fermionic_hamiltonian loop test is parametrized over carryover_type 0 and 1, with symmetrize_spin on, asserting each iteration gets SBD's carryover with identical alpha and beta lists.
  • Under symmetrize_spin, the addon ignores carryover_strings_b. For a symmetric subspace SBD's two carryover lists are bit-identical (types 1, 2, 3), so nothing is lost.
  • 8x H100, Thrust backend, grid 4x2:
case energy notes
N2 1em4, plain run_sbd_diag.py (reference) -109.0483526946 1387 x 1387 subspace
N2 SQD, type 0 -109.0483526946 matches the reference, dump written
N2 SQD, type 1 -109.0483526946 matches the reference, no dump
N2 SQD, type 3 -109.0484954322 below the reference (singles extension), dump written
Boston 45-orbital SQD, type 3, max_dim 15000, threshold 1e-4 -299.4875648624 converged in 8 iterations; 1.5 mHa below the earlier type-3 best of -299.4860424776

None of the runs had an E=0 iteration. The Boston run took 1949 s, against 659 s for the earlier result from a different driver. Most of that is probably configuration recovery over 1M bitstrings each iteration, which this PR does not change. That is an inference; the run was not profiled.

Related: #31 point 2 (cost of _sbd_dets_to_ci_strings). Type 1 skips both full-subspace conversions; the default path is unchanged.

🤖 Generated with Claude Code

Sophia Wen (hfwen0502) and others added 6 commits October 1, 2026 15:33
qiskit-addon-sqd#369 adds SCIResult.carryover, which lets a solver hand the loop
the determinants it picked itself rather than having the loop threshold our
amplitudes. SBD already makes that choice in C++ whenever carryover_type is
non-zero, so pass it along instead of letting the loop re-derive it.

Gated on the installed addon actually having the field: on an older release the
carryover is simply not reported and the amplitude-thresholding path is
unchanged. carryover_type still defaults to 0, so none of this engages unless
asked for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_solve_sci_core asked SBD to dump the full |adet| x |bdet| amplitude matrix on
every call and read it straight back, purely so the loop could threshold it. When
carryover_type is non-zero that selection already happened in C++, so the dump is
a multi-GB write-then-read of the largest object in the run with no consumer.

Stop requesting it in that case and return sci_state=None, which
qiskit-addon-sqd#369 explicitly allows for a solver that reports its own
carryover. The default path (carryover_type 0) is untouched, and an addon without
SCIResult.carryover still gets the amplitudes, since it has nothing else to build
the next subspace from.

Since the eigenvector is what carried the subspace dimension, print it instead;
run_sqd_sbd.py reports carryover sizes in place of dim and checkpoints the
carryover strings, keeping the checkpoint format and --resume_from unchanged. A
non-zero carryover_type that selects nothing now fails here, naming the cause,
rather than surfacing as the addon's "neither an SCI state nor a carryover" a
layer up.

run_sqd_sbd.py also gains the three SBD carryover flags, which it had no way to
set despite referencing --sbd_carryover_threshold in its own help. SBD reads only
one of ratio/threshold per type and silently ignores the other, and our ratio
default of 0.1 differs from upstream's 0.0, which makes threshold inert on types
1 and 2 out of the box -- so the driver warns when a setting cannot take effect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
qiskit-addon-sqd#369 merged, and review (#383/#388) made the ORDER of
SCIResult.carryover load-bearing: _select_carryover returns a solver's carryover
before applying any ranking, and a later truncation to max_dim keeps the leading
entries. #382 fixed the same class of bug on the sampling side, where a mis-ranking
let a lower-weight string survive truncation.

Only carryover_type 1 satisfies that unaided. It ranks half-determinants by
p(D^alpha) = sum_beta |c|^2, which is exactly the addon's marginal weight, and emits
them in that order -- so it alone may still skip the |adet| x |bdet| dump, which was
the point of the previous commit.

Types 2 and 3 singles-extend, and SinglesExtendHalfdets ends in sort_bitarray
(upstream extend.h), re-sorting into canonical order because SBD indexes subspaces
by binary search. That is correct for SBD and wrong for the addon: it destroys the
weight order, and the generated determinants never had a weight at all. Handing
that over unchanged would let max_dim keep the numerically smallest strings and
discard the high-weight parents.

So those two types now keep the dump and the solver re-ranks their carryover from
the amplitudes: strings that were in the subspace first, by descending marginal
weight via a stable argsort of the negated weights (matching the addon's own
_rank_carryover so the two agree on ties), then the weightless singles-generated
ones. That ordering is the right priority -- a max_dim truncation sheds additions
before it sheds a ranked parent.

Verified against merged main (not the pre-review trim-sqd branch): all four types
give -85.3976786965; type 1 returns sci_state=None with no dump; types 2/3 keep the
dump and their carryover is descending-weight with the unranked strings after. A
parametrized regression test pins that, since a silent reorder is invisible until
max_dim bites.

Also: the loop ranks sectors JOINTLY under symmetrize_spin and cannot redo that for
a carryover it receives, so our per-sector ranking will differ there; documented on
the helper. Stale reference to _select_carryover_by_threshold updated to the name
#383 consolidated it under. The addon floor stays >=0.13.1 for SPMD support, with
the carryover path documented as needing >=0.15.0 -- the gate itself introspects
SCIResult's fields rather than the version, which matters because an install from
merged main still reports 0.13.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… it goes dead

qiskit-addon-sqd 0.15.0 (#369) adds subspace policies, and its loop now raises if
it gets both policy= and carryover_threshold=. So the driver builds a
StandardPolicy and passes the threshold inside it. That is the same schedule as
before, but the driver can now serve as a template for a different policy without
tripping that check. On an addon older than 0.15.0 there are no policies, and the
driver passes carryover_threshold= directly as before.

When --sbd_carryover_type is non-zero, SBD reports its own carryover and
_select_carryover returns it before thresholding anything, so
--sqd_carryover_threshold silently does nothing. The driver now warns about that
on rank 0, as it already does for SBD's ratio/threshold pair.
--sqd_carryover_threshold defaults to None so that "was it passed?" can be
answered; it still resolves to 1e-4.

Other policies are deliberately not exposed as flags. Trim SQD needs at least four
knobs, its default trim_ratio does not fit this driver's num_batches, all of them
go inert under SBD carryover, and nothing measured here shows it helping.
examples/tpb/README.md shows how to pass a TrimPolicy from Python instead.

Verified on h2o with 2 ranks against merged addon main: the default run and the
SBD-carryover run both reproduce the -76.2359466308 gate, and the warning fires
only when the threshold is passed explicitly. Against 0.13.1, the fallback path
reproduces the gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The TPB README never mentioned the three --sbd_carryover_* flags this branch adds,
and two of its passages became false once SBD can choose the carryover: that the
carryover is always SQD thresholding the amplitudes, and that the amplitudes always
flow into the next iteration. The top README likewise said SBD has "no say in how
the subspace grows", which carryover_type now changes.

Adds an "SBD-selected carryover" section covering what each type does, the
ratio/threshold exclusivity, and what each type costs. Type 1 skips the
wavefunction dump; types 2/3 keep it so the driver can re-rank their canonically
sorted output. It also notes that a checkpoint written under carryover holds the
carryover strings rather than the full subspace.

Adds a Trim SQD recipe in place of driver flags. The recipe was run against merged
addon main: it executes on the bundled h2o pool, and passing both policy= and
carryover_threshold= raises, as the text says. The guidance on trim_ratio
(about 1/num_batches, so the merged subspace stays about one batch in size) is
labelled as a rule of thumb, because neither it nor the addon's 0.1 default has
been tuned.

GDB docs are untouched: neither GDB driver uses the SQD loop or policies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The inline comment "do not ALSO pass carryover_threshold=: it raises" read as if
it had been cut off, so it now sits under the call as a full sentence. The closing
paragraph keeps only its first sentence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sophia Wen (hfwen0502) and others added 2 commits October 7, 2026 15:01
#43's API reference documents every public-named function in sbd.sbd_solver,
because the module has no __all__. extract_carryover and
rank_carryover_from_amplitudes are internal to _solve_sci_core and would have
appeared there, so prefix them with an underscore. Nothing outside the module
calls them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@garrison Jim Garrison (garrison) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. This overall looks better than #32, which took the same approach but treated every nonzero carryover_type as weight-ordered. I have closed that in favour of this.

Claude made a few checks, which were all good:

  • Reviewed the ordering logic by executing _rank_by_marginal_weight against upstream's _rank_carryover from qiskit-addon-sqd main (b4ba0cc). It agrees on all 2000 randomized cases it tried, including tie-breaking: equal marginal weights preserve input order, matching upstream's np.argsort(-weights, kind="stable"). It is also dtype-preserving (int64/uint64/int32 pass through unchanged), correct for CI strings near the int64 limit, and tolerant of an unsorted solved array and of duplicate carryover entries. Cost is negligible — about 0.14 ms at max_dim=15000.
  • The ordering analysis is the part Claude flagged as most valuable, and it is a correctness point. Since _select_carryover returns a solver's carryover before applying any ranking, and the later max_dim truncation keeps the leading entries, the order decides which strings survive. Splitting type 1 (already ranked by sum_beta |c|^2, so the dump can be skipped) from types 2/3 (re-sorted canonically by sort_bitarray, so they need re-ranking from the amplitudes) is the right distinction, and treating all nonzero types as weight-ordered would silently drop the strings that mattered.
  • It also checked the symmetrize_spin argument by constructing symmetric and antisymmetric amplitude matrices: the two sectors' rankings come out identical in both cases, as the docstring claims. That matters because batch_to_ci_strings ignores carryover_strings_b under symmetrize_spin.

Three things to change

1. Version references: 0.15.0 should be 0.14.0.

SCIResult.carryover was on the 0.15.0 milestone when it merged, but it will actually ship in 0.14.0. There are ten references across README.md, examples/tpb/README.md, examples/tpb/run_sqd_sbd.py and python/sbd_solver.py, including the except ImportError: # qiskit-addon-sqd < 0.15.0 comments.

The code itself is unaffected — _addon_accepts_carryover() feature-detects the field rather than comparing versions, which is the right call and means the >=0.13.1 pin can stay as it is.

2. The new tests have no MPI variant.

Every other test in test/test_sqd_integration.py is paired as _standalone and _mpi (test_small_subspace_*, test_published_energy_*, test_diagonalize_fermionic_hamiltonian_*), because pytest-mpi's marker filter runs the suite both ways. test_sbd_carryover_skips_wavefunction_dump and test_sbd_carryover_is_weight_ordered each have a single variant, so neither runs under MPI.

3. Nothing exercises the carryover path through diagonalize_fermionic_hamiltonian.

Both new tests call solve_sci_batch directly and assert on the returned SCIResult. The only test that drives the addon loop is test_diagonalize_fermionic_hamiltonian_*, and SOLVER_CONFIG does not set carryover_type, so it runs the default type 0. (Perhaps this should be parametrized in the pytest sense.) The result is that these paths are not covered in CI:

  • the loop accepting sci_state=None
  • _select_carryover returning SBD's carryover, and the next subspace being built from it
  • the ordering contract mattering, i.e. a max_dim truncation keeping the strings that were ranked first
  • symmetrize_spin together with an SBD-supplied carryover

The N2 and Boston runs in the description do cover this end to end, and type 1 matching the plain-diagonalization reference to ten digits with no dump is convincing. But a single loop test with carryover_type=1 would keep it covered. The symmetrize_spin case is the one most worth a test, since the failure mode is a silently ignored beta list rather than an error.

One question, not a change request. _rank_by_marginal_weight sorts the carryover strings it finds in the solved subspace by weight, and appends the ones it does not find after them. That is the right priority: a max_dim truncation then sheds the singles-extended additions, which never had a weight, before it sheds a ranked parent.

The case Claude was unsure about is where nothing is found. Every string would land in the unranked group, the returned order would be whatever sort_bitarray produced, and a truncation would keep determinants in canonical order — which is to say arbitrarily, with respect to weight. It would be silent: no error, just a subtly wrong subspace in the next iteration.

That looks unreachable, since types 2/3 extend from the determinants SBD selected and those parents should themselves be in the carryover. But the function does not say so, and the reasoning is not local to it. Since found is already computed, the check is almost free — something like this, after the found line:

if not found.any():
    raise RuntimeError(
        f"none of the {carryover.size} carryover determinants were in the "
        "diagonalized subspace, so none of them has a marginal weight and the "
        "order returned here is SBD's canonical order, not a weight ranking. "
        "qiskit-addon-sqd truncates the carryover to max_dim by keeping the "
        "leading entries, so it would keep an arbitrary subset."
    )

not found.any() is the right bar rather than something stricter: it fires only when the ordering is genuinely meaningless. Requiring every parent to be present would fire on legitimate type 2/3 output, since SBD can extend from a determinant it did not itself keep. Either that, or a sentence in the docstring saying why the unranked group can never be everything — the point is mainly that the next reader should not have to re-derive it.


A couple of smaller notes:

  • _create_sbd_config's comment is correctly updated. Worth noting since it had said the carryover "is NOT consumed on this path", which this PR makes false.
  • Flagging the Boston timing as an unprofiled inference is the right way to report it.

On the non-rank-0 path: Claude initially raised whether other ranks should return an empty carryover for shape consistency, then found that the addon explicitly makes this a don't-care — "what it returns on the other processes is ignored entirely and need not be meaningful" — and all four result-consuming sites sit behind is_control_process(). So leaving that path alone is correct, and no change is needed.


This comment was drafted by Claude Opus 5 under my guidance.

Sophia Wen (hfwen0502) and others added 4 commits October 7, 2026 15:55
SCIResult.carryover was on the 0.15.0 milestone when #369 merged, but it ships in
0.14.0. Code is unaffected: _addon_accepts_carryover() detects the field rather than
comparing versions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
If none of a type 2/3 carryover is in the diagonalized subspace, nothing has a
marginal weight, and the order returned would be SBD's canonical order presented as
a ranking. A max_dim truncation would then keep an arbitrary subset, silently. That
should be unreachable, because the extension starts from parents that are themselves
in the carryover (measured on h2o: 35-196 found per case), but the reason is not
local to the function. Raise instead, and say why in the docstring. An empty solved
list now takes the same path rather than returning early.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…yover

Both carryover tests now follow the file's _check_* plus _standalone/_mpi pattern,
so pytest-mpi runs them both ways. The ordering test also asserts that at least one
carryover string is in the subspace; without that, its two ordering assertions would
pass on an empty selection.

The diagonalize_fermionic_hamiltonian test is parametrized over carryover_type 0
and 1. With 1 it covers the path the solver-only tests cannot: the loop accepting
sci_state=None and building its second iteration from SBD's carryover. Since the
test runs with symmetrize_spin, where the loop ignores carryover_strings_b, it also
asserts the two lists agree on every iteration, which catches a silently dropped
beta selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hfwen0502

Copy link
Copy Markdown
Member Author

Thanks. All three changes are in, plus the guard. #43 has been merged in too, so CI now builds the docs and the release note.

  1. 0.15.0 → 0.14.0 (8318cf5): all ten references, plus the release note and the PR description.
  2. MPI variants (ae4481c): both carryover tests now follow the _check_* / _standalone / _mpi pattern.
  3. Loop coverage (ae4481c): test_diagonalize_fermionic_hamiltonian_* is parametrized over carryover_type 0 and 1, with symmetrize_spin on. With type 1, a callback asserts on every iteration that sci_state is None and that the two carryover lists are equal, so a silently ignored beta list would fail the test. With max_iterations=2, the second iteration is built from SBD's carryover.

The guard (a7d515a): added as you proposed. _rank_by_marginal_weight raises when nothing in the carryover is in the subspace, and an empty solved list now takes the same path rather than returning early. I measured it on h2o before adding it: 35–196 parents are found in every type 2/3 case, so it won't fire on normal output. That check also showed the ordering test would have passed on an empty selection, so it now asserts found.any() as well.

Local: 8 serial / 15 MPI pass against qiskit-addon-sqd main, and sphinx-build -W is clean, with the two helpers absent from the API page.

CI runs against the released qiskit-addon-sqd 0.13.1, whose SCIResult has no
carryover field, so `r.carryover is None` raised AttributeError there. Read it with
getattr instead. The suite was only run locally against addon main before; it now
passes against both 0.13.1 (4 serial / 7 MPI, carryover tests skipped) and main
(8 / 15).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@garrison

Jim Garrison (garrison) commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Regarding f6e1ca1, I will add a CI workflow that tests against the latest main branch of the addon -- development version tests as mentioned in #6.

Since it is known to pass under both versions, CI should continue to be green once I add this.

EDIT: opened in #50.

Comment thread python/sbd_solver.py
---
features:
- |
SBD can now choose the carryover for the next SQD iteration. When

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
SBD can now choose the carryover for the next SQD iteration. When
When used together with `qiskit-addon-sqd`, SBD can now choose the carryover for the next SQD iteration. When

@garrison Jim Garrison (garrison) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great -- I just made a single suggestion to clarify a release note.

@hfwen0502
Sophia Wen (hfwen0502) merged commit c70d916 into main Oct 9, 2026
16 checks passed
@hfwen0502
Sophia Wen (hfwen0502) deleted the sbd-carryover-369 branch October 9, 2026 17:02
Sophia Wen (hfwen0502) added a commit that referenced this pull request Oct 9, 2026
…notes

- carryover-type-default-zero: a non-zero carryover_type now hands SBD's
  selection to the SQD loop, so only the default is a pure speedup.
- fix-fabricated-amplitudes: with carryover_type = 1 no wavefunction is
  requested and sci_state is None.
- remove-h-comm-size: name GDB's b_comm_size and t_comm_size too.
- fix-amplitude-labels: canonical order is ascending integer order by
  construction (#31), not by coincidence.
- thrust-mpi-build-options: SBD_THRUST_SAFE_MPI_ALLREDUCE may not be enough on
  its own; it leaves point-to-point device transfers unstaged.
- sqd-amplitudes-change-results: drop what repeats fix-fabricated-amplitudes.
- macos-support: match INSTALL.md (conda llvm-openmp first, Homebrew libomp as
  the fallback).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jim Garrison (garrison) added a commit that referenced this pull request Oct 9, 2026
#53 curated the autodoc list and added field tables, which changes where
this branch's documentation belongs.

The packing convention moves from `from_string` to `from_strings`. #53
documents the bulk form and deliberately leaves the singular one out, so
the convention was written on a function the rendered docs never show --
and `from_strings`, the form users are steered to, had no packing
documentation of its own. `from_string` and `makestring` now point at it
instead of restating it. The examples are shown as the `uint64` ndarray
`from_strings` actually returns rather than as nested lists.

`dump_matrix_form_wf` now appears in #53's `TPB_SBD` field table as well
as the README's, so the README row defers to the format section instead of
describing the field a second time.

Also corrects a claim this branch made about `sbd_solver.py`: since #49 the
wavefunction write is skipped when SBD selects the carryover itself
(`carryover_type` 1 against an addon that accepts it), because the next
subspace then comes back in `carryover_adet`/`carryover_bdet` and the
amplitudes never reach Python. Saying it "uses" the dump without that
qualification implied the write always happens.

`tox -e docs` passes with no warnings, and the packing convention renders
on the `from_strings` entry.

Assisted-by: Claude Opus 5
Jim Garrison (garrison) added a commit that referenced this pull request Oct 9, 2026
* Prepare 1.7.0 release

* Add mergify configuration

* Move release notes to release directory

* Move remaining note

* Tweak note text

* Release notes: correct what #49 and #46 made stale, and tighten four notes

- carryover-type-default-zero: a non-zero carryover_type now hands SBD's
  selection to the SQD loop, so only the default is a pure speedup.
- fix-fabricated-amplitudes: with carryover_type = 1 no wavefunction is
  requested and sci_state is None.
- remove-h-comm-size: name GDB's b_comm_size and t_comm_size too.
- fix-amplitude-labels: canonical order is ascending integer order by
  construction (#31), not by coincidence.
- thrust-mpi-build-options: SBD_THRUST_SAFE_MPI_ALLREDUCE may not be enough on
  its own; it leaves point-to-point device transfers unstaged.
- sqd-amplitudes-change-results: drop what repeats fix-fabricated-amplitudes.
- macos-support: match INSTALL.md (conda llvm-openmp first, Homebrew libomp as
  the fallback).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Only deploy docs from the stable branch

---------

Co-authored-by: Sophia Wen <hfwen@us.ibm.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Jim Garrison (garrison) added a commit that referenced this pull request Oct 9, 2026
…der (#55)

* Document how to retrieve TPB amplitudes, and the bitstring packing order

Two things a user has to discover by reading upstream C++ or experimenting.

TPB has two wavefunction output paths that write different formats, and
nothing said so. `savename` -- the one named in `tpb_diag`'s signature --
goes to `SaveWavefunction`, which writes one file per b_comm position
holding only that rank's block, behind a three-`size_t` header and the
block's determinant words. `sbd_data.dump_matrix_form_wf` goes to
`SaveMatrixFormWF`, which gathers the full adet x bdet subspace onto one
rank and writes a bare row-major float64 array with no header at all, or,
for a `.dat`/`.txt` path, a text table labelling each amplitude with its
own alpha and beta bitstring.

The second is what almost every caller wants, and is what this package's
own SQD integration already uses (`sbd_solver.py` sets it and reads the
result with a bare `np.fromfile`). It appeared only as a one-line pybind
docstring and a CLI flag in `run_sbd_diag.py`, absent from the README's
TPB section and from the `tpb_diag` docstring, both of which mention
`savename` alone -- so following the signature led to parsing the wrong
layout. Document both, side by side, and note that the text form is
written with default stream formatting and so carries ~6 significant
digits rather than full float64.

`from_string` and `makestring` packed words with no stated convention.
Bitstrings are packed from the right: the trailing `bit_length` characters
become word 0, so the leading characters of a multi-word string land in
the last word, not the first. Easy to get backwards, and a wrong guess
yields a valid-looking but wrong subspace rather than an error.

GDB needed no changes here: #46 documented its amplitude retrieval and
file layout in examples/gdb/README.md.

Assisted-by: Claude Opus 5

* Fit the amplitude and packing docs to the API surface #53 defined

#53 curated the autodoc list and added field tables, which changes where
this branch's documentation belongs.

The packing convention moves from `from_string` to `from_strings`. #53
documents the bulk form and deliberately leaves the singular one out, so
the convention was written on a function the rendered docs never show --
and `from_strings`, the form users are steered to, had no packing
documentation of its own. `from_string` and `makestring` now point at it
instead of restating it. The examples are shown as the `uint64` ndarray
`from_strings` actually returns rather than as nested lists.

`dump_matrix_form_wf` now appears in #53's `TPB_SBD` field table as well
as the README's, so the README row defers to the format section instead of
describing the field a second time.

Also corrects a claim this branch made about `sbd_solver.py`: since #49 the
wavefunction write is skipped when SBD selects the carryover itself
(`carryover_type` 1 against an addon that accepts it), because the next
subspace then comes back in `carryover_adet`/`carryover_bdet` and the
amplitudes never reach Python. Saying it "uses" the dump without that
qualification implied the write always happens.

`tox -e docs` passes with no warnings, and the packing convention renders
on the `from_strings` entry.

Assisted-by: Claude Opus 5
Jim Garrison (garrison) added a commit that referenced this pull request Oct 10, 2026
…der (#55) (#57)

* Document how to retrieve TPB amplitudes, and the bitstring packing order

Two things a user has to discover by reading upstream C++ or experimenting.

TPB has two wavefunction output paths that write different formats, and
nothing said so. `savename` -- the one named in `tpb_diag`'s signature --
goes to `SaveWavefunction`, which writes one file per b_comm position
holding only that rank's block, behind a three-`size_t` header and the
block's determinant words. `sbd_data.dump_matrix_form_wf` goes to
`SaveMatrixFormWF`, which gathers the full adet x bdet subspace onto one
rank and writes a bare row-major float64 array with no header at all, or,
for a `.dat`/`.txt` path, a text table labelling each amplitude with its
own alpha and beta bitstring.

The second is what almost every caller wants, and is what this package's
own SQD integration already uses (`sbd_solver.py` sets it and reads the
result with a bare `np.fromfile`). It appeared only as a one-line pybind
docstring and a CLI flag in `run_sbd_diag.py`, absent from the README's
TPB section and from the `tpb_diag` docstring, both of which mention
`savename` alone -- so following the signature led to parsing the wrong
layout. Document both, side by side, and note that the text form is
written with default stream formatting and so carries ~6 significant
digits rather than full float64.

`from_string` and `makestring` packed words with no stated convention.
Bitstrings are packed from the right: the trailing `bit_length` characters
become word 0, so the leading characters of a multi-word string land in
the last word, not the first. Easy to get backwards, and a wrong guess
yields a valid-looking but wrong subspace rather than an error.

GDB needed no changes here: #46 documented its amplitude retrieval and
file layout in examples/gdb/README.md.

Assisted-by: Claude Opus 5

* Fit the amplitude and packing docs to the API surface #53 defined

#53 curated the autodoc list and added field tables, which changes where
this branch's documentation belongs.

The packing convention moves from `from_string` to `from_strings`. #53
documents the bulk form and deliberately leaves the singular one out, so
the convention was written on a function the rendered docs never show --
and `from_strings`, the form users are steered to, had no packing
documentation of its own. `from_string` and `makestring` now point at it
instead of restating it. The examples are shown as the `uint64` ndarray
`from_strings` actually returns rather than as nested lists.

`dump_matrix_form_wf` now appears in #53's `TPB_SBD` field table as well
as the README's, so the README row defers to the format section instead of
describing the field a second time.

Also corrects a claim this branch made about `sbd_solver.py`: since #49 the
wavefunction write is skipped when SBD selects the carryover itself
(`carryover_type` 1 against an addon that accepts it), because the next
subspace then comes back in `carryover_adet`/`carryover_bdet` and the
amplitudes never reach Python. Saying it "uses" the dump without that
qualification implied the write always happens.

`tox -e docs` passes with no warnings, and the packing convention renders
on the `from_strings` entry.

Assisted-by: Claude Opus 5
(cherry picked from commit 90afb07)

Co-authored-by: Jim Garrison <garrison@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants