M3a: the twin job server — the D14 wire contract (CompiledJob in, RawAcquisition out) - #30
Merged
Merged
Conversation
The director-generated golden fixture (swept-amp pi-pulse + measure, 11 expts x 100 reps x soft_avgs 2, testbench soccfg, strumento main + qick 0.2.422) is the D14 wire contract's ground truth; the no-sweep variant (expts null: the calibration pi gain, no loop axis) is generated with the same two-line recipe (Device.load -> Seq -> StrumentoProgram -> to_compiled_job -> to_wire) for the absent-axis shaping test. Anatomy verified against qick source (NpEncoder/decode_array, dump_prog, cfg2reg, freq2reg): envelopes = base64 int16 (n,2) I/Q pages with addr + next_addr addressing; waves = freq/phase/env/gain/length/conf assignments; prog_list/labels = the tProc lane (not interpreted — the envelope-level boundary); loop_dims/avg_level/labels = the declared loop structure; acquire = the output shape contract.
…N triggers) The server rides its OWN extension — triggers Piccolo AND JSON (the payload is JSON by contract; JSON.jl 1.7 is the light pure-Julia wire codec) — never StrumentoPiccoloExt, whose triggers adding JSON would break the piccolo-only load configuration the config checks pin. Nothing new in base: a weakdep, an extensions entry, a compat pin, and the test-target injection (Pkg.test runs the full configuration; CI unchanged). The module docstring states the boundary up front — the twin models the DEVICE response at the envelope level, never tProc-v2 control-flow semantics (the assembly-faithful lane is Python SimulatorSoc's; complementary lanes) — and the two wire-stack decisions (stdlib-Sockets HTTP; lazy sibling-extension reach for TwinSoc). Slice 1 is the surface only: the TwinJobServer name and its extension attachment, reached via Base.get_extension, never leaking onto the parent module.
read_payload(soccfg, job_wire) decodes qick's dump_prog() through NpEncoder semantics: envelope pages (base64 int16 (n,2) I/Q via decode_array's spec, per-gen addressing with the next_addr watermark), the wave table (freq/phase registers via int2freq/deg2reg's inverse, gain/length codes, the conf bits via cfg2reg's layout), the played port plan (the static WPORT_WR pairing — which wave plays on which generator, read as DATA, never as control flow), and the declared loop structure + acquire block with the shape integrity check (expts realized from loop_dims/avg_level, validated against the acquire block's declaration — the same refuse-on-mismatch rule as the reference board-side agent). Wire numbering is 0-based throughout (generator channels, tProc ports, wave indices in the plan are the payload's own). The golden fixture decodes end-to-end: the 1152-sample gaussian page, the gain-0 wave at 4000 MHz, the [100, 11] loop with the averaged axis at level 0, expts 11.
The reader names the defect and its location: missing contract keys (the
{overlay_id, program, acquire} form, dump_prog's central keys), envelope
malformations (truncated base64, non-(n,2) shapes, non-int16 dtypes), wave
entries missing fields, tmux conf bits (muxed generators are outside v1's
envelope level), port-plan references past the wave table, avg_level outside
the loop structure, and the full readout-surface integrity trio — the
acquire block's ro_chs and reads_per_shot are validated against the
program's own declarations (ro_chs keys, per-channel trigs) exactly like the
reference board-side agent: a declared shape that disagrees with the shipped
program is a failed job, never a silently mis-shaped buffer.
…tructed translate_drive(payload): per generator channel, the port plan's waves in play order reconstruct the analog drive at the envelope level. Envelope samples scale from DAC codes to fractions of full scale (code/maxv), the gain applies as the amplitude scale (gain_code/maxv, qick's -1..1 contract), the carrier phase rotates the baseband quadratures, outsel is honored per cfg2reg's semantics (product | dds | input | zero — the const-pulse path included), and the segments concatenate in play order (the flat-top shape; inter-wave TIMING is the tProc's lane). The envelope word addressing resolves through the pages (wave.env is a word address INTO the page space — the flat-top ramp-down resolves the same way), with the extent validated against the page. The carrier frequency is validated against the declared Nyquist zone and carried as the drive's frame definition — the v1 boundary stated in the module docstring: the family systems are rotating-frame models at the drive frequency; detuning modeling needs a family that carries absolute transition frequencies (future record surface, honestly out of v1).
The soc-level actor over one twin face (family + confusion + seeded response), the overlay personality refusal (D25: one overlay, one snapshot), and the server-owned clock (drift advances ACROSS jobs via advance!, dt=0 per acquire — job k measures truth aged (k-1)·dt). execute_job runs the wire payload through the twin face and shapes the RawAcquisition wire form: (n_reads, [expts,] 2) IQ as JSON-safe nested lists — the expts level ABSENT when the payload declares no sweep — exactly Python's RawAcquisition.to_wire (tolist)/from_wire (asarray) round trip. Conventions documented in the docstring: quantum time in ns (dac_rate = fs in samples per ns), the averaging depth (reps × soft_avgs batched into one accumulated draw from the twin's single seeded rng), the 2-outcome IQ packing, and the v1 amplitude scale (a full-scale DAC drive is 1.0 family unit — a calibrated rad/ns-per-full-scale mapping is future record surface). The CloseLoop sweep ladder (qick's encoding-A: read_wmem → literal wave-field increments → write_wmem, compiled inside the expts loop) is decoded in read_payload as static DATA — never tProc register simulation (the expts AXIS still rides the declared loop structure). translate_drive gains the per-expt parameter: the stepped wave's gain code offsets by step × (expt − 1); a payload with no ladder replays the identical assignment. v1 realizes gain steps only; anything else (orphan WMEM_WR, register arithmetic, non-gain fields, one ladder writing a wave twice) is named, not guessed.
…expt
A committed real-span fixture (compiled_job_realspan.json, generated from the
fresh Python stack — loopback device, testbench soccfg, reps 100, soft_avgs 2,
overlay testbench-v2 — the same recipe that reproduces the zero-span golden
byte-for-byte with Sweep("amp", 0, 0, 11); the real span is
Sweep("amp", 0, 8191, 11)).
qick compiles the real span to the CloseLoop encoding-A ladder: read_wmem →
literal wave-field increments → write_wmem — and, discovered here, a SECOND
restore ladder AFTER the expts loop (the compiler leaving wave memory as it
found it, once per reps iteration). The reader anchors the ladder split on the
expts loop's back-edge (the TEST whose literal counter equals the DECLARED
expts count − 1, followed by the conditional JUMP — a consistency check
against the declared loop structure, not register simulation): the in-loop
ladder realizes per-expt steps, the out-of-loop restore is decoded and
validated but never realized. Wave-field registers are qick's own map
(QickProgramV2.REG_ALIASES); v1 realizes gain steps only, one ladder per wave,
and refuses orphan WMEM_WRs, register-arithmetic steps, and non-gain fields by
name.
The IQ-trend AC rides the fixture: the excited-state frequency rises with the
ladder-stepped gain (the rising edge pinned; the toy's detuned response is a
resonance lobe — the contract is the trend, not monotonicity). The ladder
decode is pinned as exact integers: [(1, "gain", 819)] — the declared
0..8191 span register-rounded to the realized axis, what the tProc steps.
…tocol
The queue mirrors the reference agent (examples/jobserver/server.py): submit!
enqueues incrementing-string ids FIFO; poll is the single worker's turn (one
board, one worker — hardware exclusivity is structural) and returns the status
dicts the Python JobServerClient promises — {"status": "pending"} while
queued, {"status": "done", "acquisition": {...}} on success,
{"status": "error", "error": "<type>: <message>"} on failure, the
payload never riding the status dict. A failed job is a failed JOB, not a
server crash — the next submit still runs.
serve_http speaks the two wire routes over stdlib Sockets (HTTP/1.1, one
request per connection, JSON bodies): POST /jobs → {"job_id"}, GET
/jobs/<id> → the status dict with unknown ids mapped to 404 (a failed job is
still a KNOWN job — 200 with the error as data). The accept loop is an @async
task; each connection is handled independently — a malformed request costs its
own connection, never the server. stop_http closes the listener and joins the
task; queued jobs stay pending (a stopped server is a stopped board — draining
the queue is the operator's act).
Sockets rides base [deps] as the stdlib edge it is (the Base64 precedent:
stdlibs load everywhere, no new package dependency edge, base-only consumers
unchanged) — the HTTP-via-stdlib decision the issue asks to be documented.
The contract test drives the server exactly the way the Python
JobServerClient would: submit → poll → the RawAcquisition wire form, over a
real socket, with the malformed-payload survival and the 404 mapping pinned.
…ment
The seeded-replay AC at the wire: two FRESH Julia processes (fresh twins,
fresh rngs, the same seed) produce BIT-IDENTICAL server responses — the JSON
strings compare equal, a replay pin (==), not a captured-value golden (the
campaign CI rule). Sampled mode: the shot draws are the seeded surface this
pins. Different seeds differ.
The README gains the twin-job-server section: the D14 wire diagram, the
explicit envelope-level boundary (the twin models the DEVICE response; tProc
control flow is SimulatorSoc's complementary lane), the reference agent's
queue shape, the stdlib HTTP routes, the soc-level actor's across-job drift
clock, and the documented conventions — and the stale Status line ("the wire
server [is a] later slice") now says what landed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #29.
The twin gets a WIRE presence: a Julia job server speaking the D14 contract over HTTP/JSON —
CompiledJobwire form in (qick'sdump_prog()throughNpEncoder), envelope-level translation (base64 int16 I/Q pages decoded, thewavesassignments placed on the DAC grid with freq/phase/gain applied), execution through the twin face (the same seeded confusion+binomial machinery, drift included),RawAcquisitionwire form out ((n_ch, n_reads, [expts,] 2)IQ averaged over reps × soft_avgs, shaped per the acquire block).The documented boundary: the twin models the DEVICE response at the envelope level, NOT tProc-v2 control-flow semantics — the assembly-faithful lane is Python
SimulatorSoc's; the two are complementary.Vertical-slice TDD, incremental commits. AFK (the director merges).