Skip to content

perf(validation): Collect moves its payload in one guest agent call, gzipped - #9

Merged
fadwen merged 1 commit into
mainfrom
perf/validation-collect
Oct 6, 2026
Merged

fadwen merged 1 commit into
mainfrom
perf/validation-collect

Conversation

@fadwen

@fadwen fadwen commented Oct 6, 2026

Copy link
Copy Markdown
Owner

Summary

Collect in the validation kit took 25 minutes on the lab device, almost all of it moving the payload: 18 MB of base64 read through 179 guest exec calls of 100,000 characters at about 6 seconds each. The same run now takes 3.9 minutes, and 2 minutes with -Since. Scripts going into the VM move the same way, 30,000 characters per call instead of 1,200.

Only the validation kit changes, plus one tenant object: the round-7 experiment REM-INSTALL-MODULE is no longer assigned, since it hangs for its full 60-minute timeout every hourly cycle and holds the lab's remediation runner for every experiment queued behind it.

Changes

  • GuestAgent.ps1: Send-GuestFile. Writes a file into the VM through the guest agent's file-write call, as base64 part files of 30,000 characters on the host's command line (under the 32,767 Windows allows), joined and decoded by one guest command; the VM's SHA-256 of the result is compared with the local one. Replaces the loop that wrote 1,200 characters per guest exec in both the driver and Invoke-LabGuestScript.ps1.
  • GuestAgent.ps1: Receive-GuestFile. Reads a file from the VM through the agent's file-read call, one reply of up to 16 MB, refused when the agent marks it truncated or the hash is not the VM's. Replaces Read-GuestPayload and its chunk loop.
  • Collect device script. Gzips the JSON before base64: 13.4 MB of JSON from a device with a month of hourly probe records packs to 380 KB, 507,000 base64 characters, which file-read returned in 2 seconds. The driver inflates it (Expand-GzipText).
  • Collect -Since <datetime>. Keeps only the probe records written from that time on. Record times are ISO 8601 in UTC, so the device compares them as strings. Every record is kept when the parameter is omitted, as before.
  • Lab. REM-INSTALL-MODULE's assignment removed through Graph; the policy stays in the tenant so Deploy skips it and its evidence stands. Noted in Findings and the README.
  • Tests. Send-GuestFile (part size, call shape, join and hash check, a refused write, a hash mismatch), Receive-GuestFile (call shape, truncated reply, hash mismatch, failed call), Expand-GzipText round trip, and Invoke-GuestScriptFile over the new delivery.

Verification

  • Kit unit tests: 62 pass on PowerShell 7.6.6. PSScriptAnalyzer (Error and Warning) is clean over Validation/; no line over 115 characters.
  • Live against VM 125: Invoke-LabGuestScript.ps1 delivered and ran a script in 16 seconds end to end. Collect without -Since: 3.9 minutes, 45 remediations, 11 platform scripts, 6,152 probe records, payload hash matched. Collect -Since 2026-10-05: 2 minutes, 181 probe records, oldest 2026-10-05 00:18 UTC. The previous run on the same device with the chunked transport took 25.2 minutes.
  • The gzip measurement: collect.b64 from the earlier run, 13,410,018 bytes of JSON, packed to 379,989 bytes in 0.7 seconds on the device.

Notes

  • file-read returns at most 16 MB. A payload that large after gzip would need the chunked path back; at today's 380 KB that is 40 times away.

…gzipped

The payload went through 179 guest exec calls of 100,000 characters at about 6 seconds each, 25 minutes for one Collect. The device now gzips the JSON (13.4 MB to 380 KB) and the driver reads it with the agent's file-read call in one reply; scripts go in through file-write, 30,000 characters per call instead of 1,200. Both checked by SHA-256. Collect -Since keeps only the probe records from a time on. 3.9 minutes for the same run, 2 with -Since.

REM-INSTALL-MODULE hangs for its 60-minute timeout every hour and held round 10's seven detections for 70 minutes; its assignment is removed, the policy stays.
@fadwen
fadwen merged commit 72414c4 into main Oct 6, 2026
4 checks passed
@fadwen
fadwen deleted the perf/validation-collect branch October 6, 2026 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant