Repository navigation
feat(exo-4d0): FileShare — a private file swarm on Logos Storage, the room as its tracker - #339
Merged
Merged
Conversation
…, the room as its tracker Design and plan for Jarrad's 'mini private torrent tracker': members share files sealed under a per-file key, the room log is the index and the peer list, storage_module moves ciphertext. Epic exo-4d0 with slices S0-S7 and the upstream asks to Logos Storage. Decisions: seed after a fetch by default (private fetch opt-out); seed events carry addresses for now. TDD-Exempt: a design document and its pebbles; no behavior. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
A file is sealed under a fresh key before it reaches Logos Storage: libsodium secretstream (XChaCha20-Poly1305) in 64 KiB messages, no magic bytes, a neutral upload name. Opening refuses, leaving nothing behind, unless every message authenticates, the stream ends at its FINAL message, and the plaintext has the committed size and domain-separated digest. Tested-Behavior: blob_seal_test — round trips from empty to many chunks; tamper, truncation, dropped final chunk, trailing bytes, wrong key, size or digest, and an authentic stream of other bytes all refuse with no file or partial left; a refusal never overwrites the destination. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
… folded from the log file/<fid> (a share in the sharer's name: name, size, plaintext digest, ciphertext CID, file key, scheme, storage network), file/<fid>/seed/<who> (on/off, peer id, addresses) and file/<fid>/retract, all author-signed, so a member is a seeder only by their own announce (invariant 9). reduceSwarm is order-free and idempotent (invariant 4); the file id is content-addressed over (sharer, CID) so nobody posts under another's id; a later joiner holds no earlier key until a member shares it again (invariant 7). Flow names the file, its key and the storage address to the room; the room map credits each file entry to its signer. Tested-Behavior: swarm_test — two members fold the same index; any order and duplicates give the same index; forged and unsigned seeds and shares and a squatted file id change nothing (a mutation skipping file/ signature checks turns it red); newest announce wins, ties resolve alike; only the sharer retracts; a later-epoch member sees nothing until a re-share; names carry no path; flow rows and room-map attribution. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…tch/seed flow BlobStore (F-17) with LocalBlobStore, the in-process stand-in modelled on storage_module 3.0.2: served means announced, connected peers then the DHT, a private download shows the server a mix path, and what servers and the DHT saw is recorded. SwarmNode drives share (seal, upload under a neutral name, announce), fetch (dial the seeders the room names, open and verify, save beside never over, seed by default) and seeding, from a pump. A private fetch dials no seeder and announces nothing; a refused blob is dropped, never served. LpStorage over lp_* waits on the S0 spike. Tested-Behavior: blob_store_test — content addressing, connected and DHT serving, not-served/missing/other-network failures, what a server sees for a plain and a private download, remove. swarm_live_test — share announces under a neutral manifest name; fetch verifies, saves and seeds; second fetch lands beside; a seeder serves when the sharer is gone; private fetch dials nobody and announces nothing; seeding toggles only for held blobs; a digest-mismatched blob is refused, unwritten and dropped; unknown and retracted files refused up front. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
Two observer classes, storage-peer and storage-dht. A share or a seed means a member's node serves the sealed blob, so the flow names what a served dataset always shows: the DHT the ciphertext CID and its provider, a fetching peer the ciphertext and the serving node's address — never the file, its name or its key. The room map draws those as edges from the file entry, and the flow view, card and map say who each observer is in plain words. Tested-Behavior: swarm_test §8 — storage-dht sees ciphertext-cid and provider, storage-peer sees ciphertext and node-address, neither ever sees the file or its key; the room map has leave edges from file entries to both with the swarm's mechanism. Full suite 262/262; qmllint clean on the three QML files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…ted to it The spike ran logos-storage v0.5.3 (the engine storage_module 3.0.2 and Basecamp 0.3.2 pin) and the real storage_module under logoscore. A direct connect alone cannot fetch: the manifest is asked only of DHT providers, and a connection only makes a listed one dialable. advertise=false refuses outright and its provider record outlives the stop. Mix works on logos.dev at ~15 KB/s. Basecamp's shared node is on logos.test with Mix on and seeds nothing. The design doc's §3, §5.1, §5.4, §5.5, §6 and §9 follow it, with a new ask U6 (fetch from named peers: with U1, the swarm needs no DHT at all), and LocalBlobStore now behaves as observed. Tested-Behavior: blob_store_test — only DHT providers serve; an unreachable seeder is dialable once connected, never by a private fetch; a connection without a provider record finds nothing; stale records outlive stop and remove while every listed provider refuses. swarm_live_test unchanged and green on the corrected stand-in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…ntees, via discuss-issue
Catastrophic. The restatement (s1-s5) was approved as a block before any
oracle was authored; each oracle was LLM-proposed and operator-confirmed
(sign-off on exo-4d0.4):
s1 model_check the fold is order-free and idempotent: 40320 orders as-is
and doubled, 256 subsets against the rule stated alone
s2 model_check a seeder only by their own signed announce: forged,
unsigned, other-room, edited, renamed and lifted seeds list
no one; a re-share or a chat claim lists no seeder
s3 model_check a share only in its sharer's name, under its derived id;
only the sharer retracts. v2 after mutation testing showed
v1 missed a deleted squat check: squats re-timed to sort
first and under an id no share holds
s4 model_check a later joiner holds no earlier key or CID (real sessions,
raw log checked) until a member shares the file again
s5 model_check names carry no path: hostile names through the real fetch,
the whole tree snapshotted; nothing written outside the
folder, nothing overwritten
s5 found a live bug on its first run: a dangling symlink planted under the
shown name was replaced by the fetch, because freeName asked fileExists and
dirExists, which follow the link. A symlink now counts as taken.
Tested-Behavior: scripts/grade-specs.sh exo-4d0.4 5/5; mutations caught — tie broken the other way (s1), file/ signatures unchecked (s2, s3), squat check dropped (s3 v2), retract by anyone (s3), raw names (s5), dangling symlink not taken (s5); swarm and blob tests and probes 9/9.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…in the runner's module set FileShare (the room swarm) uploads and fetches a room's sealed blobs through storage_module over lp_*. An optional dependency of muster_module, like the Monero backend, so a room works without it; bundled into the standalone runner through muster_ui's dependencies, as the RLN modules are. Pinned to 7548f41, the 3.0.2 Basecamp 0.3.2 bundles. TDD-Exempt: build wiring only; the runner build and the storage self-test exercise it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…age team A shareable write-up in the shape of the seaqt/nimside one: FileShare (the room swarm) as Jarrad's delivery-bounded private tracker, what was run and observed (graded as the S0 labbook grades it), what Muster already does on its side (sealed blobs, the room as index and peer list), and the ranked asks — U1 serve to an allowlist and U6 fetch from named peers first, which together take the DHT out entirely — each with the observation behind it, the workaround, and a possible shape. Also the pure half of the storage_module binding: replies, event payloads in every shape the host delivers, and jobs matched by upload session or download CID. Tested-Behavior: storage_events_test — StdLogosResult and bare replies, JSON-in-string values; event payloads as an argument array, arg0, data.arg0 or bare; uploads by session id, downloads by CID, progress, success and failure, another consumer's sessions moving nothing, a finished job never reopened. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…ge_module over lp_*, fileshare_* methods, the FileShare tab LpStorage drives the host's storage node over lp_*: it attaches, and inits a node only where none has been (the standalone runner, with MUSTER_STORAGE_CONFIG for the self-tests); events land in per-event queues drained on the module thread; a download finished for another consumer's session never counts as ours. muster.lidl gains fileshare_status / _share / _files / _fetch / _seed / _retract, one SwarmNode per room, pumped on the intents tick. The room details drawer gains a FileShare tab (share by path, the room's files with who shares and seeds them, fetch, fetch privately, seed, remove from list, and who sees what), reachable from the header menu; the C++ and Nim backends carry the slots; scripts/fileshare-self-test.sh drives two runners on a private storage network. Not yet run against storage_module at runtime: that needs the runner build in progress. The file picker is a path field until the view is seen on a display. Tested-Behavior: the whole module compiles with FileShare (probe_return_marshalling_host); storage_events_test, blob_store_test, swarm_test, swarm_live_test and the five derived-exo-4d0.4 probes green; qmllint clean on RoomFileShare, RoomDetails and RoomHeader. The runtime check is scripts/fileshare-self-test.sh, next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…without importFiles The UI builder's LIDL parser (basecamp's, ADR-013) refuses a method with no return type, and storage_module 3.0.2 declares importFiles(path: tstr). The runner still bundles the real module; muster_ui never calls it, and muster_module calls it over raw lp_*. Same shape as delivery's optional_depends workaround. TDD-Exempt: build wiring; the runner build exercises it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…++ keyword in the generated client) TDD-Exempt: a parameter rename; the runner build compiles the generated C++ client that refused it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…w the first end-to-end run scripts/fileshare-self-test.sh passed 3 of 3: two runners, each with its own storage_module node on a private storage network (own HOME each, so the operator's ~/.logos_storage is never written), A shares a 3 MB file, B fetches it byte-identical and seeds it, ~13 s. The design doc and CLAUDE.md record FileShare as the name, what landed, and what is open (install remedy, display, file picker, Basecamp). Tested-Behavior: scripts/fileshare-self-test.sh 3/3 against the rebuilt runner; scripts/two-instance-proof.sh still green on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…he shared config kept, no retry storm; install prompt
On the operator's machine `make run` beside Basecamp failed: the runner's own
storage node opened ~/.logos_storage/data, which Basecamp's node holds
("Should create metadata store!", LevelDB's lock), and Muster called init
again every 2 s. Now, where Muster has to start a node itself, it keeps it to
this instance (its own data dir and a free port), puts the shared
~/.logos_storage/config.json back exactly as it was (init rewrites it), adds the
node's loopback address to what it announces, waits 30 s after a failed start
and 60 s for a start to come up.
FileShare readiness asks the host's registry first (storage_readiness.nim):
not installed names storage_module and the FileShare tab offers Install
Logos Storage through Basecamp's packages.install, as a card's remedy does;
installed but not running waits without calling it; only a ready module, or
a host that cannot say, is asked about its node.
Tested-Behavior: storage_readiness_test (not installed names the package and asks nothing else; not running never calls; error carries the reason; ready or unknown defers to the node). FILESHARE_ONE_HOME=1 scripts/fileshare-self-test.sh was red on the previous runner — the shared config rewritten, B's init refused and retried 340 times; the rebuilt runner is checked next.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session-Id: d70aed18-858f-5636-9c07-b85804e17891
TDD-Exempt: pebble notes only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
…uliano's reply Giuliano (Storage lead): we want a permissioned tracker, and 'serve to named peers, fetch from named peers' would wire the overlay by hand. Right. The asks now name the split he described: the room supplies the peer lookup for its own datasets, over the Discovery seam that exists (find/provide, storage/discovery.nim, used by the manifest fetch, block discovery and the advertiser), and Storage keeps sampling, scheduling and failure recovery. Ask 2 is serving without announcing; admission is optional. A layer table gives the whole picture he asked for; the open questions ask what the swarm needs from a lookup. room-swarm.md §6 follows. TDD-Exempt: documentation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
TDD-Exempt: pebble notes only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: d70aed18-858f-5636-9c07-b85804e17891
Fergulati
pushed a commit
to Fergulati/muster
that referenced
this pull request
Oct 24, 2026
…dress, the wallet button Found on the Monero send flow (2026-10-23): - Mark received before anything arrived closed the send unpaid. The payer's link went away, and their card said the payee's wallet had confirmed it. Now Mark received takes a second press, its warning in place of the share's status, and Cancel undoes the first. On Monero it is offered only once the payer has said they paid: the payee's own wallet confirms on its own, so before a report there is nothing to mark. A share closed by the payee's word now reads "<payee> marked it received" on the payer's card, not "<payee>'s wallet confirmed your payment". The module still takes the payee's word (a request paid in cash, probe_xmr_no_spend_stepper). - A second send to someone reused their last shared Monero address, which already backs a payment, and the module refused it (shared-address-used) with no way forward. The form now applies the module's rule: a used address doesn't count, the form says so, and it offers "Ask <name> for a new Monero address". - Pay with Monero Wallet raised monero.accounts.manage, which Monero Wallet answers by opening its Settings tab. Monero Wallet 0.1.1 takes no address or amount from another app, so nothing can prefill its Send yet. The button is now "Copy address, open Monero Wallet": it copies the address and brings the wallet up on its own view through the shell's basecamp.apps.launch (declared in uses). The steps say the wallet can't be filled in from here. A prefilled Send needs an upstream intent: exo-914. Also: FakeBackend.qml regenerated (stale on main since corpetty#339's FileShare slots). Tested-Behavior: gallery fixture card-mark-received (a recorded split-self-test snapshot of the payee with an agreed, unpaid split): the first press arms with the warning and Cancel, Cancel disarms, the second press calls confirmPart — 5 checks, 0 QML warnings; the full gallery 152 ok, one failure (settings-btc-payer) that fails identically on main (exo-a908); ui-lint 0 new; MUSTER_UI=dev ui-parity 12/12. The Monero-only parts (the gate on a report, the used-address form, the wallet launch) were not rendered: no recorded Monero room exists, and the runner bundles no Monero wallet Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Session-Id: 836705fe-fda9-501b-8ac2-1ce50b5af64a
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FileShare (epic exo-4d0): members of a room share files with each other and seed them for each other, with the room as a private tracker and Logos Storage moving the bytes. Muster's first use of Logos Storage. Design:
docs/design/room-swarm.md.crypto/blob_seal.nim):.binname, so Storage's manifest names nothing;coordination/swarm.nim):file/<fid>,file/<fid>/seed/<who>andfile/<fid>/retractare author-signed log events, folded byreduceSwarm;Typed spec
derived-exo-4d0.4(catastrophic, via discuss-issue) has five model-checked guarantees. Its s5 probe found a live bug, now fixed: a planted dangling symlink was replaced by a fetch.storage/).BlobStore, with a local stand-in modelled on what the S0 spike saw at runtime.LpStoragetalks tostorage_module3.0.2 over lp_*:~/.logos_storage/config.json;fileshare_status,_share,_files,_fetch,_seedand_retractinmuster.lidl, pumped on the intents tick. Two new observers, storage-peer and storage-dht, appear in the flow view and the room map.packages.installwhen it is missing.storage_moduleis optional formuster_moduleand bundled in the runner;importFiles, which that builder's parser refuses.docs/labbook/storage-from-muster.md);docs/design/storage-asks.md), rewritten after their lead's reply as a permissioned tracker: a consumer-supplied peer lookup over the existingDiscoveryseam, plus serving without announcing.Rebased onto main, which resolved conflicts in
authorship.nim(both sides added author kinds; both are kept), the generatedmuster_gen.nim(regenerated, not hand-merged) andCLAUDE.md. The pre-rebase history stays onfeat/room-swarm.Not done yet: a try in Basecamp, a native file picker (a path field for now), and a fetch with the sharer gone across three real instances (held in process only).
Test plan
All against a runner built on bugger from this branch's head (45cde06), uncapped:
module/tests/run-suite.sh: 275/275 (unit tests and invariant probes, including the fivederived-exo-4d0.4probes).scripts/grade-specs.sh exo-4d0.4: 5/5. Mutation checks during authoring (tie order, unchecked file signatures, a dropped squat check, retract by anyone, raw names, symlinks) each turned a probe red.scripts/fileshare-self-test.sh: two runners on a private storage network; B fetches A's 3 MB file byte-identical and seeds it. Green on this head; 3/3 before the rebase.FILESHARE_ONE_HOME=1 scripts/fileshare-self-test.sh: both runners on one HOME that already holds a storage config, the shape ofmake runbeside Basecamp. It was red before the fix (config rewritten, 340 init retries) and is green after; the shared config is left byte-identical.scripts/ui-parity.sh(with foundry from nixpkgs): 12/12.scripts/ui-lint.sh: 0 new findings against the baseline.🤖 Generated with Claude Code