diff --git a/.env.example b/.env.example index 34e41ed..ccbfc89 100644 --- a/.env.example +++ b/.env.example @@ -94,18 +94,18 @@ SYSCOIN_BLOCKBOOK_URL= # Blank disables management. No operator keys belong in this file or service. SYSCOIN_SENTRY_GENESIS_HASH= -# Optional public Tornado Classic ETH pool index. Disabled by default. -# Set BOTH values to opt in. The RPC must support Ethereum mainnet finalized +# Public Tornado Classic ETH pool index. Enabled by default when the RPC below +# is configured; set PRIVACY_INDEX_ENABLED=false to disable it. The RPC must support finalized # headers, historical state and eth_getLogs from the 2019 deployment blocks. # HTTPS only; no wallet keys or caller-supplied endpoints. First request opts a # pool into backfill and periodic refresh; previously used pools resume on restart. # Unused pools stay idle. See docs/privacy-index.md for lifecycle and feed trust. -PRIVACY_INDEX_ENABLED=false +PRIVACY_INDEX_ENABLED= PRIVACY_INDEX_RPC_URL= # Optional anonymous public relayer directory. Uses the operator-controlled # Ethereum mainnet RPC above but is independent of PRIVACY_INDEX_ENABLED. -# Never accepts caller URLs or notes. Defaults off; first bulk read opts in. -# See docs/privacy-relayers.md before enabling; no relayers are contacted by -# installation, tests or deployments while this flag remains false. -PRIVACY_RELAYER_DIRECTORY_ENABLED=false +# Never accepts caller URLs or notes. Enabled by default with the RPC above; +# set PRIVACY_RELAYER_DIRECTORY_ENABLED=false to disable it. The first bulk read +# starts discovery; see docs/privacy-relayers.md for lifecycle and privacy. +PRIVACY_RELAYER_DIRECTORY_ENABLED= diff --git a/README.md b/README.md index 2509161..c7cc765 100644 --- a/README.md +++ b/README.md @@ -21,9 +21,9 @@ should use lowercase. - `GET /mnlist` — fresh `masternode_list` RPC passthrough - `POST /mnsearch` — paginated/searched view over the in-memory tracker snapshot - `POST /govlist` — active + historical governance proposals (Syscoin Core `gobject list`) -- `GET /api/privacy/v1/pools/:poolId/snapshot` — opt-in anonymous Classic ETH +- `GET /api/privacy/v1/pools/:poolId/snapshot` — anonymous Classic ETH pool snapshots for local wallet verification; see [configuration and protocol](docs/privacy-index.md). -- `GET /api/privacy/v1/relayers` — separately opt-in anonymous Classic ETH +- `GET /api/privacy/v1/relayers` — anonymous Classic ETH relayer discovery; see [source checks and endpoint boundaries](docs/privacy-relayers.md). Authenticated routes (cookie + CSRF, same-site): diff --git a/docs/privacy-index.md b/docs/privacy-index.md index d27e260..bd093ba 100644 --- a/docs/privacy-index.md +++ b/docs/privacy-index.md @@ -18,9 +18,11 @@ These are pinned from the original ## Configuration and deployment -The feature is disabled unless `PRIVACY_INDEX_ENABLED=true` and -`PRIVACY_INDEX_RPC_URL` supplies an operator-controlled HTTPS Ethereum mainnet -RPC. URLs containing credentials or fragments are rejected. A provider API +The feature is enabled by default when `PRIVACY_INDEX_RPC_URL` supplies an +operator-controlled HTTPS Ethereum mainnet RPC. Set `PRIVACY_INDEX_ENABLED=false` +to disable it independently. With no RPC it remains disabled; explicitly setting +`PRIVACY_INDEX_ENABLED=true` without an RPC is a configuration error. No default +RPC is selected. URLs containing credentials or fragments are rejected. A provider API token may be in its path/query; it never enters responses or application logs. The RPC must support `finalized` headers, historical `eth_call`/`eth_getCode`, and `eth_getLogs` from the deployment blocks. Scans start with 5,000-block @@ -30,7 +32,8 @@ successful full ranges, the scanner doubles the range toward the original fallback to unfinalized data or to a different provider. A provider which cannot serve even one block of historical logs needs a suitable endpoint before enabling. -An unused pool sends no RPC. Its first request durably activates it and starts +Enabling the endpoint does not start a new pool's backfill by itself. An unused +pool sends no RPC. Its first request durably activates it and starts a job, bounded to 16 chunks and a 45-second scheduling budget (an already-started RPC can take another 10 seconds). A global queue runs only one job at a time; requests for the same pool share it. Activated pools continue unfinished backfill @@ -39,6 +42,8 @@ requiring another client request. Failures back off for 30 seconds. On restart, only previously activated pools resume. The single scheduled timer is unreferenced; `index.close()` cancels future work and waits for the current bounded job. The standalone app exposes its index handle for test/server lifecycle cleanup. +Any caller with access to the endpoint can activate a pool, and the resulting +backfill and refresh consume provider quota and may incur charges. Per-process IP limits allow 60 requests/minute and apply IPv6 subnet bucketing. Run one writer process for this file store; multiple workers require explicit shared locking and shared request limits before deployment. @@ -46,12 +51,12 @@ shared locking and shared request limits before deployment. The backend process needs write access to `data/privacy-index/`, resolved from its working directory. Checkpoints and complete snapshots live in separate files with checksums and are published by same-directory atomic rename. An -allowlisted per-pool activation marker preserves opt-in intent even if the first +allowlisted per-pool activation marker preserves activation even if the first RPC fails. Existing snapshot/checkpoint filenames also identify previously used pools during migration; unknown filenames activate nothing. They contain only public data, can be rebuilt, and never change `sysnode.db` or its schema. The existing deployment preserves `data/`. The feature needs no SQL -migration, but the opt-in environment values and nginx route must be configured +migration, but the Ethereum RPC and nginx route must be configured separately; automatic deployment does not modify the host environment or nginx. The internal `/privacy/v1` mount also supports existing proxies which strip `/api`; clients still use the public `/api/privacy/v1` URL. See the diff --git a/docs/privacy-relayers.md b/docs/privacy-relayers.md index bc9b903..bd0a17c 100644 --- a/docs/privacy-relayers.md +++ b/docs/privacy-relayers.md @@ -7,11 +7,12 @@ balance computation and the optional direct-withdrawal path. ## Deployment -The directory is disabled by default. Opt in with -`PRIVACY_RELAYER_DIRECTORY_ENABLED=true` and the existing -`PRIVACY_INDEX_RPC_URL` set to an operator-controlled HTTPS Ethereum mainnet RPC. -`PRIVACY_INDEX_ENABLED` does not enable the directory and need not be enabled -for this route. The RPC needs historical `eth_getLogs`, `eth_getStorageAt`, +The directory is enabled by default when `PRIVACY_INDEX_RPC_URL` is set to an +operator-controlled HTTPS Ethereum mainnet RPC. Set +`PRIVACY_RELAYER_DIRECTORY_ENABLED=false` to disable it independently. With no +RPC it remains disabled; explicitly setting the directory flag to `true` without +an RPC is a configuration error. `PRIVACY_INDEX_ENABLED` does not control the +directory. The RPC needs historical `eth_getLogs`, `eth_getStorageAt`, `eth_getCode` and `eth_call`, plus finalized block support. There is no fallback to `latest` or a caller-provided provider. Existing deployments stripping `/api` before proxying are supported by the internal `/privacy/v1` mount; clients @@ -21,7 +22,10 @@ proxy already forwards this namespace. Access logs should be disabled for the public privacy namespace, including any upstream proxy/CDN, if operators do not want to retain client IP/access metadata. -The first request activates one directory scanner. It persists only public ENS +Any caller with access to the endpoint can activate one directory scanner. The +first request can initiate RPC traffic +and relayer status checks; a configured provider may charge for those reads. +Enabling the endpoint alone does not start a new scan. The scanner persists only public ENS registration candidates and a canonical scan checkpoint under `data/privacy-relayers/registry-checkpoint.json`. A persisted checkpoint resumes after restart; an unused directory sends no RPC or relayer requests. Endpoint diff --git a/lib/privacyIndex.js b/lib/privacyIndex.js index 776859d..d3faba7 100644 --- a/lib/privacyIndex.js +++ b/lib/privacyIndex.js @@ -272,8 +272,11 @@ function createPrivacyIndex({ rpc, store = createPrivacyStore(), now = Date.now, function configuredPrivacyIndex({ enabled = process.env.PRIVACY_INDEX_ENABLED, rpcUrl = process.env.PRIVACY_INDEX_RPC_URL, ...options } = {}) { if (enabled !== undefined && enabled !== '' && enabled !== 'true' && enabled !== 'false') throw new Error('PRIVACY_INDEX_ENABLED_invalid'); - if (enabled !== 'true') return null; - if (!rpcUrl) throw new Error('PRIVACY_INDEX_RPC_URL_required'); + if (enabled === 'false') return null; + if (!rpcUrl) { + if (enabled === 'true') throw new Error('PRIVACY_INDEX_RPC_URL_required'); + return null; + } return createPrivacyIndex({ rpc: createPrivacyRpc(rpcUrl), ...options }); } diff --git a/lib/privacyRelayerDirectory.js b/lib/privacyRelayerDirectory.js index 2931889..35146fc 100644 --- a/lib/privacyRelayerDirectory.js +++ b/lib/privacyRelayerDirectory.js @@ -194,8 +194,11 @@ function createPrivacyRelayerDirectory({ rpc, inspect, store, now = Date.now, se function configuredPrivacyRelayerDirectory(env = process.env) { const enabled = env.PRIVACY_RELAYER_DIRECTORY_ENABLED; if (enabled !== undefined && enabled !== '' && enabled !== 'true' && enabled !== 'false') throw new Error('PRIVACY_RELAYER_DIRECTORY_ENABLED_invalid'); - if (enabled !== 'true') return null; - if (!env.PRIVACY_INDEX_RPC_URL) throw new Error('privacy_relayer_rpc_required'); + if (enabled === 'false') return null; + if (!env.PRIVACY_INDEX_RPC_URL) { + if (enabled === 'true') throw new Error('privacy_relayer_rpc_required'); + return null; + } const directory = createPrivacyRelayerDirectory({ rpc: createPrivacyRelayerRpc(env.PRIVACY_INDEX_RPC_URL), inspect: createPrivacyRelayerStatus(), store: createRelayerCheckpointStore(path.resolve('data/privacy-relayers')) }); // Persisted public registry checkpoints resume; a never-used directory is idle. diff --git a/tests/privacyIndex.test.js b/tests/privacyIndex.test.js index 525defc..7d9d27b 100644 --- a/tests/privacyIndex.test.js +++ b/tests/privacyIndex.test.js @@ -40,11 +40,41 @@ describe('public privacy index lifecycle', () => { afterEach(async () => { await fs.rm(dir, { recursive: true, force: true }); }); const setup = (dir, rpc, extra = {}) => createPrivacyIndex({ rpc, store: createPrivacyStore(dir), autoStart: false, refreshMs: 0, retryMs: 0, continueMs: 0, ...extra }); - test('disabled config performs no RPC or file IO and opt-in validates configuration', () => { + test('default configuration enables the index only with an RPC and supports explicit opt-out', async () => { + const post = jest.spyOn(require('axios'), 'post'); + expect(configuredPrivacyIndex({ enabled: '', rpcUrl: '' })).toBeNull(); expect(configuredPrivacyIndex({ enabled: 'false', rpcUrl: 'broken' })).toBeNull(); expect(() => configuredPrivacyIndex({ enabled: 'true', rpcUrl: '' })).toThrow('PRIVACY_INDEX_RPC_URL_required'); expect(() => configuredPrivacyIndex({ enabled: 'yes' })).toThrow('PRIVACY_INDEX_ENABLED_invalid'); expect(() => configuredPrivacyIndex({ enabled: 'true', rpcUrl: 'http://localhost' })).toThrow('privacy_rpc_url_invalid'); + expect(() => configuredPrivacyIndex({ enabled: '', rpcUrl: 'http://localhost' })).toThrow('privacy_rpc_url_invalid'); + for (const enabled of ['', 'true']) { + const index = configuredPrivacyIndex({ enabled, rpcUrl: 'https://operator.example', store: createPrivacyStore(dir) }); + expect(index).not.toBeNull(); + await index.start(); + expect(post).not.toHaveBeenCalled(); + await index.close(); + } + post.mockRestore(); + }); + + test('omitting the flag uses the configured process RPC by default', async () => { + const previousFlag = process.env.PRIVACY_INDEX_ENABLED; + const previousRpc = process.env.PRIVACY_INDEX_RPC_URL; + delete process.env.PRIVACY_INDEX_ENABLED; + delete process.env.PRIVACY_INDEX_RPC_URL; + try { + expect(configuredPrivacyIndex()).toBeNull(); + process.env.PRIVACY_INDEX_RPC_URL = 'https://operator.example'; + const index = configuredPrivacyIndex({ store: createPrivacyStore(dir) }); + expect(index).not.toBeNull(); + await index.close(); + } finally { + if (previousFlag === undefined) delete process.env.PRIVACY_INDEX_ENABLED; + else process.env.PRIVACY_INDEX_ENABLED = previousFlag; + if (previousRpc === undefined) delete process.env.PRIVACY_INDEX_RPC_URL; + else process.env.PRIVACY_INDEX_RPC_URL = previousRpc; + } }); test('returns progress first and only publishes complete pinned snapshots', async () => { diff --git a/tests/privacyRelayerDirectory.test.js b/tests/privacyRelayerDirectory.test.js index bb5a549..97784e5 100644 --- a/tests/privacyRelayerDirectory.test.js +++ b/tests/privacyRelayerDirectory.test.js @@ -40,9 +40,29 @@ function setup({ saved, candidates = [candidate], end = DEPLOYMENT_BLOCK + 2, ov } async function ready(s) { await s.directory.read(); await s.directory.idle(); return s.directory.read(); } -test('directory is opt-in and configuration requires a separately enabled flag', () => { - expect(configuredPrivacyRelayerDirectory({ PRIVACY_INDEX_RPC_URL: 'https://operator.example' })).toBeNull(); - expect(() => configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: 'true' })).toThrow('rpc_required'); +test('directory defaults on with an RPC, remains idle until requested, and supports opt-out', async () => { + const post = jest.spyOn(require('axios'), 'post'); + // A developer's real data directory may contain an active checkpoint; this + // test must exercise a fresh install and never resume that scanner. + const stat = jest.spyOn(fs, 'stat').mockRejectedValue(Object.assign(new Error('not found'), { code: 'ENOENT' })); + try { + expect(configuredPrivacyRelayerDirectory({})).toBeNull(); + expect(configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: '', PRIVACY_INDEX_RPC_URL: '' })).toBeNull(); + expect(configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: 'false', PRIVACY_INDEX_RPC_URL: 'broken' })).toBeNull(); + expect(() => configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: 'true' })).toThrow('rpc_required'); + expect(() => configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: 'yes' })).toThrow('PRIVACY_RELAYER_DIRECTORY_ENABLED_invalid'); + expect(() => configuredPrivacyRelayerDirectory({ PRIVACY_INDEX_RPC_URL: 'http://localhost' })).toThrow('privacy_rpc_url_invalid'); + for (const flag of [undefined, '', 'true']) { + const directory = configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: flag, PRIVACY_INDEX_RPC_URL: 'https://operator.example' }); + expect(directory).not.toBeNull(); + await directory.start(); + expect(post).not.toHaveBeenCalled(); + await directory.close(); + } + } finally { + stat.mockRestore(); + post.mockRestore(); + } }); test('unused directory stays idle; first bulk read publishes only public anchored observations', async () => { const s = setup(); await s.directory.start(); expect(s.rpc.head).not.toHaveBeenCalled();