Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -94,18 +94,18 @@ SYSCOIN_BLOCKBOOK_URL=
# Blank disables management. No operator keys belong in this file or service.
SYSCOIN_SENTRY_GENESIS_HASH=

# Optional public Tornado Classic ETH pool index. Disabled by default.
# Set BOTH values to opt in. The RPC must support Ethereum mainnet finalized
# Public Tornado Classic ETH pool index. Enabled by default when the RPC below
# is configured; set PRIVACY_INDEX_ENABLED=false to disable it. The RPC must support finalized
# headers, historical state and eth_getLogs from the 2019 deployment blocks.
# HTTPS only; no wallet keys or caller-supplied endpoints. First request opts a
# pool into backfill and periodic refresh; previously used pools resume on restart.
# Unused pools stay idle. See docs/privacy-index.md for lifecycle and feed trust.
PRIVACY_INDEX_ENABLED=false
PRIVACY_INDEX_ENABLED=
PRIVACY_INDEX_RPC_URL=

# Optional anonymous public relayer directory. Uses the operator-controlled
# Ethereum mainnet RPC above but is independent of PRIVACY_INDEX_ENABLED.
# Never accepts caller URLs or notes. Defaults off; first bulk read opts in.
# See docs/privacy-relayers.md before enabling; no relayers are contacted by
# installation, tests or deployments while this flag remains false.
PRIVACY_RELAYER_DIRECTORY_ENABLED=false
# Never accepts caller URLs or notes. Enabled by default with the RPC above;
# set PRIVACY_RELAYER_DIRECTORY_ENABLED=false to disable it. The first bulk read
# starts discovery; see docs/privacy-relayers.md for lifecycle and privacy.
PRIVACY_RELAYER_DIRECTORY_ENABLED=
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ should use lowercase.
- `GET /mnlist` — fresh `masternode_list` RPC passthrough
- `POST /mnsearch` — paginated/searched view over the in-memory tracker snapshot
- `POST /govlist` — active + historical governance proposals (Syscoin Core `gobject list`)
- `GET /api/privacy/v1/pools/:poolId/snapshot` — opt-in anonymous Classic ETH
- `GET /api/privacy/v1/pools/:poolId/snapshot` — anonymous Classic ETH
pool snapshots for local wallet verification; see [configuration and protocol](docs/privacy-index.md).
- `GET /api/privacy/v1/relayers` — separately opt-in anonymous Classic ETH
- `GET /api/privacy/v1/relayers` — anonymous Classic ETH
relayer discovery; see [source checks and endpoint boundaries](docs/privacy-relayers.md).

Authenticated routes (cookie + CSRF, same-site):
Expand Down
17 changes: 11 additions & 6 deletions docs/privacy-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@ These are pinned from the original

## Configuration and deployment

The feature is disabled unless `PRIVACY_INDEX_ENABLED=true` and
`PRIVACY_INDEX_RPC_URL` supplies an operator-controlled HTTPS Ethereum mainnet
RPC. URLs containing credentials or fragments are rejected. A provider API
The feature is enabled by default when `PRIVACY_INDEX_RPC_URL` supplies an
operator-controlled HTTPS Ethereum mainnet RPC. Set `PRIVACY_INDEX_ENABLED=false`
to disable it independently. With no RPC it remains disabled; explicitly setting
`PRIVACY_INDEX_ENABLED=true` without an RPC is a configuration error. No default
RPC is selected. URLs containing credentials or fragments are rejected. A provider API
token may be in its path/query; it never enters responses or application logs.
The RPC must support `finalized` headers, historical `eth_call`/`eth_getCode`,
and `eth_getLogs` from the deployment blocks. Scans start with 5,000-block
Expand All @@ -30,7 +32,8 @@ successful full ranges, the scanner doubles the range toward the original
fallback to unfinalized data or to a different provider. A provider which cannot
serve even one block of historical logs needs a suitable endpoint before enabling.

An unused pool sends no RPC. Its first request durably activates it and starts
Enabling the endpoint does not start a new pool's backfill by itself. An unused
pool sends no RPC. Its first request durably activates it and starts
a job, bounded to 16 chunks and a 45-second scheduling budget (an already-started
RPC can take another 10 seconds). A global queue runs only one job at a time;
requests for the same pool share it. Activated pools continue unfinished backfill
Expand All @@ -39,19 +42,21 @@ requiring another client request. Failures back off for 30 seconds. On restart,
only previously activated pools resume. The single scheduled timer is unreferenced;
`index.close()` cancels future work and waits for the current bounded job. The
standalone app exposes its index handle for test/server lifecycle cleanup.
Any caller with access to the endpoint can activate a pool, and the resulting
backfill and refresh consume provider quota and may incur charges.
Per-process IP limits allow 60 requests/minute and apply IPv6 subnet bucketing.
Run one writer process for this file store; multiple workers require explicit
shared locking and shared request limits before deployment.

The backend process needs write access to `data/privacy-index/`, resolved from
its working directory. Checkpoints and complete snapshots live in separate
files with checksums and are published by same-directory atomic rename. An
allowlisted per-pool activation marker preserves opt-in intent even if the first
allowlisted per-pool activation marker preserves activation even if the first
RPC fails. Existing snapshot/checkpoint filenames also identify previously used
pools during migration; unknown filenames activate nothing. They
contain only public data, can be rebuilt, and never change `sysnode.db` or its
schema. The existing deployment preserves `data/`. The feature needs no SQL
migration, but the opt-in environment values and nginx route must be configured
migration, but the Ethereum RPC and nginx route must be configured
separately; automatic deployment does not modify the host environment or nginx.
The internal `/privacy/v1` mount also supports existing proxies which strip
`/api`; clients still use the public `/api/privacy/v1` URL. See the
Expand Down
16 changes: 10 additions & 6 deletions docs/privacy-relayers.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@ balance computation and the optional direct-withdrawal path.

## Deployment

The directory is disabled by default. Opt in with
`PRIVACY_RELAYER_DIRECTORY_ENABLED=true` and the existing
`PRIVACY_INDEX_RPC_URL` set to an operator-controlled HTTPS Ethereum mainnet RPC.
`PRIVACY_INDEX_ENABLED` does not enable the directory and need not be enabled
for this route. The RPC needs historical `eth_getLogs`, `eth_getStorageAt`,
The directory is enabled by default when `PRIVACY_INDEX_RPC_URL` is set to an
operator-controlled HTTPS Ethereum mainnet RPC. Set
`PRIVACY_RELAYER_DIRECTORY_ENABLED=false` to disable it independently. With no
RPC it remains disabled; explicitly setting the directory flag to `true` without
an RPC is a configuration error. `PRIVACY_INDEX_ENABLED` does not control the
directory. The RPC needs historical `eth_getLogs`, `eth_getStorageAt`,
`eth_getCode` and `eth_call`, plus finalized block support. There is no fallback
to `latest` or a caller-provided provider. Existing deployments stripping `/api`
before proxying are supported by the internal `/privacy/v1` mount; clients
Expand All @@ -21,7 +22,10 @@ proxy already forwards this namespace. Access logs should be disabled for the
public privacy namespace, including any upstream proxy/CDN, if operators do not
want to retain client IP/access metadata.

The first request activates one directory scanner. It persists only public ENS
Any caller with access to the endpoint can activate one directory scanner. The
first request can initiate RPC traffic
and relayer status checks; a configured provider may charge for those reads.
Enabling the endpoint alone does not start a new scan. The scanner persists only public ENS
registration candidates and a canonical scan checkpoint under
`data/privacy-relayers/registry-checkpoint.json`. A persisted checkpoint resumes
after restart; an unused directory sends no RPC or relayer requests. Endpoint
Expand Down
7 changes: 5 additions & 2 deletions lib/privacyIndex.js
Original file line number Diff line number Diff line change
Expand Up @@ -272,8 +272,11 @@ function createPrivacyIndex({ rpc, store = createPrivacyStore(), now = Date.now,
function configuredPrivacyIndex({ enabled = process.env.PRIVACY_INDEX_ENABLED,
rpcUrl = process.env.PRIVACY_INDEX_RPC_URL, ...options } = {}) {
if (enabled !== undefined && enabled !== '' && enabled !== 'true' && enabled !== 'false') throw new Error('PRIVACY_INDEX_ENABLED_invalid');
if (enabled !== 'true') return null;
if (!rpcUrl) throw new Error('PRIVACY_INDEX_RPC_URL_required');
if (enabled === 'false') return null;
if (!rpcUrl) {
if (enabled === 'true') throw new Error('PRIVACY_INDEX_RPC_URL_required');
return null;
}
return createPrivacyIndex({ rpc: createPrivacyRpc(rpcUrl), ...options });
}

Expand Down
7 changes: 5 additions & 2 deletions lib/privacyRelayerDirectory.js
Original file line number Diff line number Diff line change
Expand Up @@ -194,8 +194,11 @@ function createPrivacyRelayerDirectory({ rpc, inspect, store, now = Date.now, se
function configuredPrivacyRelayerDirectory(env = process.env) {
const enabled = env.PRIVACY_RELAYER_DIRECTORY_ENABLED;
if (enabled !== undefined && enabled !== '' && enabled !== 'true' && enabled !== 'false') throw new Error('PRIVACY_RELAYER_DIRECTORY_ENABLED_invalid');
if (enabled !== 'true') return null;
if (!env.PRIVACY_INDEX_RPC_URL) throw new Error('privacy_relayer_rpc_required');
if (enabled === 'false') return null;
if (!env.PRIVACY_INDEX_RPC_URL) {
if (enabled === 'true') throw new Error('privacy_relayer_rpc_required');
return null;
}
const directory = createPrivacyRelayerDirectory({ rpc: createPrivacyRelayerRpc(env.PRIVACY_INDEX_RPC_URL),
inspect: createPrivacyRelayerStatus(), store: createRelayerCheckpointStore(path.resolve('data/privacy-relayers')) });
// Persisted public registry checkpoints resume; a never-used directory is idle.
Expand Down
32 changes: 31 additions & 1 deletion tests/privacyIndex.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,41 @@ describe('public privacy index lifecycle', () => {
afterEach(async () => { await fs.rm(dir, { recursive: true, force: true }); });
const setup = (dir, rpc, extra = {}) => createPrivacyIndex({ rpc, store: createPrivacyStore(dir), autoStart: false, refreshMs: 0, retryMs: 0, continueMs: 0, ...extra });

test('disabled config performs no RPC or file IO and opt-in validates configuration', () => {
test('default configuration enables the index only with an RPC and supports explicit opt-out', async () => {
const post = jest.spyOn(require('axios'), 'post');
expect(configuredPrivacyIndex({ enabled: '', rpcUrl: '' })).toBeNull();
expect(configuredPrivacyIndex({ enabled: 'false', rpcUrl: 'broken' })).toBeNull();
expect(() => configuredPrivacyIndex({ enabled: 'true', rpcUrl: '' })).toThrow('PRIVACY_INDEX_RPC_URL_required');
expect(() => configuredPrivacyIndex({ enabled: 'yes' })).toThrow('PRIVACY_INDEX_ENABLED_invalid');
expect(() => configuredPrivacyIndex({ enabled: 'true', rpcUrl: 'http://localhost' })).toThrow('privacy_rpc_url_invalid');
expect(() => configuredPrivacyIndex({ enabled: '', rpcUrl: 'http://localhost' })).toThrow('privacy_rpc_url_invalid');
for (const enabled of ['', 'true']) {
const index = configuredPrivacyIndex({ enabled, rpcUrl: 'https://operator.example', store: createPrivacyStore(dir) });
expect(index).not.toBeNull();
await index.start();
expect(post).not.toHaveBeenCalled();
await index.close();
}
post.mockRestore();
});

test('omitting the flag uses the configured process RPC by default', async () => {
const previousFlag = process.env.PRIVACY_INDEX_ENABLED;
const previousRpc = process.env.PRIVACY_INDEX_RPC_URL;
delete process.env.PRIVACY_INDEX_ENABLED;
delete process.env.PRIVACY_INDEX_RPC_URL;
try {
expect(configuredPrivacyIndex()).toBeNull();
process.env.PRIVACY_INDEX_RPC_URL = 'https://operator.example';
const index = configuredPrivacyIndex({ store: createPrivacyStore(dir) });
expect(index).not.toBeNull();
await index.close();
} finally {
if (previousFlag === undefined) delete process.env.PRIVACY_INDEX_ENABLED;
else process.env.PRIVACY_INDEX_ENABLED = previousFlag;
if (previousRpc === undefined) delete process.env.PRIVACY_INDEX_RPC_URL;
else process.env.PRIVACY_INDEX_RPC_URL = previousRpc;
}
});

test('returns progress first and only publishes complete pinned snapshots', async () => {
Expand Down
26 changes: 23 additions & 3 deletions tests/privacyRelayerDirectory.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,29 @@ function setup({ saved, candidates = [candidate], end = DEPLOYMENT_BLOCK + 2, ov
}
async function ready(s) { await s.directory.read(); await s.directory.idle(); return s.directory.read(); }

test('directory is opt-in and configuration requires a separately enabled flag', () => {
expect(configuredPrivacyRelayerDirectory({ PRIVACY_INDEX_RPC_URL: 'https://operator.example' })).toBeNull();
expect(() => configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: 'true' })).toThrow('rpc_required');
test('directory defaults on with an RPC, remains idle until requested, and supports opt-out', async () => {
const post = jest.spyOn(require('axios'), 'post');
// A developer's real data directory may contain an active checkpoint; this
// test must exercise a fresh install and never resume that scanner.
const stat = jest.spyOn(fs, 'stat').mockRejectedValue(Object.assign(new Error('not found'), { code: 'ENOENT' }));
try {
expect(configuredPrivacyRelayerDirectory({})).toBeNull();
expect(configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: '', PRIVACY_INDEX_RPC_URL: '' })).toBeNull();
expect(configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: 'false', PRIVACY_INDEX_RPC_URL: 'broken' })).toBeNull();
expect(() => configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: 'true' })).toThrow('rpc_required');
expect(() => configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: 'yes' })).toThrow('PRIVACY_RELAYER_DIRECTORY_ENABLED_invalid');
expect(() => configuredPrivacyRelayerDirectory({ PRIVACY_INDEX_RPC_URL: 'http://localhost' })).toThrow('privacy_rpc_url_invalid');
for (const flag of [undefined, '', 'true']) {
const directory = configuredPrivacyRelayerDirectory({ PRIVACY_RELAYER_DIRECTORY_ENABLED: flag, PRIVACY_INDEX_RPC_URL: 'https://operator.example' });
expect(directory).not.toBeNull();
await directory.start();
expect(post).not.toHaveBeenCalled();
await directory.close();
}
} finally {
stat.mockRestore();
post.mockRestore();
}
});
test('unused directory stays idle; first bulk read publishes only public anchored observations', async () => {
const s = setup(); await s.directory.start(); expect(s.rpc.head).not.toHaveBeenCalled();
Expand Down
Loading