test: unify gateway integration architecture

_context/wiki/index.md

@@ -25,7 +25,7 @@ then follow only the links that are relevant.
 
 - **Repo**: `contextforge-data-plane` — the Rust ContextForge external dataplane.
 - **Core invariant**: the ContextForge external dataplane is pure routing logic. No IAM, UI, or metrics storage.
-- **Protocol target**: the ContextForge external dataplane supports MCP `2026-07-28` and `2025-11-25` over Streamable HTTP. Both use stateless request handling; `initialize` remains supported but does not establish external-dataplane session state. Legacy SSE paths are being removed from the external dataplane.
+- **Protocol target**: new external-dataplane behavior targets MCP `2026-07-28` over Streamable HTTP with `server/discover` and per-request client metadata. The remaining `2025-11-25`/`initialize` paths are temporary compatibility coverage, not an expansion surface. Legacy SSE is outside the external dataplane.
 - **Status convention**: project, architecture, routing, and operations pages
   describe the current implementation. The page under **Upcoming** describes
   the tentative ContextForge 2.0 target and migration roadmap.

_context/wiki/routing.md

@@ -12,7 +12,7 @@ The control plane builds and publishes the routing tables to Redis; the dataplan
 
 ## Routing table shape
 
-```rust
+```text
 VirtualHost { backends: HashMap<String, BackendMCPGateway>,
               tools: HashMap<String, ServiceRoute>,
               resources: HashMap<String, ServiceRoute>,

_context/wiki/testing.md

@@ -13,26 +13,50 @@ CI runs these on every change; run them locally before pushing:
 
 ```bash
 cargo fmt --all --check
-cargo clippy --locked --workspace --all-targets -- -D warnings
-cargo nextest run --locked --workspace
+cargo clippy --locked --workspace --all-targets --all-features -- -D warnings
+cargo nextest run --locked --workspace --all-features
+cargo shear --check-test-targets --deny-warnings --locked
 ```
 
 Use `cargo test` when nextest is unavailable. For wiki changes, also run `mdbook build _context/wiki` and `mdbook test _context/wiki`.
 
-Protocol-sensitive tests and fixtures must cover MCP `2026-07-28` and `2025-11-25` in all four incoming-client/selected-backend combinations. The same-version paths are supported directly; the two cross-version paths are best effort and tests must cover both successful adaptation and explicit failure for semantics that cannot be translated without state. Every case must prove request independence: no required `Mcp-Session-Id`, session affinity, or retained backend transport. Keep `2026-07-28` coverage for `server/discover` and required per-request client metadata, and retain `initialize` coverage as a stateless compatibility request. SSE remains outside the external-dataplane contract.
+New protocol-sensitive tests target MCP `2026-07-28`, connect through
+`server/discover`, and send the required per-request client metadata. A small
+`compatibility` module retains the active `2025-11-25`/`initialize` cases until
+that production compatibility surface is removed in a dedicated change; do not
+add new behavior to that lane. Every case must remain request-independent, with
+no required `Mcp-Session-Id`, session affinity, or retained backend transport.
+SSE remains outside the external-dataplane contract.
 
 ## In-Repo Integration Tests
 
-`crates/contextforge-data-plane-lib/tests/` exercises the gateway against in-process mock MCP backends (shared helpers live in `tests/support/`):
+`crates/contextforge-data-plane-lib/tests/gateway.rs` is the single library
+integration target. It exercises the public gateway API against in-process MCP
+backends without recompiling a shared support tree for every feature file.
 
-| Test file | Covers |
+| Area | Covers |
 | --- | --- |
-| `gateway_list_tools.rs` | List fanout, prefixing, and merged output. |
-| `gateway_prompts.rs` | Prompt listing and prefixed `get_prompt` routing. |
-| `gateway_resource_templates.rs` | Template fanout with prefixed names and URI templates, plus `read_resource` round-trips. |
-| `gateway_plugins.rs` | Request-scoped parameter-header validation/forwarding, CPEX pre/post tool hooks around `call_tool` and stream events, and prompt hooks around `get_prompt`. |
+| `gateway/{tools,prompts,resources,subscriptions}.rs` | Active routed operations and exact routing failures. |
+| `gateway/plugins.rs` | Gateway-owned CPEX ordering, mutation, denial, progress, and prompt seams using deterministic recording plugins. Concrete plugin behavior stays in each plugin crate. |
+| `gateway/harness/` | Authentication, modern and compatibility clients, in-memory configuration, concrete mock backends, and owned server fixtures. |
+| `gateway/future_contracts/` | Deferred fanout, pagination, TLS, completions, subscriptions, and cancellation contracts. |
 
-These run in `cargo nextest run` with no Docker dependencies.
+`TestServer` binds `127.0.0.1:0` before spawning, uses cooperative
+cancellation, and has a `Drop` fallback. `GatewayFixture` owns the gateway and
+all backend servers. Tests should request the minimum topology: one virtual host
+and one backend by default, with extra backends declared explicitly by the case.
+
+The workspace currently keeps 13 ignored tests: 11 library future contracts
+and two real-process Redis/binary E2E tests. Ignored tests are not dead tests:
+keep them compiling, keep their intended assertions, give each a concrete
+blocker reason, and list them with:
+
+```bash
+cargo nextest list --locked --workspace --all-features --run-ignored only
+```
+
+The two binary E2E tests and `tests/conformance/` remain separate infrastructure
+boundaries. Active in-process tests run with no Docker or Redis dependency.
 
 Parameter-header integration tests verify that calls without a published tool
 schema skip local `Mcp-Param-*` validation and still reach the backend. Unit and

crates/contextforge-data-plane-lib/Cargo.toml

@@ -59,8 +59,6 @@ with_tools = []
 [dev-dependencies]
 opentelemetry_sdk.workspace = true
 cpex.workspace = true
-openport.workspace = true
-cpex-secrets-detection.workspace = true
 test-log = "0.2.20"
 axum-server = { version = "0.8.0", features = ["tls-rustls"] }
 

crates/contextforge-data-plane-lib/src/lib.rs

@@ -70,7 +70,7 @@ pub struct Gateway {
 impl Gateway {
     pub async fn run_gateway(self) -> Result<()> {
         let config = self.config.clone();
-        let app = self.build_app().await?;
+        let app = self.into_router().await?;
 
         let mut handlers = vec![];
 
@@ -98,7 +98,11 @@ impl Gateway {
         Ok(())
     }
 
-    async fn build_app(self) -> Result<axum::Router> {
+    /// Builds the complete gateway application without binding a network listener.
+    ///
+    /// This supports embedding the dataplane in an existing Axum server and lets
+    /// callers bind listeners before starting the service.
+    pub async fn into_router(self) -> Result<axum::Router> {
         let Gateway { config, session_manager, user_config_store_type, plugin_runtime, authorization_service } = self;
         let user_config_store = match user_config_store_type {
             UserConfigStoreType::Redis => Arc::new(get_config_store(&config).await?),
@@ -212,7 +216,7 @@ mod tests {
             .with_session_manager(Arc::new(LocalSessionManager::default()))
             .with_user_config_store_type(UserConfigStoreType::Test(Arc::new(UnusedConfigStore)))
             .build()
-            .build_app()
+            .into_router()
             .await
             .expect("Expecting this to work");
         let request = Request::builder()

crates/contextforge-data-plane-lib/tests/gateway.rs

@@ -0,0 +1,19 @@
+#[path = "gateway/harness/mod.rs"]
+mod harness;
+
+#[path = "gateway/compatibility.rs"]
+mod compatibility;
+#[path = "gateway/completions.rs"]
+mod completions;
+#[path = "gateway/future_contracts/mod.rs"]
+mod future_contracts;
+#[path = "gateway/plugins.rs"]
+mod plugins;
+#[path = "gateway/prompts.rs"]
+mod prompts;
+#[path = "gateway/resources.rs"]
+mod resources;
+#[path = "gateway/subscriptions.rs"]
+mod subscriptions;
+#[path = "gateway/tools.rs"]
+mod tools;

crates/contextforge-data-plane-lib/tests/gateway/compatibility.rs

@@ -0,0 +1,81 @@
+use std::sync::Arc;
+
+use contextforge_data_plane_cpex::CpexRuntimeRegistry;
+use contextforge_data_plane_lib::Result;
+use rmcp::model::{CallToolRequestParams, ErrorCode, ProtocolVersion, ReadResourceRequestParams, ResourceContents};
+
+use crate::harness::{
+    TEST_USER_ID, connect_client_with_protocol, connect_modern_client, create_client, modern_client_info,
+    start_counter_gateway, start_gateway, start_legacy_counter_gateway,
+};
+
+const DECREMENT_TOOL: &str = "00000000-0000-0000-0000-000000000001-decrement";
+const MEMO_RESOURCE: &str = "00000000-0000-0000-0000-000000000001-memo://insights";
+const EXPECTED_MEMO: &str = "Business Intelligence Memo\n\nAnalysis has revealed 5 key insights ...";
+
+#[tokio::test]
+async fn plaintext_call_prefixed_backend_tools_modern_legacy() -> Result<()> {
+    let fixture = start_legacy_counter_gateway(TEST_USER_ID).await?;
+    assert_decrement(fixture.gateway_url, ProtocolVersion::V_2026_07_28).await
+}
+
+#[tokio::test]
+async fn plaintext_call_prefixed_backend_tools_legacy_modern() -> Result<()> {
+    let fixture = start_counter_gateway(TEST_USER_ID).await?;
+    assert_decrement(fixture.gateway_url, ProtocolVersion::V_2025_11_25).await
+}
+
+#[tokio::test]
+async fn plaintext_call_prefixed_read_resources_modern_legacy() -> Result<()> {
+    let fixture = start_legacy_counter_gateway(TEST_USER_ID).await?;
+    assert_memo_read(fixture.gateway_url, ProtocolVersion::V_2026_07_28).await
+}
+
+#[tokio::test]
+async fn plaintext_call_prefixed_read_resources_legacy_modern() -> Result<()> {
+    let fixture = start_counter_gateway(TEST_USER_ID).await?;
+    assert_memo_read(fixture.gateway_url, ProtocolVersion::V_2025_11_25).await
+}
+
+#[tokio::test]
+async fn legacy_tool_call_without_published_schema_reaches_backend() {
+    let gateway = start_gateway(TEST_USER_ID, false, Arc::new(CpexRuntimeRegistry::default())).await;
+    let service = gateway.connect_legacy(TEST_USER_ID).await;
+    let error = service.call_tool(CallToolRequestParams::new("missing_schema_tool")).await.unwrap_err();
+    let rmcp::service::ServiceError::McpError(error) = error else {
+        panic!("expected backend MCP error, got {error:?}");
+    };
+    assert_eq!(ErrorCode::METHOD_NOT_FOUND, error.code);
+    let backend_calls = gateway.backend_state.calls.lock().expect("backend calls lock poisoned");
+    assert_eq!("missing_schema_tool", backend_calls[0].tool_name);
+}
+
+async fn assert_decrement(gateway_url: String, client_protocol: ProtocolVersion) -> Result<()> {
+    let service = connect_for_protocol(gateway_url, client_protocol).await?;
+    let result = service.call_tool(CallToolRequestParams::new(DECREMENT_TOOL)).await?;
+    let text = result.content.first().and_then(|content| content.as_text()).expect("text tool result");
+    assert_eq!("-1", text.text);
+    Ok(())
+}
+
+async fn assert_memo_read(gateway_url: String, client_protocol: ProtocolVersion) -> Result<()> {
+    let service = connect_for_protocol(gateway_url, client_protocol).await?;
+    let response = service.read_resource(ReadResourceRequestParams::new(MEMO_RESOURCE)).await?;
+    let Some(ResourceContents::TextResourceContents { text, .. }) = response.contents.first() else {
+        panic!("expected one text resource, got {:?}", response.contents);
+    };
+    assert_eq!(EXPECTED_MEMO, text);
+    Ok(())
+}
+
+async fn connect_for_protocol(
+    gateway_url: String,
+    client_protocol: ProtocolVersion,
+) -> Result<rmcp::service::RunningService<rmcp::RoleClient, rmcp::model::InitializeRequestParams>> {
+    let client = create_client(TEST_USER_ID);
+    if client_protocol == ProtocolVersion::V_2026_07_28 {
+        Ok(connect_modern_client(&gateway_url, client, modern_client_info()).await)
+    } else {
+        connect_client_with_protocol(gateway_url, client, client_protocol).await
+    }
+}

crates/contextforge-data-plane-lib/tests/gateway/completions.rs

@@ -0,0 +1,15 @@
+use contextforge_data_plane_lib::Result;
+
+use crate::harness::{TEST_USER_ID, connect_modern_client, create_client, modern_client_info, start_counter_gateway};
+
+#[tokio::test]
+async fn plaintext_complete_for_unrouted_reference_errors() -> Result<()> {
+    let fixture = start_counter_gateway(TEST_USER_ID).await?;
+    let service = connect_modern_client(&fixture.gateway_url, create_client(TEST_USER_ID), modern_client_info()).await;
+
+    service
+        .complete_prompt_simple("unrouted_prompt", "message", "h")
+        .await
+        .expect_err("an unrouted completion reference must fail");
+    Ok(())
+}

crates/contextforge-data-plane-lib/tests/gateway/future_contracts/completions.rs

@@ -0,0 +1,41 @@
+use contextforge_data_plane_lib::Result;
+
+use crate::harness::{TEST_USER_ID, connect_modern_client, create_client, modern_client_info, start_counter_gateway};
+
+#[tokio::test]
+#[ignore = "blocked on federated prompt-completion capability and routing"]
+async fn plaintext_completes_prompt_argument_through_prefixed_backend() -> Result<()> {
+    let fixture = start_counter_gateway(TEST_USER_ID).await?;
+    let service = connect_modern_client(&fixture.gateway_url, create_client(TEST_USER_ID), modern_client_info()).await;
+
+    assert!(
+        service.peer_info().and_then(|info| info.capabilities.completions.clone()).is_some(),
+        "gateway must advertise completions before clients can call completion/complete"
+    );
+    let prompts = service.list_prompts(None).await?;
+    let prompt_name = prompts
+        .prompts
+        .iter()
+        .find(|prompt| prompt.name.ends_with("-example_prompt"))
+        .map(|prompt| prompt.name.clone())
+        .ok_or("expected a federated example_prompt")?;
+    let values = service.complete_prompt_simple(prompt_name, "message", "h").await?;
+    assert!(values.contains(&"hello".to_owned()), "expected backend prompt completions, got {values:?}");
+    Ok(())
+}
+
+#[tokio::test]
+#[ignore = "blocked on federated resource-completion capability and routing"]
+async fn plaintext_completes_resource_argument_through_prefixed_backend() -> Result<()> {
+    let fixture = start_counter_gateway(TEST_USER_ID).await?;
+    let service = connect_modern_client(&fixture.gateway_url, create_client(TEST_USER_ID), modern_client_info()).await;
+
+    let resources = service.list_resources(None).await?;
+    let uri = resources.resources.first().ok_or("expected at least one federated resource")?.uri.clone();
+    let values = service.complete_resource_simple(uri, "path", "").await?;
+    assert!(
+        values.first().is_some_and(|value| !value.starts_with("backend-")),
+        "expected a stripped backend URI, got {values:?}"
+    );
+    Ok(())
+}

crates/contextforge-data-plane-lib/tests/gateway/future_contracts/mod.rs

@@ -0,0 +1,7 @@
+mod completions;
+mod pagination;
+mod plugins;
+mod prompts;
+mod resource_templates;
+mod subscriptions;
+mod transport;