fix: keep distributed layer-slice HC on active tier

[edoardolegnaro]

Hi, I was tring to run DS4 on two nodes with 4 l40s gpus each.
The distributed deployment wasn't working, so I've asked GPT5.6 Sol to fix it and identified this minimal changes that were needed.
Below is output of codex after completion:

Summary

• select the embedding or first local layer GPU tier before distributed decode and batched-prefill input writes
• bind the final batched hidden-state row to the GPU tier active after the local layer loop
• keep the change isolated to distributed multi-GPU layer-slice evaluation

Errors fixed

1. active_tier could still point at the preceding request's output tier when a new distributed hop wrote token embeddings or a remote hidden state.
2. Batched prefill captured its output source tier before processing local layers, so a slice spanning multiple GPU tiers could attach the final hidden state to its input tier.
3. The output head could consequently consume stale hidden state and sample EOS immediately on later requests.

Reproduction

Tested with DeepSeek V4 Flash Q4 imatrix on a two-node CUDA pipeline. The coordinator owned layers 0:21; the worker owned 22:output and used multiple L40S GPUs.

Against one long-lived ds4-server process:

request 1: correct response
request 2: finish_reason=stop, completion_tokens=0, empty content
request 3: finish_reason=stop, completion_tokens=0, empty content

Restarting the server restored only the next first response.

Verification

After the fix, repeated and unrelated requests through the same server remained valid:

request 1: "The capital of France is Paris."
request 2: "2 plus 2 equals 4."
request 3: non-empty response explaining why the sky is blue

Consecutive non-streaming and streaming requests also completed through the OpenAI-compatible endpoint at a 1,048,576-token context.

Build and regression checks:

git diff --check
make -j8 tests/test_engine_mgpu_placement tests/test_gpu_model_cache
./tests/test_engine_mgpu_placement
./tests/test_gpu_model_cache
make -j8 ds4 ds4-server CUDA_ARCH=native

Results:

test_engine_mgpu_placement: 98/98 checks passed
test_gpu_model_cache PASS (devs=4)
ds4 and ds4-server CUDA builds completed successfully

The sequential-request regression requires a sliced multi-node CUDA route and model artifacts, so no source-only test was added.

make test was also attempted. The CUDA build node did not have the default ds4flash.gguf test path, and the idle L40S validation node did not have /usr/local/cuda/bin/nvcc. The targeted CPU build, CUDA build, placement test, GPU cache test, and live distributed regression above completed successfully.