Skip to content

Wait for the background join to end before looking for the root password it leaves - #1193

Merged
vharseko merged 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:fix/docker-test-join-password-race
Oct 14, 2026
Merged

vharseko merged 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:fix/docker-test-join-password-race

Conversation

@vharseko

Copy link
Copy Markdown
Member

Problem

Since #1184, build-docker and build-docker-alpine fail every time, on master too (run 38390334148) and on every PR (e.g. #1190, run 38382364790):

=== [2608 s] pre-roles srs volume
##[error]the root password is left in /dev/shm/opendj-join.4444.YKR3cc of dj-1

The phase "pre-roles srs volume" (.github/scripts/docker-test-replication.sh) recreates dj-1 on its kept volume, which has neither .initialize-pending nor .rejoin-pending. So run.sh writes the health marker at once and starts join.sh in the background. The test waits for "already a member of the replication topology" and for healthy, which both hold at once, and runs the password check straight away. At that point join.sh is still in joined() (cleanup_departed, publish_state, repair_replication_servers) and keeps its password file in /dev/shm. Its EXIT trap removes the file only when it ends. The dj-1 log in CI does not even show "the health check may probe it" yet.

The image itself is fine: the file goes away when the join ends. The test checks too early. It slipped through because only the role phases, not the whole script, ran locally in round 4 of #1184.

Fix

  • bootstrap_ended <container>: true once no bootstrap/join.sh or bootstrap/replicate.sh process is left in the container. It looks for them in /proc/*/cmdline; the bracket in [j]oin keeps the pattern from matching the sh -c that greps for it.
  • The password check waits up to 300 s for it on each container. A join that never ends still fails the test, by timeout.

Testing

  • bash -n passes.
  • The helper was run against a stand-in bootstrap/join.sh / bootstrap/replicate.sh (sleep 8) in ubuntu:24.04, alpine:3.20 and the OpenDJ image: "running" while the script runs, "ended" after it. The image's PID 1 is /opt/opendj/run.sh, which does not match.
  • The whole Docker replication test was not run locally. The build-docker jobs of this PR are the check.

…ord it leaves

The pre-roles srs phase (OpenIdentityPlatform#1184) recreates dj-1 on a volume that reports itself
healthy at once, and the password check right after it found the join's
/dev/shm password file while join.sh was still in joined(). Wait until no
bootstrap script runs in the container before checking.
@vharseko vharseko added bug docker CI tests Test suites: fixing, enabling, un-disabling labels Oct 14, 2026
@vharseko
vharseko requested a review from maximthomas October 14, 2026 07:25

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: The wait sits on exactly the check that went red, and it cannot hang the job.

  • bootstrap_ended greps /proc/[0-9]*/cmdline for bootstrap/[j]oin\.sh and bootstrap/[r]eplicate\.sh; the brackets keep the sh -c running the grep from matching itself (.github/scripts/docker-test-replication.sh:183).
  • wait_until 300 ... bootstrap_ended $c is bounded, so a join that never ends still fails the test by timeout (.github/scripts/docker-test-replication.sh:782).

@maximthomas
maximthomas self-requested a review October 14, 2026 07:43

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vharseko
vharseko merged commit a887d5f into OpenIdentityPlatform:master Oct 14, 2026
24 checks passed
@vharseko
vharseko deleted the fix/docker-test-join-password-race branch October 14, 2026 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug CI docker tests Test suites: fixing, enabling, un-disabling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants