Skip to content

ci(deploy): deploy dtq-dev-767-base to instance 8633 on dev-20 - #1717

Merged
Kasinhou merged 7 commits into
dtq-dev-767-basefrom
dtq-767-deploy-8633
Nov 10, 2026
Merged

Kasinhou merged 7 commits into
dtq-dev-767-basefrom
dtq-767-deploy-8633

Conversation

@jr-rk

@jr-rk jr-rk commented Nov 9, 2026 •

Copy link
Copy Markdown

Problem description

dtq-dev-767-base still carries the stale dev-5 deploy setup (instances 5/8 on dspace-dep-1), so it cannot deploy, redeploy, erase or import its own instance, dspace8633 on dev-20. A manual "Deploy DSpace" run from this branch targets dev-5 instead.

This PR ports the instance deploy setup from dtq-dev-9-base (#1383 and follow-ups #1386, #1388, #1391, #1452), retargeted from 8603 on dev-6 to 8633 on dev-20.

  • deploy.yml: taken from dtq-dev-9-base (f6cf600b81) with 8603→8633, 6603→6633, dev-6→dev-20. Jobs probe (read-only report of the target machine), deploy-8633 and import-8633 on dspace-${DEV_MACHINE}-dep-1; inputs DEV_MACHINE, DUMP_PATH, ASSETSTORE_PATH, PROBE_ONLY.
  • start.sh: honours INSTANCE_OVERLAY, so /opt/dspace-envs/<instance> (with ports: !override) replaces the host ports the repo compose files publish. Sets COMPOSE_PROJECT_NAME for the CLI container.
  • docker.yml: deploy after a push uses this branch's deploy.yml and targets 8633, instead of dtq-dev's deploy.yml and instance 5.
  • import-weekly.yml → redeploy-weekly.yml: the old file erased and imported instance 8. The new one only redeploys 8633 (IMPORT and ERASE_DB both false).
  • playwright-tests.yml: HOME_URL points at http://dev-20.pc:8633/repository/.

Analysis

Differences from 9-base, each with a reason:

  • v7 specifics kept: chown -R dspace:dspace after copying the assetstore and -o dataquest on create-administrator (the v7 image runs as dspace; 9-base dropped both for v9).
  • DUMP_PATH defaults to /opt/dspace-envs/8633/dump/dspace_dev5_dump_26.07.16_anon.sql: the same dev-5 dump 9-base imports, anonymized (accounts, e-mails, tokens) before it was placed on dev-20. IMPORT defaults to true on manual runs, as in 9-base; push and the weekly redeploy pass false.
  • Workflow inputs reach the shell through env, not ${{ inputs.* }} inside run: (Copilot finding, shell injection).
  • The dump is loaded with psql -v ON_ERROR_STOP=1: the anonymized dump restores with zero errors, so any error means a partial restore that the later steps would not catch.
  • After import the dev user is recreated (dspace user --add, as in start.sh): the import replaces the whole DB and the REST tests after import log in as that user.
  • The probe verdict first checks docker compose config -q (the greps hid compose errors) and also flags 6+ digit ports, because on this branch the repo files build ports from INSTANCE (808${INSTANCE} → 8088633).

Server side (dev-20, done outside this PR, mirroring /opt/dspace-envs/8603 on dev-6): /opt/dspace-envs/8633 holds .env, docker-compose.yml and docker-compose-rest.yml with ports: !override, dump/ and assetstore/. A dry-run docker compose config with the repo + server files resolves to ports 4633/6633/10633/11633/21633/22633/23633, subnet 172.2.33.0/24 and the dtq-dev-767-base images the running 8633 uses.

To wipe 8633 and load the dump: run Deploy DSpace on dtq-dev-767-base with ERASE_DB and IMPORT checked.

Problems

  • The runner dspace-dev-20-dep-1 is not verified (no permission to list runners). Suggested first run: Deploy DSpace with PROBE_ONLY checked; it changes nothing.
  • Merging triggers a deploy to 8633 right away (docker.yml → deploy.yml, no erase, no import).

Checks done locally: the workflows parse as YAML, every run: block passes bash -n, the probe verdict was exercised with a stubbed docker (compose error / inherited port / OK). actionlint was not run; the workflow has not run on dev-20 yet.

Sync verification

If en.json5 or cs.json5 translation files were updated:

  • Run yarn run sync-i18n -t src/assets/i18n/cs.json5 -i to synchronize messages, and changes are included in this PR.

Manual Testing (if applicable)

Copilot review

  • Requested review from Copilot

jr-rk added 2 commits November 9, 2026 15:24
Port the instance deploy setup from dtq-dev-9-base (#1383 and follow-ups)
to dtq-dev-767-base, retargeted from 8603 to 8633.

- deploy.yml: replace the stale dev-5 jobs (instances 5/8) with probe,
  deploy-8633 and import-8633 on dspace-${DEV_MACHINE}-dep-1. Adds
  DEV_MACHINE, DUMP_PATH, ASSETSTORE_PATH and PROBE_ONLY inputs.
- start.sh: honour INSTANCE_OVERLAY so the per-instance overlay's
  `ports: !override` replaces the fixed host ports; set
  COMPOSE_PROJECT_NAME for the CLI container.
- docker.yml: deploy after a push via this branch's deploy.yml to 8633
  instead of dtq-dev's deploy.yml to instance 5.
- Replace import-weekly.yml (erase + import of instance 8) with the
  non-destructive redeploy-weekly.yml.
- playwright-tests.yml: point HOME_URL at dev-6.pc:8633.

Differences from 9-base: IMPORT defaults to false on manual runs and
DUMP_PATH has no default, since 8633 should start empty. The v7-specific
`chown dspace:dspace` on the assetstore and `-o dataquest` on
create-administrator are kept.
@jr-rk jr-rk changed the title ci(deploy): deploy dtq-dev-767-base to instance 8633 on dev-6 ci(deploy): deploy dtq-dev-767-base to instance 8633 on dev-20 Nov 9, 2026
@jr-rk jr-rk self-assigned this Nov 9, 2026
@jr-rk
jr-rk requested a lite review from Copilot November 9, 2026 14:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The change still permits shell-command injection through direct workflow input interpolation, can mask SQL restore failures, and may run validation against a different host than the one deployed.

4 open issues
What changed in this PR

This PR retargets dtq-dev-767-base deployment automation from the stale dev-5 setup to instance 8633 on dev-20, adding instance overlays, probing, import controls, and non-destructive redeployment.

Changes:

  • Adds configurable deployment, machine, dump, assetstore, and probe inputs.
  • Updates start.sh for instance-specific Compose overlays and CLI networking.
  • Retargets Docker-triggered deployment and Playwright tests to instance 8633.
  • Replaces the destructive weekly import workflow with scheduled redeployment.
  • Unresolved findings require changes: .github/workflows/deploy.yml contains a critical DUMP_PATH shell-injection risk (lines 90 and 213; 3 votes), plus moderate issues involving SQL import error handling (line 228; 4 votes), missing dev-user recreation after imports (line 224; 1 vote), failed Compose resolution being treated as successful probing (line 121; 1 vote), inconsistent wildcard handling (line 162; 1 vote), and REST tests ignoring DEV_MACHINE (lines 328 and 357; 2 votes). .github/workflows/playwright-tests.yml also hard-codes dev-20.pc despite the machine override (line 22; moderate; 2 votes).

This pull request requires changes.

File Description
build-scripts/​run/​start.sh Applies per-instance Compose overlays and sets the Compose project name for CLI containers.
.github/​workflows/​redeploy-weekly.yml Adds a non-destructive weekly redeployment workflow.
.github/​workflows/​playwright-tests.yml Points Playwright tests at the instance 8633 deployment URL; the fixed host does not honor deployment-machine overrides.
.github/​workflows/​import-weekly.yml Removes the obsolete weekly database import and erase workflow.
.github/​workflows/​docker.yml Routes pushed-image deployment to instance 8633 on dev-20.
.github/​workflows/​deploy.yml Adds probing, deployment, database import, assetstore restoration, and post-deployment checks for instance 8633; contains the unresolved security, import, probing, wildcard, and host-targeting findings listed above.

Review effort: Lite

Comment thread .github/workflows/deploy.yml Resolved
Comment thread .github/workflows/deploy.yml Resolved
Comment thread .github/workflows/deploy.yml Resolved
Comment thread .github/workflows/playwright-tests.yml Resolved
Restore deploy.yml verbatim from dtq-dev-9-base (f6cf600) with only
8603->8633, 6603->6633, dev-6->dev-20 substituted, plus the two v7
specifics: chown of the assetstore and `-o dataquest` on
create-administrator. Restore template wording in start.sh comment.
DUMP_PATH and DEV_MACHINE were interpolated straight into `run:` scripts,
so a crafted workflow_dispatch value could inject shell commands on the
self-hosted runner. Expose them as env vars and reference those instead.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Approval is blocked by active critical and moderate issues in the deployment workflows, including unsafe dump-path interpolation, destructive default import behavior, incomplete restore error handling, inconsistent target selection, and hardcoded test hosts.

6 open issues
1 previously missed issue

In code that hasn't changed since last review

Medium severity Remove unsupported wildcard instance handling

.github/​workflows/​deploy.yml:167

This condition still advertises INSTANCE: '*' support, but the job hardcodes INSTANCE: '8633' while CONFIG_PATH and the import job use inputs.INSTANCE. A reusable caller passing '*' therefore probes an invalid /opt/dspace-envs/* path and mixes target names. Since this workflow only supports 8633, remove the wildcard branch (or implement fan-out consistently).

Review effort: Lite

Comment thread .github/workflows/deploy.yml Resolved
Comment thread .github/workflows/deploy.yml Resolved
The dev-5 dump is now anonymized (accounts, e-mails, tokens) before it is
placed on dev-20 as /opt/dspace-envs/8633/dump/dspace_dev5_dump_26.07.16_anon.sql.
…rrors

- psql loads the dump with ON_ERROR_STOP=1: the anonymized dump restores
  with zero errors, so any error now means a partial restore that later
  steps would not catch.
- The import replaces the whole DB, dropping the dev user start.sh
  created; recreate it, since the REST tests after import log in as it.
- The probe verdict printed "OK" when `compose config` failed and only
  looked for the dtq-dev-9-base fixed ports. Check `config -q` first and
  also flag the 6+ digit ports this branch builds from INSTANCE.
@Kasinhou
Kasinhou merged commit e75409f into dtq-dev-767-base Nov 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants