From 38a3a073005d019146f016f88712c5eafeb0a0b0 Mon Sep 17 00:00:00 2001 From: Sean Dietrich Date: Tue, 22 Sep 2026 15:29:33 -0700 Subject: [PATCH 1/2] Every WP-CLI subcommand, run through `wp` rather than through PHP CI exercised three of the thirteen subcommands and asserted nothing about any of them beyond an exit status, plus one hand-rolled check that `import --mode=replace` refuses without `--yes`. The other ten were covered by nobody: not by the integration suite, which calls PHP and so never sees WP-CLI, and not by CI. That gap is not theoretical. What PHPUnit cannot reach is everything WP-CLI owns -- whether a method is registered as a subcommand at all, whether its `## OPTIONS` block parses, whether a flag the docblock advertises is accepted, and what the process exits with. A command can be correct PHP and unreachable from a shell, which is the only place anybody runs it. tests/cli/commands.sh runs all thirteen and makes 25 assertions about what they print and what they exit with. It runs the same way by hand and in CI: BFW_WP="ddev wp" bash tests/cli/commands.sh Two behaviours are pinned because they are the ones that would be read wrong. `check` exits 0 whether or not the address was blocked -- the status reports whether the query ran, so a deploy script treating non-zero as "blocked" would be wrong on every run. And a destructive command must exit NON-zero when it refuses: WP_CLI::confirm() calls halt(0) on EOF, so the failure being guarded against is a prompt nobody answers followed by an exit 0 that reads as success. Agreeing to a destructive command is opt-in through BFW_CLI_DESTRUCTIVE=1. CI opts in, because its site goes away with the runner; a hand run does not, so pointing this at a site you care about does not empty its block list. The refusals are still asserted either way, which is the half that has actually broken. Nothing here reads a file back from the script's own filesystem. `ddev wp` runs inside a container, so a path this script can see is not a path the command can write to -- the first draft used mktemp and failed on exactly that. Found while running them by hand, and left alone pending a decision: `sources` lists presets while `refresh-sources` re-fetches the lists a rule references. Two different things wearing one word. The README now says so; renaming is free while nothing is released, and is not mine to do unasked. `refresh-sources` was also missing from the README's command list entirely, which is fixed. --- .github/workflows/ci.yml | 28 +++--- README.md | 16 ++++ tests/cli/commands.sh | 188 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 217 insertions(+), 15 deletions(-) create mode 100755 tests/cli/commands.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index da6605f..8ad650e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -134,12 +134,6 @@ jobs: - name: Integration tests run: vendor/bin/phpunit -c phpunit-integration.xml.dist - - name: WP-CLI smoke test - run: | - wp basic-firewall status --path=/tmp/wp --allow-root - wp basic-firewall rebuild --path=/tmp/wp --allow-root - wp basic-firewall rules --path=/tmp/wp --allow-root - - name: End-to-end tests over real HTTP run: | # The only tests here that exercise the mu-plugin, the compiled file @@ -149,15 +143,19 @@ jobs: sleep 3 BFW_E2E_URL=http://127.0.0.1:8080 vendor/bin/phpunit -c phpunit-e2e.xml.dist - - name: A destructive import must refuse without --yes - run: | - wp basic-firewall export --file=/tmp/fw.yml --path=/tmp/wp --allow-root - if wp basic-firewall import /tmp/fw.yml --mode=replace \ - --path=/tmp/wp --allow-root < /dev/null; then - echo "::error::import --mode=replace exited 0 without --yes" - exit 1 - fi - echo "Refused, as it must." + # Every subcommand, through `wp` rather than through PHP. What PHPUnit + # cannot reach is everything WP-CLI owns: whether a method is registered + # as a subcommand at all, whether its `## OPTIONS` block parses, and what + # the process exits with -- including that a destructive command exits + # NON-zero when it refuses, which is the one that has actually broken. + # + # Last in the job, and destructive: it replaces the rule set and empties + # the block list on a site that is thrown away with the runner. + - name: WP-CLI commands + env: + BFW_WP: wp --path=/tmp/wp --allow-root + BFW_CLI_DESTRUCTIVE: 1 + run: bash tests/cli/commands.sh package: name: Build and verify the release zip diff --git a/README.md b/README.md index e022cf7..ebae15e 100644 --- a/README.md +++ b/README.md @@ -976,6 +976,7 @@ wp basic-firewall status # what the firewall is doing right now wp basic-firewall rules # rules in evaluation order wp basic-firewall rebuild # recompile the configuration wp basic-firewall sources # available presets +wp basic-firewall refresh-sources # re-fetch the lists rules reference wp basic-firewall check IP # is this address blocked? wp basic-firewall block IP # block it @@ -994,6 +995,21 @@ status reports whether the *query* ran. Read the `blocked` field. Anything destructive refuses to run without `--yes` and exits non-zero when it refuses. It will never exit 0 having done nothing. +`sources` lists **presets**; `refresh-sources` re-fetches the **lists a rule +references**. Two different things, and the names do not currently say so. + +All of it is covered by `tests/cli/commands.sh`, which runs `wp` for real rather +than calling PHP, and so is the only place the subcommand registration and the +exit statuses are checked: + +```bash +BFW_WP="ddev wp" bash tests/cli/commands.sh +``` + +Agreeing to a destructive command is opt-in through `BFW_CLI_DESTRUCTIVE=1`, so +running it against a site you care about does not empty its block list. CI opts +in, because its site goes away with the runner. + ## Multisite Per-site. Each site gets its own settings, its own block list, its own counters diff --git a/tests/cli/commands.sh b/tests/cli/commands.sh new file mode 100755 index 0000000..36af85b --- /dev/null +++ b/tests/cli/commands.sh @@ -0,0 +1,188 @@ +#!/usr/bin/env bash +# +# Every WP-CLI subcommand, exercised against a real site. +# +# The integration suite calls PHP; this calls `wp`. Between the two sits +# everything WP-CLI itself owns and PHPUnit cannot reach: whether a method is +# registered as a subcommand at all, whether its `## OPTIONS` block parses, +# whether a flag the docblock advertises is accepted, and what the process +# exits with. A command can be perfectly correct PHP and still be unreachable +# from a shell, which is the only place anybody actually runs it. +# +# Exit status is the point of several of these. `check` exits 0 whether or not +# the address was blocked, because the status reports whether the query ran -- +# a deploy script that reads a non-zero exit as "blocked" would be wrong. And +# anything destructive must exit NON-zero when it refuses, because the failure +# this guards against is a command that prints a prompt nobody answers and then +# exits 0 having changed nothing. +# +# Usage: +# +# BFW_WP="ddev wp" bash tests/cli/commands.sh +# BFW_WP="wp --path=/tmp/wp --allow-root" bash tests/cli/commands.sh +# +# Nothing here reads a file back from the script's own filesystem, and that is +# deliberate rather than tidy: `ddev wp` runs inside a container, so a path this +# script can see is not a path the command can write to. Assertions are made on +# what the command printed, and the one file involved is named in BFW_TMP as the +# *command* sees it. +# +# Destructive coverage -- `clear-blocked --yes` and `import --mode=replace +# --yes` -- is opt-in through BFW_CLI_DESTRUCTIVE=1, and CI opts in because its +# site is thrown away afterwards. Without it the refusals are still asserted, +# which is the half that has actually broken before; what is skipped is only +# the confirmation that agreeing works. Running this against a site you care +# about therefore does not empty its block list. + +set -uo pipefail + +WP=${BFW_WP:-wp} +DESTRUCTIVE=${BFW_CLI_DESTRUCTIVE:-0} + +# A writable directory as the `wp` process sees it, which under ddev is inside +# the container and not this shell's /tmp. +TMP=${BFW_TMP:-/tmp} +readonly EXPORT_FILE="$TMP/bfw-cli-commands.yml" + +# RFC 5737 reserves this range for documentation, so it can never be an address +# this site legitimately has an opinion about. +readonly ADDRESS=203.0.113.242 + +passed=0 +failed=0 + +cleanup() { + # Best effort, and quiet: the address is ours, and leaving it blocked would + # make a second run of this script start from a different place than the + # first. + $WP basic-firewall unblock "$ADDRESS" >/dev/null 2>&1 + + printf '\n%s\n' "----------------------------------------------------------------" + + if [ "$failed" -gt 0 ]; then + printf 'wp-cli: %d passed, %d FAILED\n' "$passed" "$failed" + exit 1 + fi + + printf 'wp-cli: %d passed\n' "$passed" +} +trap cleanup EXIT + +# Record a result. Keeps the reason next to the assertion rather than leaving a +# bare "failed" for somebody to go and reproduce. +report() { + local outcome=$1 description=$2 detail=${3:-} + + if [ "$outcome" = pass ]; then + passed=$(( passed + 1 )) + printf ' ok %s\n' "$description" + + return + fi + + failed=$(( failed + 1 )) + printf ' FAIL %s\n' "$description" + [ -n "$detail" ] && printf ' %s\n' "$detail" +} + +# Run a subcommand, asserting the exit status and optionally what it printed. +# +# Arguments: +# Stdin is closed, because a command that waits for an answer nobody is there +# to give must fail rather than hang the run. +expect() { + local want=$1 description=$2 needle=$3 + shift 3 + + local output status + output=$($WP basic-firewall "$@" &1) + status=$? + + if [ "$want" = ok ] && [ "$status" -ne 0 ]; then + report fail "$description" "expected exit 0, got $status: $(printf '%s' "$output" | head -1)" + + return + fi + + if [ "$want" = fail ] && [ "$status" -eq 0 ]; then + report fail "$description" 'expected a non-zero exit, got 0' + + return + fi + + if [ -n "$needle" ] && ! printf '%s' "$output" | grep -qF "$needle"; then + report fail "$description" "output did not contain '$needle': $(printf '%s' "$output" | head -1)" + + return + fi + + report pass "$description" +} + +printf 'wp-cli: reporting commands\n' + +expect ok 'status reports the library version' 'Library' status +expect ok 'status renders as json' '"Enabled"' status --format=json +expect ok 'rules lists in evaluation order' '' rules +expect ok 'sources lists the presets' '' sources +expect ok 'blocked lists the block list' '' blocked +expect ok 'rebuild recompiles' 'Compiled to' rebuild + +printf '\nwp-cli: the block list\n' + +expect ok 'check answers for an address that is not blocked' 'false' check "$ADDRESS" +expect ok 'block takes a duration and a reason' 'is blocked' \ + block "$ADDRESS" --duration=600 --reason='CLI coverage' +expect ok 'check sees the block it just made' 'true' check "$ADDRESS" +expect ok 'check reports the reason back' 'CLI coverage' check "$ADDRESS" +expect ok 'blocked lists the blocked address' "$ADDRESS" blocked +expect ok 'unblock releases it' 'unblocked' unblock "$ADDRESS" +expect ok 'check sees it released' 'false' check "$ADDRESS" + +# The exit status reports whether the query ran, not what the answer was. A +# deploy script reading non-zero as "this address is blocked" would be wrong on +# every run, so this is pinned rather than left to read like an accident. +expect ok 'check exits 0 for an address that is not blocked' '' check 198.51.100.7 + +printf '\nwp-cli: references and lists\n' + +expect fail 'find-reference refuses a reference that does not exist' 'was not found' \ + find-reference BFW-NO-SUCH-REFERENCE +expect ok 'refresh-sources previews without fetching' '' refresh-sources --dry-run + +printf '\nwp-cli: export and import\n' + +expect ok 'export writes to standard output' 'Basic Firewall configuration export' export + +# The invariant the whole export path exists to hold, asserted on the document +# itself rather than on a unit test's idea of it: a credential is removed, and +# the document says which, so a receiving site knows what it has to supply. +expect ok 'the export names the credentials it stripped' 'REMOVED from this export' export + +expect ok 'export writes a document to a file' 'Written to' export --file="$EXPORT_FILE" +expect fail 'export refuses a rule that does not exist' 'no rule with the identifier' \ + export --rule=bfw-no-such-rule + +expect ok 'import previews without changing anything' 'nothing was changed' \ + import "$EXPORT_FILE" --dry-run + +printf '\nwp-cli: destructive commands refuse without --yes\n' + +# The one that has actually gone wrong. WP_CLI::confirm() calls halt(0) when it +# reads EOF, so with stdin closed the old behaviour was a prompt nobody answered +# and an exit 0 that a deploy script reads as success. +expect fail 'import --mode=replace refuses without --yes' 'Nothing was changed' \ + import "$EXPORT_FILE" --mode=replace +expect fail 'clear-blocked refuses without --yes' 'Nothing was changed' clear-blocked + +if [ "$DESTRUCTIVE" != 1 ]; then + printf ' skip agreeing to a destructive command (set BFW_CLI_DESTRUCTIVE=1)\n' + + exit 0 +fi + +printf '\nwp-cli: destructive commands proceed with --yes\n' + +expect ok 'import --mode=replace proceeds with --yes' 'Imported and recompiled' \ + import "$EXPORT_FILE" --mode=replace --yes +expect ok 'clear-blocked proceeds with --yes' 'Released' clear-blocked --yes From 6314851c232fd06f6f85f6b8e70414f0590ce5b8 Mon Sep 17 00:00:00 2001 From: Sean Dietrich Date: Tue, 22 Sep 2026 22:01:00 -0700 Subject: [PATCH 2/2] Move CI from GitHub Actions to CircleCI (#3) Same nine jobs, same matrix, same refusal to let anything fail softly. Kanopi runs CircleCI everywhere else, and a plugin whose CI lives somewhere other than the rest of the estate is one nobody watches. Not built on kanopi/ci-tools. The orb's `composer` job is what the site repos use for PHPCS and PHPStan, and it is a good fit there -- but it carries Pantheon deploy machinery and the kanopi-code context, and this repo needs a PHP matrix and a MySQL service that the orb job does not express. The config is self-contained instead, in the same house style: versions as anchors at the top, cimg images, caches keyed on the lock. Two things the images forced. `cimg/php` carries Composer, mysqli, pdo_mysql, unzip and dockerize, and carries neither the mysql client nor WP-CLI. So WP-CLI is installed from the phar, and the root password is set in the service's own environment rather than ALTERed afterwards -- which removes the mysql client the old workflow needed, and is a simplification the move paid for rather than a compromise it forced. Better than what it replaces, in three small ways. `dockerize -wait` instead of `sleep 3` before the database and before the built-in web server, which is the fragility behind the one e2e flake seen on nightly. JUnit output from all three suites into `store_test_results`, so a failure is a named test in the UI rather than a log to scroll. And the zip as a build artifact on every run, which is the first time it has been retrievable without running the build yourself. Verified by running the jobs, not by validating the YAML. `circleci config validate` passes and `config process` expands all nine job names, but that proves only that the file parses. Each job was then run in the image it will actually use, with MySQL in a shared network namespace so it answers on 127.0.0.1 exactly as CircleCI arranges it: static cimg/php:8.1 platform-reqs, phpcs, phpstan green unit cimg/php:8.2 55 tests, junit written green unit cimg/php:8.4 55 tests, junit written green integration cimg/php:8.1 122 + 6 e2e + 25 wp-cli green package cimg/php:8.3 scoped zip, installs with no composer on PATH green That run also caught the kind of thing validation cannot: `wp core download` takes an optional positional download URL, so a stray flag becomes the URL and the command fails with "Version option is not available for URL downloads". Mine was in the throwaway script rather than in the config, but the config would have failed the same way. The repo still has to be enabled in the CircleCI app before any of this runs, and GitHub's required checks still name the old jobs. --- .circleci/config.yml | 261 +++++++++++++++++++++++++++++++++++++++ .github/workflows/ci.yml | 196 ----------------------------- README.md | 24 ++++ 3 files changed, 285 insertions(+), 196 deletions(-) create mode 100644 .circleci/config.yml delete mode 100644 .github/workflows/ci.yml diff --git a/.circleci/config.yml b/.circleci/config.yml new file mode 100644 index 0000000..8a3bf91 --- /dev/null +++ b/.circleci/config.yml @@ -0,0 +1,261 @@ +# Nothing here is allowed to fail softly. A security plugin whose test matrix is +# advisory is a security plugin nobody has tested. + +# 8.1 is the floor, declared in composer.json, in the plugin header, in +# readme.txt, and enforced at runtime by the guard in basic-firewall.php. It is +# also what config.platform resolves the lock against, so it is the version the +# shipped dependency tree is chosen for. +PHP_FLOOR: &PHP_FLOOR "8.1" + +# Not the floor. PHP-Scoper needs 8.3, and it is the only thing that runs on +# this PHP: the tree it rewrites is installed from the plugin's own lock. The +# zip is identical whichever PHP builds it. +PHP_TOOLS: &PHP_TOOLS "8.3" + +MYSQL_IMAGE: &MYSQL_IMAGE "cimg/mysql:8.0" + +version: 2.1 + +commands: + composer_install: + description: Install dependencies, cached against the lock file. + steps: + - restore_cache: + name: Restore Composer cache + keys: + # The PHP version is in the key because the cache holds vendor/ + # itself. The lock resolves identically on every version -- that is + # what config.platform buys -- but a cache shared across images is + # a bet on that staying true. + - composer-v1-{{ checksum "composer.lock" }}-{{ arch }}-<< parameters.php >> + - run: + name: Install dependencies + command: composer install --no-interaction --no-progress + - save_cache: + name: Save Composer cache + key: composer-v1-{{ checksum "composer.lock" }}-{{ arch }}-<< parameters.php >> + paths: + - vendor + parameters: + php: + type: string + + install_wp_cli: + description: WP-CLI, which the cimg/php images do not carry. + steps: + - run: + name: Install WP-CLI + command: | + curl -fsSL -o /tmp/wp-cli.phar \ + https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar + chmod +x /tmp/wp-cli.phar + sudo mv /tmp/wp-cli.phar /usr/local/bin/wp + wp --info --allow-root + +jobs: + static: + docker: + - image: cimg/php:<< parameters.php >> + parameters: + php: + type: string + default: *PHP_FLOOR + steps: + - checkout + - composer_install: + php: << parameters.php >> + + # The plugin claims PHP 8.1, so nothing it ships may require more. The + # lock is resolved against that floor by config.platform; this asserts it, + # because the failure mode is a fatal on a customer's site rather than + # anything a test would catch. + - run: + name: The shipped tree must install on the declared floor + command: composer check-platform-reqs --no-dev + - run: + name: WordPress Coding Standards + command: composer lint + - run: + name: PHPStan + command: composer analyse + + unit: + docker: + - image: cimg/php:<< parameters.php >> + parameters: + php: + type: string + steps: + - checkout + - composer_install: + php: << parameters.php >> + - run: + name: Unit tests + command: | + mkdir -p /tmp/test-results + vendor/bin/phpunit --testsuite unit \ + --log-junit /tmp/test-results/unit.xml + - store_test_results: + path: /tmp/test-results + + integration: + docker: + - image: cimg/php:<< parameters.php >> + environment: + # The plugin is symlinked into a throwaway site, and PHP resolves + # __DIR__ through the symlink back to the checkout, so the test + # bootstrap cannot find WordPress by walking up. It is told where the + # site is instead. + BASIC_FIREWALL_WP_ROOT: /tmp/wp + - image: *MYSQL_IMAGE + environment: + MYSQL_ROOT_PASSWORD: << pipeline.parameters.db_password >> + MYSQL_DATABASE: wordpress_test + parameters: + php: + type: string + wordpress: + type: string + steps: + - checkout + - composer_install: + php: << parameters.php >> + - install_wp_cli + - run: + name: Wait for MySQL + command: dockerize -wait tcp://127.0.0.1:3306 -timeout 2m + - run: + name: Install WordPress << parameters.wordpress >> + command: | + wp core download --version=<< parameters.wordpress >> --path=/tmp/wp --allow-root + wp config create --path=/tmp/wp --dbname=wordpress_test --dbuser=root \ + --dbpass='<< pipeline.parameters.db_password >>' --dbhost=127.0.0.1 --allow-root + wp core install --path=/tmp/wp --url=http://localhost:8080 \ + --title=CI --admin_user=admin --admin_password=password \ + --admin_email=ci@example.com --skip-email --allow-root + - run: + name: Link the plugin into the site + command: | + mkdir -p /tmp/wp/wp-content/plugins + ln -s "$PWD" /tmp/wp/wp-content/plugins/basic-firewall + wp plugin activate basic-firewall --path=/tmp/wp --allow-root + - run: + name: Integration tests + command: | + mkdir -p /tmp/test-results + vendor/bin/phpunit -c phpunit-integration.xml.dist \ + --log-junit /tmp/test-results/integration.xml + + # The only tests here that exercise the mu-plugin, the compiled file on + # disk, the library's parse cache and the web server together -- which is + # where every defect this plugin has shipped so far lived. + - run: + name: Serve the site + command: php -S 127.0.0.1:8080 -t /tmp/wp + background: true + - run: + name: Wait for the site + command: dockerize -wait tcp://127.0.0.1:8080 -timeout 1m + - run: + name: End-to-end tests over real HTTP + command: | + mkdir -p /tmp/test-results + BFW_E2E_URL=http://127.0.0.1:8080 vendor/bin/phpunit -c phpunit-e2e.xml.dist \ + --log-junit /tmp/test-results/e2e.xml + + # Every subcommand, through `wp` rather than through PHP. What PHPUnit + # cannot reach is everything WP-CLI owns: whether a method is registered + # as a subcommand at all, whether its `## OPTIONS` block parses, and what + # the process exits with -- including that a destructive command exits + # NON-zero when it refuses, which is the one that has actually broken. + # + # Last in the job, and destructive: it replaces the rule set and empties + # the block list on a site that is thrown away with the container. + - run: + name: WP-CLI commands + environment: + BFW_WP: wp --path=/tmp/wp --allow-root + BFW_CLI_DESTRUCTIVE: 1 + command: bash tests/cli/commands.sh + - store_test_results: + path: /tmp/test-results + + package: + docker: + - image: cimg/php:<< parameters.php >> + parameters: + php: + type: string + default: *PHP_TOOLS + steps: + - checkout + - install_wp_cli + - run: + name: Build the zip + command: bash build/build-zip.sh + + # The point of the whole exercise: a zip that works where there is no + # Composer at all. Installed onto a clean WordPress with the Composer + # binary removed from PATH, then actually evaluated against a request. + - run: + name: Install the zip on a clean WordPress with no Composer + command: | + wp core download --path=/tmp/clean --allow-root + wp config create --path=/tmp/clean --dbname=wp --dbuser=root \ + --dbpass=root --dbhost=127.0.0.1 --skip-check --allow-root + sudo mv "$(command -v composer)" /usr/local/bin/composer.hidden + ! command -v composer + unzip -q build/dist/*.zip -d /tmp/clean/wp-content/plugins/ + test -d /tmp/clean/wp-content/plugins/basic-firewall/vendor + echo "vendor/ is present in the installed plugin." + - store_artifacts: + path: build/dist + destination: zip + +parameters: + db_password: + type: string + + # Not a secret: an ephemeral database that exists for one job and is thrown + # away with the container. It is long on purpose. CredentialHandlingTest + # searches the compiled configuration for the literal password to prove it + # never reaches disk, and skips that search -- loudly -- when the password + # is short enough for the search to prove nothing. A two-character password + # makes the assertion meaningless, which is the state the ddev site is in. + # + # A pipeline parameter rather than a YAML anchor because it is needed inside + # a command string, and an anchor cannot be referenced from one. + default: "circleci-integration-password-not-a-secret" + +workflows: + ci: + jobs: + - static + - unit: + matrix: + parameters: + # 8.1 is the floor and 8.4 is what the library is developed + # against. Testing only the floor would let the plugin rot on + # modern PHP; testing only the top would let it break on the + # version it claims. + php: ["8.1", "8.2", "8.3", "8.4"] + name: unit-php-<< matrix.php >> + - integration: + matrix: + parameters: + php: ["8.1"] + wordpress: ["6.4"] + name: integration-wp6.4-php8.1 + - integration: + matrix: + parameters: + php: ["8.3"] + wordpress: ["latest"] + name: integration-wplatest-php8.3 + - integration: + matrix: + parameters: + php: ["8.4"] + wordpress: ["nightly"] + name: integration-wpnightly-php8.4 + - package diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml deleted file mode 100644 index 8ad650e..0000000 --- a/.github/workflows/ci.yml +++ /dev/null @@ -1,196 +0,0 @@ -name: CI - -on: - push: - branches: [ main ] - pull_request: - -# Nothing here is allowed to fail softly. A security plugin whose test matrix is -# advisory is a security plugin nobody has tested. -permissions: - contents: read - -jobs: - static: - name: Coding standards and static analysis - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - uses: shivammathur/setup-php@v2 - with: - php-version: '8.1' - coverage: none - tools: composer:v2 - - - name: Install dependencies - run: composer install --no-interaction --no-progress - - # The plugin claims PHP 8.1, so nothing it ships may require more. The - # lock is resolved against that floor by config.platform; this asserts it, - # because the failure mode is a fatal on a customer's site rather than - # anything a test would catch. - - name: The shipped tree must install on the declared floor - run: composer check-platform-reqs --no-dev - - - name: WordPress Coding Standards - run: composer lint - - - name: PHPStan - run: composer analyse - - unit: - name: Unit tests, PHP ${{ matrix.php }} - runs-on: ubuntu-latest - strategy: - fail-fast: false - matrix: - # 8.1 is the floor, declared in composer.json and enforced at runtime by - # the guard in basic-firewall.php. 8.4 is what the library is developed - # against. Testing only the floor would let the plugin rot on modern PHP; - # testing only the top would let it break on the version it claims. - php: [ '8.1', '8.2', '8.3', '8.4' ] - steps: - - uses: actions/checkout@v4 - - - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ matrix.php }} - coverage: none - tools: composer:v2 - - - name: Install dependencies - run: composer install --no-interaction --no-progress - - - name: Unit tests - run: composer test:unit - - integration: - name: Integration, WP ${{ matrix.wordpress }} / PHP ${{ matrix.php }} - runs-on: ubuntu-latest - # The plugin is symlinked into a throwaway site, and PHP resolves __DIR__ - # through the symlink back to the checkout, so the test bootstrap cannot - # find WordPress by walking up. It is told where the site is instead. - env: - BASIC_FIREWALL_WP_ROOT: /tmp/wp - strategy: - fail-fast: false - matrix: - include: - # The oldest WordPress the plugin header claims, on the oldest PHP. - - { wordpress: '6.4', php: '8.1' } - # Current WordPress on current PHP. - - { wordpress: 'latest', php: '8.3' } - # Next WordPress, so a break is found before a release rather than after. - - { wordpress: 'nightly', php: '8.4' } - services: - mysql: - image: mysql:8.0 - env: - MYSQL_ROOT_PASSWORD: root - MYSQL_DATABASE: wordpress_test - ports: [ '3306:3306' ] - options: >- - --health-cmd="mysqladmin ping" - --health-interval=10s - --health-timeout=5s - --health-retries=5 - steps: - - uses: actions/checkout@v4 - - - uses: shivammathur/setup-php@v2 - with: - php-version: ${{ matrix.php }} - extensions: mysqli, pdo_mysql - coverage: none - tools: composer:v2, wp-cli - - - name: Install dependencies - run: composer install --no-interaction --no-progress - - - name: Install WordPress - env: - # A long random password so the literal-password assertion in - # CredentialHandlingTest actually runs. On a site whose DB_PASSWORD is - # "db" or "root" that search is meaningless and the test says so - # instead of passing quietly -- see assertLiteralPasswordAbsent(). - DB_PASSWORD: ${{ github.run_id }}-${{ github.run_attempt }}-integration-password - run: | - mysql -h 127.0.0.1 -u root -proot -e \ - "ALTER USER 'root'@'%' IDENTIFIED BY '${DB_PASSWORD}';" - wp core download --version=${{ matrix.wordpress }} --path=/tmp/wp --allow-root - wp config create --path=/tmp/wp --dbname=wordpress_test --dbuser=root \ - --dbpass="${DB_PASSWORD}" --dbhost=127.0.0.1 --allow-root - wp core install --path=/tmp/wp --url=http://localhost:8080 \ - --title=CI --admin_user=admin --admin_password=password \ - --admin_email=ci@example.com --skip-email --allow-root - - - name: Link the plugin into the site - run: | - mkdir -p /tmp/wp/wp-content/plugins - ln -s "$GITHUB_WORKSPACE" /tmp/wp/wp-content/plugins/basic-firewall - wp plugin activate basic-firewall --path=/tmp/wp --allow-root - - - name: Integration tests - run: vendor/bin/phpunit -c phpunit-integration.xml.dist - - - name: End-to-end tests over real HTTP - run: | - # The only tests here that exercise the mu-plugin, the compiled file - # on disk, the library's parse cache and the web server together -- - # which is where every defect this plugin has shipped so far lived. - php -S 127.0.0.1:8080 -t /tmp/wp & - sleep 3 - BFW_E2E_URL=http://127.0.0.1:8080 vendor/bin/phpunit -c phpunit-e2e.xml.dist - - # Every subcommand, through `wp` rather than through PHP. What PHPUnit - # cannot reach is everything WP-CLI owns: whether a method is registered - # as a subcommand at all, whether its `## OPTIONS` block parses, and what - # the process exits with -- including that a destructive command exits - # NON-zero when it refuses, which is the one that has actually broken. - # - # Last in the job, and destructive: it replaces the rule set and empties - # the block list on a site that is thrown away with the runner. - - name: WP-CLI commands - env: - BFW_WP: wp --path=/tmp/wp --allow-root - BFW_CLI_DESTRUCTIVE: 1 - run: bash tests/cli/commands.sh - - package: - name: Build and verify the release zip - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - # Not the plugin's floor. PHP-Scoper needs 8.3, and it is the only thing - # here that runs on this PHP: the tree it rewrites is installed from the - # plugin's own lock, which is resolved for 8.1 by config.platform. The - # zip is identical whichever PHP builds it. - - uses: shivammathur/setup-php@v2 - with: - php-version: '8.3' - coverage: none - tools: composer:v2, wp-cli - - - name: Build the zip - run: bash build/build-zip.sh - - # The point of the whole exercise: a zip that works where there is no - # Composer at all. Installed onto a clean WordPress with the Composer - # binary removed from PATH, then actually evaluated against a request. - - name: Install the zip on a clean WordPress with no Composer - run: | - wp core download --path=/tmp/clean --allow-root - wp config create --path=/tmp/clean --dbname=wp --dbuser=root \ - --dbpass=root --dbhost=127.0.0.1 --skip-check --allow-root - sudo mv "$(command -v composer)" /usr/local/bin/composer.hidden - ! command -v composer - unzip -q build/dist/*.zip -d /tmp/clean/wp-content/plugins/ - test -d /tmp/clean/wp-content/plugins/basic-firewall/vendor - echo "vendor/ is present in the installed plugin." - - - uses: actions/upload-artifact@v4 - with: - name: basic-firewall-zip - path: build/dist/*.zip diff --git a/README.md b/README.md index ebae15e..ac2f825 100644 --- a/README.md +++ b/README.md @@ -1028,6 +1028,30 @@ shows the resulting table names. There is no network-wide settings screen. Configuring 200 sites means `wp site list --field=url` and a loop. +## Continuous integration + +CircleCI, in `.circleci/config.yml`. Nine jobs, and none of them advisory: + +| Job | Runs | +|---|---| +| `static` | `check-platform-reqs --no-dev`, PHPCS, PHPStan — on 8.1, the declared floor | +| `unit-php-*` | The unit suite on 8.1, 8.2, 8.3 and 8.4 | +| `integration-*` | Integration, end-to-end over real HTTP, and the WP-CLI suite, against WP 6.4/PHP 8.1, WP latest/PHP 8.3 and WP nightly/PHP 8.4 | +| `package` | Builds the zip and installs it on a clean WordPress with the Composer binary removed from `PATH` | + +`static` runs on the floor deliberately: the lock is resolved for PHP 8.1 by +`config.platform`, and `check-platform-reqs` there is what stops a dependency +bump quietly raising the version the plugin claims to support. `package` runs on +8.3 because PHP-Scoper needs it — the builder's PHP never reaches the zip, since +the tree it rewrites comes from the plugin's own lock. + +Every job can be reproduced locally in the image CI uses: + +```bash +docker run --rm -v "$PWD":/src:ro cimg/php:8.1 bash -c \ + 'cp -a /src/. /w/ && cd /w && composer install && composer lint && composer analyse' +``` + ## Building a release ```bash