Repository navigation
Move CI from GitHub Actions to CircleCI - #3
Merged
Merged
Conversation
Same nine jobs, same matrix, same refusal to let anything fail softly.
Kanopi runs CircleCI everywhere else, and a plugin whose CI lives
somewhere other than the rest of the estate is one nobody watches.
Not built on kanopi/ci-tools. The orb's `composer` job is what the site
repos use for PHPCS and PHPStan, and it is a good fit there -- but it
carries Pantheon deploy machinery and the kanopi-code context, and this
repo needs a PHP matrix and a MySQL service that the orb job does not
express. The config is self-contained instead, in the same house style:
versions as anchors at the top, cimg images, caches keyed on the lock.
Two things the images forced. `cimg/php` carries Composer, mysqli,
pdo_mysql, unzip and dockerize, and carries neither the mysql client nor
WP-CLI. So WP-CLI is installed from the phar, and the root password is
set in the service's own environment rather than ALTERed afterwards --
which removes the mysql client the old workflow needed, and is a
simplification the move paid for rather than a compromise it forced.
Better than what it replaces, in three small ways. `dockerize -wait`
instead of `sleep 3` before the database and before the built-in web
server, which is the fragility behind the one e2e flake seen on nightly.
JUnit output from all three suites into `store_test_results`, so a
failure is a named test in the UI rather than a log to scroll. And the
zip as a build artifact on every run, which is the first time it has
been retrievable without running the build yourself.
Verified by running the jobs, not by validating the YAML. `circleci
config validate` passes and `config process` expands all nine job names,
but that proves only that the file parses. Each job was then run in the
image it will actually use, with MySQL in a shared network namespace so
it answers on 127.0.0.1 exactly as CircleCI arranges it:
static cimg/php:8.1 platform-reqs, phpcs, phpstan green
unit cimg/php:8.2 55 tests, junit written green
unit cimg/php:8.4 55 tests, junit written green
integration cimg/php:8.1 122 + 6 e2e + 25 wp-cli green
package cimg/php:8.3 scoped zip, installs with no
composer on PATH green
That run also caught the kind of thing validation cannot: `wp core
download` takes an optional positional download URL, so a stray flag
becomes the URL and the command fails with "Version option is not
available for URL downloads". Mine was in the throwaway script rather
than in the config, but the config would have failed the same way.
The repo still has to be enabled in the CircleCI app before any of this
runs, and GitHub's required checks still name the old jobs.
sean-e-dietrich
added a commit
that referenced
this pull request
Sep 23, 2026
* Every WP-CLI subcommand, run through `wp` rather than through PHP
CI exercised three of the thirteen subcommands and asserted nothing about
any of them beyond an exit status, plus one hand-rolled check that
`import --mode=replace` refuses without `--yes`. The other ten were
covered by nobody: not by the integration suite, which calls PHP and so
never sees WP-CLI, and not by CI.
That gap is not theoretical. What PHPUnit cannot reach is everything
WP-CLI owns -- whether a method is registered as a subcommand at all,
whether its `## OPTIONS` block parses, whether a flag the docblock
advertises is accepted, and what the process exits with. A command can be
correct PHP and unreachable from a shell, which is the only place anybody
runs it.
tests/cli/commands.sh runs all thirteen and makes 25 assertions about
what they print and what they exit with. It runs the same way by hand and
in CI:
BFW_WP="ddev wp" bash tests/cli/commands.sh
Two behaviours are pinned because they are the ones that would be read
wrong. `check` exits 0 whether or not the address was blocked -- the
status reports whether the query ran, so a deploy script treating
non-zero as "blocked" would be wrong on every run. And a destructive
command must exit NON-zero when it refuses: WP_CLI::confirm() calls
halt(0) on EOF, so the failure being guarded against is a prompt nobody
answers followed by an exit 0 that reads as success.
Agreeing to a destructive command is opt-in through
BFW_CLI_DESTRUCTIVE=1. CI opts in, because its site goes away with the
runner; a hand run does not, so pointing this at a site you care about
does not empty its block list. The refusals are still asserted either
way, which is the half that has actually broken.
Nothing here reads a file back from the script's own filesystem. `ddev
wp` runs inside a container, so a path this script can see is not a path
the command can write to -- the first draft used mktemp and failed on
exactly that.
Found while running them by hand, and left alone pending a decision:
`sources` lists presets while `refresh-sources` re-fetches the lists a
rule references. Two different things wearing one word. The README now
says so; renaming is free while nothing is released, and is not mine to
do unasked. `refresh-sources` was also missing from the README's command
list entirely, which is fixed.
* Move CI from GitHub Actions to CircleCI (#3)
Same nine jobs, same matrix, same refusal to let anything fail softly.
Kanopi runs CircleCI everywhere else, and a plugin whose CI lives
somewhere other than the rest of the estate is one nobody watches.
Not built on kanopi/ci-tools. The orb's `composer` job is what the site
repos use for PHPCS and PHPStan, and it is a good fit there -- but it
carries Pantheon deploy machinery and the kanopi-code context, and this
repo needs a PHP matrix and a MySQL service that the orb job does not
express. The config is self-contained instead, in the same house style:
versions as anchors at the top, cimg images, caches keyed on the lock.
Two things the images forced. `cimg/php` carries Composer, mysqli,
pdo_mysql, unzip and dockerize, and carries neither the mysql client nor
WP-CLI. So WP-CLI is installed from the phar, and the root password is
set in the service's own environment rather than ALTERed afterwards --
which removes the mysql client the old workflow needed, and is a
simplification the move paid for rather than a compromise it forced.
Better than what it replaces, in three small ways. `dockerize -wait`
instead of `sleep 3` before the database and before the built-in web
server, which is the fragility behind the one e2e flake seen on nightly.
JUnit output from all three suites into `store_test_results`, so a
failure is a named test in the UI rather than a log to scroll. And the
zip as a build artifact on every run, which is the first time it has
been retrievable without running the build yourself.
Verified by running the jobs, not by validating the YAML. `circleci
config validate` passes and `config process` expands all nine job names,
but that proves only that the file parses. Each job was then run in the
image it will actually use, with MySQL in a shared network namespace so
it answers on 127.0.0.1 exactly as CircleCI arranges it:
static cimg/php:8.1 platform-reqs, phpcs, phpstan green
unit cimg/php:8.2 55 tests, junit written green
unit cimg/php:8.4 55 tests, junit written green
integration cimg/php:8.1 122 + 6 e2e + 25 wp-cli green
package cimg/php:8.3 scoped zip, installs with no
composer on PATH green
That run also caught the kind of thing validation cannot: `wp core
download` takes an optional positional download URL, so a stray flag
becomes the URL and the command fails with "Version option is not
available for URL downloads". Mine was in the throwaway script rather
than in the config, but the config would have failed the same way.
The repo still has to be enabled in the CircleCI app before any of this
runs, and GitHub's required checks still name the old jobs.
Merged
1 task done
Closed
5 tasks
sean-e-dietrich
added a commit
that referenced
this pull request
Sep 30, 2026
… README: CI jobs HttpEvaluationTest installs a recorder mu-plugin for basic_firewall_decision and checks what the README promises: allowed is announced at plugins_loaded on either path; on the mu-plugin path blocked and challenged are announced at shutdown, enforced; on the wp-config.php path a refusal never reaches WordPress; a log-mode match is announced as blocked and not enforced.
8 tasks done
sean-e-dietrich
added a commit
that referenced
this pull request
Sep 30, 2026
…usal, live Redis and APCu (#43) (#57) * Uninstall: remove runtime.json from a pre-existing private directory (#43) * CI: live Redis and APCu in the integration job, and a multisite cell (#43) Redis runs as a service container; ext-redis and ext-apcu are installed with pecl on the WP latest/PHP 8.3, multisite and nightly/PHP 8.4 cells, with apc.enable_cli=1. The other cells keep running the tests that need the extensions absent. Tests read BASIC_FIREWALL_TEST_REDIS, BASIC_FIREWALL_TEST_APCU and BASIC_FIREWALL_MULTISITE through Test_Services, and fail rather than skip when a declared service is missing. BlockListWriteTest gains a live Redis backend case and a round trip checked on the server, and builds a fresh block list per backend: the cached storage meant every provider case had been testing one backend. CacheBackendTest writes to APCu from the compiled file and clears it. * Multisite tests: the early path, the network-wide loader, Site Health and per-site uninstall (#43) MultisiteTest runs on the CI network cell: the bootstrap stepping aside on a real network's wp-config.php constants, the mu-loader surviving a subsite deactivating (network-active, or active on another site) and going with the last site or a network deactivation, Site Health's snippet warning and healthy state on a network, and uninstall removing every site's options and private directory. Single-site assertions that failed on a network now step aside there: subdirectory layouts filtered through home, the early path's evaluation-check reports, and the bootstrap check (inverted on a network). The wp-content fallback uninstall test uses the site's own suffix on a network and checks a foreign one is left. * Test the runner's refusal of an unanswered early verdict over HTTP (#36, #43) RunnerRefusalTest serves real WordPress under PHP's built-in server behind a router that stands in for an early path which records a challenge, redirect or block and returns without answering it. Each is refused with a 503 and Retry-After, logged, saved as the last anomaly and raised by Site Health; a solved challenge and a request with no verdict are served. The e2e raw-socket request now sends the port in its Host header: a network answers only the domain it was installed on and redirected the bare host before the firewall loaded. * Test decision actions over HTTP with an mu-plugin listener (#3, #43); README: CI jobs HttpEvaluationTest installs a recorder mu-plugin for basic_firewall_decision and checks what the README promises: allowed is announced at plugins_loaded on either path; on the mu-plugin path blocked and challenged are announced at shutdown, enforced; on the wp-config.php path a refusal never reaches WordPress; a log-mode match is announced as blocked and not enforced. * CI: a password-protected Redis, so the live Redis tests authenticate (#43) With #56 merged, RedisPasswordTest's live-server test reads BASIC_FIREWALL_TEST_REDIS and BASIC_FIREWALL_TEST_REDIS_PASSWORD. The CI Redis now requires a password and the extension cells export both; the block-list and uninstall Redis tests hand it to the plugin through the settings, as a site would.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces
.github/workflows/ci.ymlwith.circleci/config.yml. Eleven jobs — the nine the workflow had, plus PHP 8.5 on both unit and integration — and none of them advisory.Stacked on #2 — the config runs
tests/cli/commands.sh, which that PR adds. Base iscli-command-coverageso the diff here is only the CI change. Merge #2 first.Why not
kanopi/ci-toolsThe orb's
composerjob is what the site repos use for PHPCS and PHPStan, and it's a good fit there. It carries Pantheon deploy machinery and thekanopi-codecontext, and this repo needs a PHP matrix and a MySQL service the orb job doesn't express. Self-contained instead, in the same house style: versions as anchors at the top,cimgimages, caches keyed on the lock. Happy to switch the two static jobs onto the orb if you'd rather have the consistency — I couldn't introspect it without an API token.What the images forced
cimg/phpcarries Composer, mysqli, pdo_mysql, unzip and dockerize. It carries neither themysqlclient nor WP-CLI. So WP-CLI is installed from the phar, and the root password is set in the MySQL service's own environment rather thanALTERed afterwards — which removes themysqlclient the old workflow needed. That's a simplification the move paid for, not a compromise it forced.Three things better than what it replaces
dockerize -waitinstead ofsleep 3, before both the database and the built-in web server. That sleep is the fragility behind the one e2e flake we saw on nightly.store_test_results, so a failure is a named test in the CircleCI UI rather than a log to scroll.Verified by running the jobs, not by validating the YAML
circleci config validatepasses andconfig processexpands all nine job names, but that only proves the file parses. Each job was then run in the image it will actually use, with MySQL in a shared network namespace so it answers on127.0.0.1exactly as CircleCI arranges it:staticcimg/php:8.1unitcimg/php:8.2unitcimg/php:8.4integrationcimg/php:8.1unitcimg/php:8.5integrationcimg/php:8.5packagecimg/php:8.3composeronPATH— greenThat run also caught the kind of thing validation cannot:
wp core downloadtakes an optional positional download URL, so a stray flag becomes the URL and the command dies with "Version option is not available for URL downloads". Mine was in the throwaway harness rather than in the config, but the config would have failed identically.PHP 8.5
kanopi/firewallhas supported 8.1 through 8.5 since 2.31.0 and the plugin was testing a subset of it. 8.5 is in the unit matrix, and gets a full integration job paired with the existing nightly one rather than replacing it — the two differ only in PHP, so a failure on 8.5 that passes on 8.4 says "PHP 8.5" and not "WordPress trunk moved". Integration is where every defect this plugin has shipped has lived, which is why the newest PHP gets a run of it rather than unit coverage alone. Easy to drop that fourth row if you'd rather trim CI minutes.Two deprecations surface on 8.5 and neither is ours —
react/promiseandphp-cli-tools, both inside WP-CLI's own phar. Nothing from the plugin, and the unit suite (which unlike the integration suite runs withfailOnWarning) is clean.staticstays on 8.1 andpackageon 8.3. Neither is about breadth: static runs on the floor because that is wherecheck-platform-reqsmeans something, and package runs on 8.3 because that is PHP-Scoper's floor.Before this can go green
Two things outside the repo, both yours:
basic-firewall-wp). Until then this PR has no CI at all — GitHub Actions is deleted in it.static,unit-php-8.1…unit-php-8.5,integration-wp6.4-php8.1,integration-wplatest-php8.3,integration-wpnightly-php8.4,integration-wpnightly-php8.5,package.