diff --git a/AGENTS.md b/AGENTS.md index 80265b2c..29203c32 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,9 +15,16 @@ These are not preferences. A change that breaks one of them is a bug, whatever i the embedded framework would compile the site against a different version of Hyde than the project declares. This is the single most important guarantee in the codebase. 3. **No external PHP in Portable mode.** `RuntimeManager` is the only thing that resolves a - PHP binary, and it never looks at `PATH`. If you need to run PHP, ask it. -4. **No Composer execution during a Portable build.** Composer is invoked in exactly one - place: `hyde new --composer`. + PHP binary, and it never looks at `PATH`. If you need to run PHP, ask it. That includes + `hyde php` and `hyde composer`. In a released executable that is always the embedded + runtime; in a source checkout, which embeds none, it is the PHP process already + running the code, and never one found on the search path. +4. **Composer is never invoked implicitly.** Nothing about building or serving a project + runs it. It runs when the user asks for it and nowhere else: `hyde composer`, and + `hyde new --composer`. Both prefer the Composer bundled in the executable, and they + differ where none is bundled: `hyde new --composer` falls back to the host's, since + it needs *a* Composer to create a project, while `hyde composer` fails — it means + "the Composer Hyde supplies", and in a source checkout the developer has their own. 5. **No mixing of dependency graphs.** The embedded `vendor/` and a project's `vendor/` never share a process. Composer projects are dispatched into a separate process. 6. **Detection and dispatch run before the autoloader.** The `app/Launcher` classes are @@ -33,10 +40,10 @@ These are not preferences. A change that breaks one of them is a bug, whatever i | `hyde` | The console entry point. Detection and dispatch happen here, first. | | `app/Launcher/` | The project model, runtime management and dispatch. Plain PHP, no framework. | | `app/Foundation/` | Overrides that let the framework boot out of a read-only executable. | -| `app/Commands/` | The commands the executable owns: `info`, `new`, `serve`, `self-update`. | +| `app/Commands/` | The commands the executable owns: `info`, `new`, `serve`, `self-update`, and the bundled programs `php` and `composer`. | | `app/Support/` | Small helpers with no framework dependencies. | -| `bin/` | The build scripts. `build-native.sh` and `build-native.ps1` drive static-php-cli; `build-phar.php` assembles the executable. | -| `build/runtime.json` | The single build configuration: pinned PHP version and the extension set, with a reason for each. | +| `bin/` | The build scripts. `build-native.sh` and `build-native.ps1` drive static-php-cli; `build-phar.php` assembles the executable; `lib/composer-patches.php` is what the bundled Composer is patched with, and why. | +| `build/runtime.json` | The single build configuration: pinned PHP version, the extension set with a reason for each, and the pinned Composer release with its checksum. | | `tests/System/` | Runtime acceptance in POSIX shell and PowerShell, for hosts with no PHP. | ## Testing @@ -82,5 +89,4 @@ framework requirement or to a test that proves it necessary. ## Things that are deliberately not implemented - `hyde eject`, and Portable to Composer conversion. -- Bundling Composer inside the executable. - Hybrid autoloading, or loading Composer addons into a Portable project. diff --git a/CLAUDE.md b/CLAUDE.md index 89f47629..adb77630 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,9 +15,16 @@ These are not preferences. A change that breaks one of them is a bug, whatever i the embedded framework would compile the site against a different version of Hyde than the project declares. This is the single most important guarantee in the codebase. 3. **No external PHP in Portable mode.** `RuntimeManager` is the only thing that resolves a - PHP binary, and it never looks at `PATH`. If you need to run PHP, ask it. -4. **No Composer execution during a Portable build.** Composer is invoked in exactly one - place: `hyde new --composer`. + PHP binary, and it never looks at `PATH`. If you need to run PHP, ask it. That includes + `hyde php` and `hyde composer`. In a released executable that is always the embedded + runtime; in a source checkout, which embeds none, it is the PHP process already + running the code, and never one found on the search path. +4. **Composer is never invoked implicitly.** Nothing about building or serving a project + runs it. It runs when the user asks for it and nowhere else: `hyde composer`, and + `hyde new --composer`. Both prefer the Composer bundled in the executable, and they + differ where none is bundled: `hyde new --composer` falls back to the host's, since + it needs *a* Composer to create a project, while `hyde composer` fails — it means + "the Composer Hyde supplies", and in a source checkout the developer has their own. 5. **No mixing of dependency graphs.** The embedded `vendor/` and a project's `vendor/` never share a process. Composer projects are dispatched into a separate process. 6. **Detection and dispatch run before the autoloader.** The `app/Launcher` classes are @@ -33,10 +40,10 @@ These are not preferences. A change that breaks one of them is a bug, whatever i | `hyde` | The console entry point. Detection and dispatch happen here, first. | | `app/Launcher/` | The project model, runtime management and dispatch. Plain PHP, no framework. | | `app/Foundation/` | Overrides that let the framework boot out of a read-only executable. | -| `app/Commands/` | The commands the executable owns: `info`, `new`, `serve`, `self-update`. | +| `app/Commands/` | The commands the executable owns: `info`, `new`, `serve`, `self-update`, and the bundled programs `php` and `composer`. | | `app/Support/` | Small helpers with no framework dependencies. | -| `bin/` | The build scripts. `build-native.sh` and `build-native.ps1` drive static-php-cli; `build-phar.php` assembles the executable. | -| `build/runtime.json` | The single build configuration: pinned PHP version and the extension set, with a reason for each. | +| `bin/` | The build scripts. `build-native.sh` and `build-native.ps1` drive static-php-cli; `build-phar.php` assembles the executable; `lib/composer-patches.php` is what the bundled Composer is patched with, and why. | +| `build/runtime.json` | The single build configuration: pinned PHP version, the extension set with a reason for each, and the pinned Composer release with its checksum. | | `tests/System/` | Runtime acceptance in POSIX shell and PowerShell, for hosts with no PHP. | ## Testing @@ -82,7 +89,6 @@ framework requirement or to a test that proves it necessary. ## Things that are deliberately not implemented - `hyde eject`, and Portable to Composer conversion. -- Bundling Composer inside the executable. - Hybrid autoloading, or loading Composer addons into a Portable project. diff --git a/README.md b/README.md index dcda0339..e7934965 100644 --- a/README.md +++ b/README.md @@ -11,10 +11,12 @@ The HydePHP CLI is a single-file executable for the static site generator HydePHP. -It carries its own PHP runtime and its own copy of the framework, so you can build a site on a -machine with **no PHP and no Composer installed**. Point it at a directory of Markdown files -and it will build a site; point it at a full HydePHP Composer project and it will run that -project through its own dependencies. +It carries its own PHP runtime, its own copy of the framework, and its own Composer, so you can +build a site on a machine with **no PHP and no Composer installed**. Point it at a directory of +Markdown files and it will build a site; point it at a full HydePHP Composer project and it +will run that project through its own dependencies. + +The runtime it carries is not kept to itself: `hyde php` and `hyde composer` hand it to you. ## The two kinds of project @@ -106,7 +108,7 @@ hyde info # Create a new site hyde new my-site # asks which kind you want hyde new my-site --portable # content only, nothing to install -hyde new my-site --composer # a full Composer project (requires Composer) +hyde new my-site --composer # a full Composer project, created with the bundled Composer # Build a site using source files in the working directory hyde build @@ -115,6 +117,34 @@ hyde build hyde serve ``` +### The bundled runtime + +The PHP and the Composer inside the executable are available as commands of their own, so a +machine that has neither still has both: + +```bash +hyde php -v # the bundled PHP CLI +hyde php script.php +hyde php -r 'echo PHP_VERSION;' + +hyde composer install # the bundled Composer, on the bundled PHP +hyde composer require hyde/framework +``` + +Arguments and exit statuses pass through untouched. This is *Hyde's* PHP rather than a general +distribution: its extensions are the ones Hyde needs, listed with their reasons in +[`build/runtime.json`](build/runtime.json), so a script that needs something outside that set +will say so. + +`hyde composer` is answered before the project is even looked at, which is deliberate: a +Composer project with a missing `vendor/` is the one state the CLI refuses to build, and +`hyde composer install` is what repairs it. + +Both commands mean *the programs Hyde supplies*, so neither falls back to one installed on +the machine. `hyde composer self-update` is refused for the same reason: the bundled +Composer is versioned with the executable and verified on every run, so `hyde self-update` +is what gets you a newer one. + ## Resources ### Changelog diff --git a/app/Commands/ComposerCommand.php b/app/Commands/ComposerCommand.php new file mode 100644 index 00000000..1e341aa2 --- /dev/null +++ b/app/Commands/ComposerCommand.php @@ -0,0 +1,29 @@ +line('Runtime: '.($runtime->hasEmbeddedRuntime() ? sprintf('PHP %s bundled for %s', $runtime->manifest()['version'], $runtime->manifest()['platform']) : 'none bundled (running from a source checkout)')); + $this->line('Composer: '.$this->composer($runtime)); $this->line('Extensions: '.implode(', ', $this->loadedExtensions())); $this->newLine(); } + /** + * The bundled Composer, and whether this executable modified it. + * + * The CLI distributes a package manager, so what it distributes is reported rather + * than left to be discovered: a build that carries a patch against the published + * archive says which one, and why is in `bin/lib/composer-patches.php`. + */ + protected function composer(RuntimeManager $runtime): string + { + $version = $runtime->composerVersion(); + + if ($version === null) { + return 'none bundled (running from a source checkout)'; + } + + $patches = $runtime->composerPatches(); + + return $patches === [] + ? sprintf('%s bundled', $version) + : sprintf('%s bundled, patched (%s)', $version, implode(', ', $patches)); + } + /** @return list */ protected function loadedExtensions(): array { diff --git a/app/Commands/Internal/BundledProgramCommand.php b/app/Commands/Internal/BundledProgramCommand.php new file mode 100644 index 00000000..7d2a8c68 --- /dev/null +++ b/app/Commands/Internal/BundledProgramCommand.php @@ -0,0 +1,44 @@ +ignoreValidationErrors(); + } + + public function handle(): int + { + return $this->dispatcher()->run((string) $this->getName(), (new Launcher())->argumentsFor($_SERVER['argv'] ?? [])); + } + + protected function dispatcher(): RuntimeDispatcher + { + return new RuntimeDispatcher($this->laravel->make(RuntimeManager::class)); + } +} diff --git a/app/Commands/Internal/Describer.php b/app/Commands/Internal/Describer.php index 1ca6acf1..8e9a4e7b 100644 --- a/app/Commands/Internal/Describer.php +++ b/app/Commands/Internal/Describer.php @@ -4,31 +4,170 @@ namespace App\Commands\Internal; -use Illuminate\Console\Command; +use App\Launcher\Launcher; +use Illuminate\Console\Application; +use Illuminate\Contracts\Config\Repository; +use Symfony\Component\Console\Output\OutputInterface; use NunoMaduro\LaravelConsoleSummary\Describer as BaseDescriber; +use NunoMaduro\LaravelConsoleSummary\Contracts\DescriberContract; -use function usort; -use function strcmp; +use function max; +use function ksort; +use function sprintf; +use function explode; +use function implode; +use function defined; +use function in_array; +use function mb_strlen; +use function str_repeat; +use function array_values; /** - * @internal Custom Laravel summary command describer implementation. + * @internal Renders the command list, in sections that say where each command runs. * - * @depends on https://github.com/nunomaduro/laravel-console-summary/pull/20 + * The `hyde` executable answers some commands itself, wherever it is run, and hands + * everything else to the project it was invoked against. That distinction decides + * what a command can do and what it acts on, so the list states it rather than + * leaving one flat alphabetical run of names to imply they are all alike. + * + * Membership of the CLI section is read from the launcher's own constants, so the + * list cannot come to disagree with the routing it is describing. + * + * @see \App\Launcher\Launcher::ownedCommands() */ class Describer extends BaseDescriber { - protected static function sortCommandsInGroup(array &$commands): void + /** The heading for the commands the executable answers itself, and what it means. */ + protected const CLI_SECTION = ['HYDE CLI', '(the executable itself, in any directory)']; + + /** The heading for everything the project owns, and what it means. */ + protected const PROJECT_SECTION = ['PROJECT', '(the Hyde site in the current directory)']; + + public function __construct(private readonly Repository $config) + { + parent::__construct($config); + } + + /** + * Describe the usage line. + * + * The package's own configuration defaults the binary name to null rather than + * leaving it unset, so its fallback to `ARTISAN_BINARY` never fires and the line + * reads `USAGE: `. The name of the program is the useful half. + */ + protected function describeUsage(OutputInterface $output): DescriberContract + { + $binary = $this->config->get('laravel-console-summary.binary') + ?: (defined('ARTISAN_BINARY') ? ARTISAN_BINARY : 'hyde'); + + $output->write(" USAGE: {$binary} [options] [arguments]\n"); + + return $this; + } + + protected function describeCommands(Application $application, OutputInterface $output): DescriberContract { - // This makes so the `new` project command is always the first one in the list. - - usort($commands, function (Command $a, Command $b): int { - if ($a->getName() === 'new' && $b->getName() !== 'new') { - return -1; - } elseif ($a->getName() !== 'new' && $b->getName() === 'new') { - return 1; - } else { - return strcmp($a->getName(), $b->getName()); + $commands = $this->visibleCommands($application); + + $width = 0; + + foreach ($commands as $command) { + $width = max($width, mb_strlen((string) $command->getName())); + } + + $cli = []; + + foreach (Launcher::ownedCommands() as $name) { + if (isset($commands[$name])) { + $cli[] = $commands[$name]; + + unset($commands[$name]); + } + } + + if ($cli !== []) { + $this->describeSection($output, self::CLI_SECTION, [$cli], $width); + } + + $this->describeSection($output, self::PROJECT_SECTION, $this->groupByNamespace($commands), $width); + + $output->writeln(''); + + return $this; + } + + /** + * The commands that belong in the list at all, keyed by name. + * + * Hidden commands, and anything the `laravel-console-summary.hide` configuration + * names, are left out. Wildcards there match a whole namespace. + * + * @return array + */ + protected function visibleCommands(Application $application): array + { + $hidden = (array) $this->config->get('laravel-console-summary.hide', []); + + $commands = []; + + foreach ($application->all() as $command) { + $name = (string) $command->getName(); + + if ($command->isHidden() || in_array($name, $hidden, true) || in_array(explode(':', $name)[0].':*', $hidden, true)) { + continue; + } + + $commands[$name] = $command; + } + + ksort($commands); + + return $commands; + } + + /** + * Split the project's commands into their namespaces, ungrouped ones first. + * + * @param array $commands + * @return list> + */ + protected function groupByNamespace(array $commands): array + { + $groups = []; + + foreach ($commands as $name => $command) { + $parts = explode(':', $name); + + $groups[isset($parts[1]) ? $parts[0] : ''][] = $command; + } + + ksort($groups); + + return array_values($groups); + } + + /** + * @param array{0: string, 1: string} $section + * @param list> $groups + */ + protected function describeSection(OutputInterface $output, array $section, array $groups, int $width): void + { + $output->write(sprintf("\n %s %s\n", $section[0], $section[1])); + + foreach ($groups as $index => $commands) { + if ($index > 0) { + $output->write("\n"); + } + + foreach ($commands as $command) { + $output->write(sprintf( + " %s%s%s%s\n", + $command->getName(), + str_repeat(' ', $width - mb_strlen((string) $command->getName()) + 1), + $command->getAliases() ? '['.implode('|', $command->getAliases()).'] ' : '', + $command->getDescription() + )); } - }); + } } } diff --git a/app/Commands/NewProjectCommand.php b/app/Commands/NewProjectCommand.php index 1ac7706e..1ff893c1 100644 --- a/app/Commands/NewProjectCommand.php +++ b/app/Commands/NewProjectCommand.php @@ -8,6 +8,7 @@ use App\Application; use App\Launcher\Project; use App\Support\ComposerBinary; +use App\Launcher\RuntimeManager; use App\Support\PortableProjectBuilder; use Illuminate\Console\Command; use Illuminate\Support\Facades\File; @@ -114,9 +115,11 @@ protected function createPortableProject(string $name): int protected function createComposerProject(string $name): int { + $runtime = $this->runtime(); + // The check happens before anything is written, so a machine without Composer // is never left with a half-created project directory. - $composer = ComposerBinary::locate(); + $composer = ComposerBinary::command($runtime); if ($composer === null) { $this->newLine(); @@ -128,6 +131,12 @@ protected function createComposerProject(string $name): int $path = $this->resolvePath($name); $existed = is_dir($path); + if ($composer === ComposerBinary::bundled($runtime)) { + // The machine may never have had a Composer installed, so say where this one + // came from, and which one it is: the project is created by it. + $this->line(sprintf(' Using the Composer bundled with this executable (%s)', $runtime->composerVersion())); + } + $result = Process::forever()->path($this->workingDirectory())->run( $this->createProjectCommand($composer, $name), $this->bufferedOutput() @@ -165,14 +174,16 @@ protected function createComposerProject(string $name): int * `HYDE_PROJECT_SOURCE`. That variable is a development mechanism, is never set for a * released executable, and is the only thing that can move this command off Packagist. * + * @param list $composer The command that runs Composer, which may be the + * bundled PHP runtime followed by the bundled PHAR. * @return list */ - protected function createProjectCommand(string $composer, string $name): array + protected function createProjectCommand(array $composer, string $name): array { $source = static::developmentSource(); // Composer takes the package, then the directory, then the version constraint. - $command = [$composer, 'create-project', 'hyde/hyde', $name, $this->projectConstraint($source)]; + $command = [...$composer, 'create-project', 'hyde/hyde', $name, $this->projectConstraint($source)]; $command[] = '--prefer-dist'; $command[] = $this->withAnsi() ? '--ansi' : '--no-ansi'; @@ -225,6 +236,11 @@ protected function workingDirectory(): string return $this->laravel->make(Project::class)->workingDirectory; } + protected function runtime(): RuntimeManager + { + return $this->laravel->make(RuntimeManager::class); + } + protected function withAnsi(): bool { return ! $this->option('no-ansi') || $this->option('ansi'); diff --git a/app/Commands/PhpCommand.php b/app/Commands/PhpCommand.php new file mode 100644 index 00000000..430a2e51 --- /dev/null +++ b/app/Commands/PhpCommand.php @@ -0,0 +1,27 @@ + + */ + public const LAUNCHER_COMMANDS = ['new', 'info', 'self-update']; + + /** + * Commands answered by the programs the executable carries, without booting anything. + * + * These are not about a project at all: they hand the bundled PHP runtime, and the + * Composer shipped beside it, to the user. They are answered before the project is + * even considered, which is what lets `hyde composer install` repair a Composer + * project whose dependencies are missing — the one state in which the launcher + * refuses to run anything else. + * + * @var list + */ + public const RUNTIME_COMMANDS = ['php', 'composer']; + + /** + * Global options whose value is the *next* token rather than part of this one. + * + * Symfony works this out from the input definition, which does not exist here yet: + * the launcher runs before the application. So the one global option that takes a + * value is named. `--env development` is two tokens and the second is not the + * command; `--env=development` needs no help, being one token that is an option. + * + * `--env` is declared `VALUE_OPTIONAL` by `illuminate/console`, and Laravel's own + * environment detector reads it in both spellings. Every other global option the + * application defines takes no value at all. * * @var list */ - public const LAUNCHER_COMMANDS = ['info', 'new', 'self-update']; + public const VALUE_OPTIONS = ['--env']; private static ?Project $project = null; public function __construct( private readonly ProjectDetector $detector = new ProjectDetector(), private readonly ProjectDispatcher $dispatcher = new ProjectDispatcher(), + private readonly RuntimeDispatcher $runtime = new RuntimeDispatcher(), ) { // } @@ -66,6 +100,13 @@ public function __construct( */ public function run(array $argv): ?int { + // Answered before anything else, including detection: these commands are about + // the executable's own runtime, and one of them exists to repair the broken + // project state that detection would otherwise refuse to go any further in. + if ($this->isRuntimeCommand($argv)) { + return $this->runtime->run((string) $this->commandName($argv), $this->argumentsFor($argv)); + } + $project = $this->detect(); // The CLI's own source checkout is itself a Hyde Composer project. When the file @@ -160,21 +201,86 @@ public static function swap(?Project $project): void */ public function commandName(array $argv): ?string { - foreach (array_slice($argv, 1) as $argument) { + $index = $this->commandIndex($argv); + + return $index === null ? null : $argv[$index]; + } + + /** + * Where in `$argv` the command name is, if it is there at all. + * + * Not simply the first token that does not begin with a dash: the value of a global + * option is such a token, and `hyde --env development php -v` names the command + * `php`. Getting this wrong routes the call to a command nobody typed. + * + * @param list $argv + */ + public function commandIndex(array $argv): ?int + { + $skip = false; + + foreach (array_slice($argv, 1, preserve_keys: true) as $index => $argument) { + if ($skip) { + // The previous token was an option that takes this one as its value. + $skip = false; + + continue; + } + + // A bare `--` ends the options, so whatever follows it is the command. + if ($argument === '--') { + return isset($argv[$index + 1]) ? $index + 1 : null; + } + if (! str_starts_with($argument, '-')) { - return $argument; + return $index; } + + $skip = in_array($argument, self::VALUE_OPTIONS, true); } return null; } + /** + * Everything that was typed after the command name. + * + * Measured from the command rather than from the program name, so that an option + * meant for the CLI — `hyde -v php -r '...'` — is not passed on to the program + * being run, which knows nothing about it. + * + * @param list $argv + * @return list + */ + public function argumentsFor(array $argv): array + { + $index = $this->commandIndex($argv); + + return $index === null ? [] : array_values(array_slice($argv, $index + 1)); + } + /** @param list $argv */ public function isLauncherCommand(array $argv): bool { return in_array($this->commandName($argv), self::LAUNCHER_COMMANDS, true); } + /** @param list $argv */ + public function isRuntimeCommand(array $argv): bool + { + return in_array($this->commandName($argv), self::RUNTIME_COMMANDS, true); + } + + /** + * Every command the executable answers for itself, whichever way it answers them. + * + * @return list + */ + public static function ownedCommands(): array + { + return array_merge(self::LAUNCHER_COMMANDS, self::RUNTIME_COMMANDS); + } + /** * Resolve, and define, the paths the embedded application boots against. * diff --git a/app/Launcher/ProjectDispatcher.php b/app/Launcher/ProjectDispatcher.php index c3e4355c..08bc380c 100644 --- a/app/Launcher/ProjectDispatcher.php +++ b/app/Launcher/ProjectDispatcher.php @@ -9,16 +9,8 @@ use function fread; use function fclose; use function getenv; -use function defined; -use function dirname; use function realpath; -use function is_string; -use function strcasecmp; -use function array_keys; -use function proc_open; -use function proc_close; use function array_merge; -use function str_replace; use function array_values; use function str_starts_with; @@ -56,13 +48,12 @@ public function dispatch(Project $project, array $arguments = []): int $this->guardAgainstRecursion($entryPoint); - $process = @proc_open($this->command($entryPoint, $arguments), $this->descriptors(), $pipes, $project->root, $this->environment($entryPoint)); - - if ($process === false) { - throw new LauncherException("Unable to start the project's Hyde executable at $entryPoint."); - } - - return proc_close($process); + return Subprocess::run( + $this->command($entryPoint, $arguments), + $project->root, + $this->environment($entryPoint), + "Unable to start the project's Hyde executable at $entryPoint." + ); } /** @@ -112,17 +103,10 @@ protected function guardAgainstRecursion(string $entryPoint): void */ protected function environment(string $entryPoint): array { - $environment = getenv(); + $environment = Subprocess::environmentWith($this->runtime->path()); $environment[self::DISPATCH_MARKER] = $entryPoint; - // Windows stores the search path as `Path`, so the existing key is found rather - // than assumed: adding a second, differently cased one would leave the child - // with two search paths and no say in which of them is used. - $key = $this->searchPathKey($environment); - - $environment[$key] = $this->searchPath(is_string($environment[$key] ?? null) ? $environment[$key] : ''); - return $environment; } @@ -133,29 +117,7 @@ protected function environment(string $entryPoint): array */ public function searchPathKey(array $environment): string { - foreach (array_keys($environment) as $name) { - if (strcasecmp((string) $name, 'PATH') === 0) { - return (string) $name; - } - } - - return 'PATH'; - } - - /** - * Put the directory holding the bundled PHP runtime at the front of the search path. - * - * This is the launcher doing its job rather than a fallback. A Hyde project shells out - * to a bare `php` for its own subprocesses — the realtime compiler's server, most - * obviously — and on a machine with no PHP installed there would be nothing for it - * to find. The executable supplies the runtime, which is what makes `hyde serve` - * work in a Composer project on a machine that has no PHP. - */ - protected function searchPath(string $path): string - { - $runtime = dirname($this->runtime->path()); - - return $path === '' ? $runtime : $runtime.PATH_SEPARATOR.$path; + return Subprocess::searchPathKey($environment); } /** @@ -234,26 +196,4 @@ protected function isPhpScript(string $path): bool return str_starts_with($head, '#!') || str_starts_with($head, ' - */ - protected function descriptors(): array - { - if (defined('STDIN') && defined('STDOUT') && defined('STDERR')) { - return [STDIN, STDOUT, STDERR]; - } - - return [ - ['file', 'php://stdin', 'r'], - ['file', 'php://stdout', 'w'], - ['file', 'php://stderr', 'w'], - ]; - } } diff --git a/app/Launcher/RuntimeDispatcher.php b/app/Launcher/RuntimeDispatcher.php new file mode 100644 index 00000000..a099fb6e --- /dev/null +++ b/app/Launcher/RuntimeDispatcher.php @@ -0,0 +1,171 @@ + + */ + public const SELF_UPDATE = ['self-update', 'selfupdate']; + + public function __construct(private readonly RuntimeManager $runtime = new RuntimeManager()) + { + // + } + + /** + * Run one of the bundled programs and return its exit status. + * + * @param list $arguments The arguments to forward, exactly as they were typed. + * + * @throws \App\Launcher\LauncherException If the program cannot be provided or started. + */ + public function run(string $command, array $arguments = []): int + { + return match ($command) { + 'php' => $this->php($arguments), + 'composer' => $this->composer($arguments), + default => throw new LauncherException("The executable bundles no `$command` program."), + }; + } + + /** + * Run the bundled PHP CLI. + * + * Nothing is added to what the user typed, and nothing is interpreted: the runtime + * behaves exactly as the same PHP binary would if it were installed on the host. + * + * @param list $arguments + * + * @throws \App\Launcher\LauncherException + */ + public function php(array $arguments = []): int + { + $php = $this->runtime->path(); + + return $this->start(array_merge([$php], array_values($arguments)), $php); + } + + /** + * Run the bundled Composer, using the bundled PHP runtime. + * + * Composer is a PHAR, so the runtime is named as the program and Composer as its + * first argument. That is also what decides which PHP the install runs against: + * the one this executable ships, and never whatever a shebang would find. + * + * @param list $arguments + * + * @throws \App\Launcher\LauncherException + */ + public function composer(array $arguments = []): int + { + $refused = $this->refuseSelfUpdate($arguments); + + if ($refused !== null) { + return $refused; + } + + $php = $this->runtime->path(); + + return $this->start(array_merge([$php, $this->runtime->composerPath()], array_values($arguments)), $php); + } + + /** + * Refuse to let the bundled Composer replace itself. + * + * It is extracted from the executable and verified against the checksum recorded at + * build time on every run, so an update to the cached copy would be repaired away by + * the very next command — silently, since repairing a copy that fails verification + * is exactly what the extraction does. Saying so beats appearing to work. + * + * The check reads the first argument that is not an option, and stops looking at the + * first one that is not the updater: an option that took *its* value as a separate + * token would otherwise be read as a command name. A guard is allowed to miss; + * it is not allowed to block something the user did not ask for. + * + * This is not raised as a `LauncherException`: nothing failed to start, and that is + * what the launcher reports those as. + * + * @param list $arguments + * @return int|null The exit status to stop with, or null to carry on. + */ + protected function refuseSelfUpdate(array $arguments): ?int + { + foreach ($arguments as $argument) { + if (str_starts_with($argument, '-')) { + continue; + } + + if (! in_array($argument, self::SELF_UPDATE, true)) { + return null; + } + + fwrite(STDERR, sprintf(<<<'TEXT' + + `composer %s` is not available here. + + The Composer inside this executable is versioned with the HydeCLI, and is + verified against the checksum recorded when the executable was built. An + update would be replaced again by the next command that needs it. + + Run `hyde self-update` for an executable carrying a newer Composer, or + install Composer yourself and run that one. + + + TEXT, $argument)); + + return 1; + } + + return null; + } + + /** + * Start a bundled program, with the runtime on its search path. + * + * The working directory is inherited rather than set to the project root, because + * these commands are not about the project: `hyde php script.php` has to resolve + * that path from wherever the user is standing. + * + * @param list $command + * + * @throws \App\Launcher\LauncherException + */ + private function start(array $command, string $runtime): int + { + return Subprocess::run($command, null, Subprocess::environmentWith($runtime), sprintf('Unable to start the bundled PHP runtime at %s.', $runtime)); + } +} diff --git a/app/Launcher/RuntimeManager.php b/app/Launcher/RuntimeManager.php index 682fbe4c..98d1abac 100644 --- a/app/Launcher/RuntimeManager.php +++ b/app/Launcher/RuntimeManager.php @@ -16,9 +16,12 @@ use function getenv; use function getmypid; use function dirname; +use function basename; use function hash_file; use function is_string; use function is_array; +use function array_map; +use function array_values; use function is_writable; use function json_decode; use function hash_equals; @@ -62,6 +65,10 @@ * It never resolves `php` from PATH. The only non-embedded interpreter it will * ever use is the very CLI process that is already running the code, which * only happens when the CLI is run from a source checkout during development. + * + * Composer is bundled the same way, and extracted the same way. It is a PHAR + * rather than a binary, so it is run by the runtime above rather than on its + * own, and it is cached by its own version rather than by platform. */ final class RuntimeManager { @@ -71,6 +78,15 @@ final class RuntimeManager /** The build manifest describing the embedded runtime. */ public const MANIFEST_FILE = 'runtime.json'; + /** + * The Composer archive the executable ships beside the runtime. + * + * Composer is a PHAR rather than a native binary, so the same bundled PHP runs it. + * It is here so that `hyde composer` and `hyde new --composer` work on a machine + * that has neither PHP nor Composer, which is the whole promise of the CLI. + */ + public const COMPOSER_FILE = 'composer.phar'; + /** * The suffix the embedded runtime binary carries inside the archive. * @@ -103,6 +119,8 @@ final class RuntimeManager private ?string $resolvedArchive = null; + private ?string $resolvedComposer = null; + private readonly Platform $platform; public function __construct(?Platform $platform = null, private readonly ?string $applicationRoot = null) @@ -386,32 +404,147 @@ private function resolve(): string public function extract(): string { $manifest = $this->manifest(); - $directory = $this->cacheDirectory($manifest); - $target = $directory.'/'.$manifest['filename']; + + return $this->extractResource( + $this->embeddedBinaryPath(), + $this->cacheDirectory($manifest).'/'.$manifest['filename'], + $manifest['checksum'], + 'PHP runtime', + executable: true + ); + } + + /** + * Get the path to the bundled Composer archive, extracting it if needed. + * + * @throws \App\Launcher\LauncherException If the executable ships no Composer. + */ + public function composerPath(): string + { + return $this->resolvedComposer ??= $this->extractComposer(); + } + + public function hasBundledComposer(): bool + { + return is_file($this->embeddedComposerPath()) && $this->composerManifest() !== null; + } + + /** + * What the build recorded about the bundled Composer, or null when none was bundled. + * + * This is deliberately a separate reading of the manifest rather than a wider + * {@see self::manifest()}: the PHP runtime is what the executable cannot work + * without, and describing it must not start depending on Composer being there. + * + * @return array{version: string, filename: string, checksum: string, patches: list}|null + */ + public function composerManifest(): ?array + { + if (! is_file($this->manifestPath())) { + return null; + } + + $manifest = json_decode((string) @file_get_contents($this->manifestPath()), true); + + $composer = is_array($manifest) ? ($manifest['composer'] ?? null) : null; + + if (! is_array($composer) || ! isset($composer['version'], $composer['checksum'])) { + return null; + } + + return [ + 'version' => (string) $composer['version'], + 'filename' => (string) ($composer['filename'] ?? self::COMPOSER_FILE), + 'checksum' => (string) $composer['checksum'], + + // What this build changed in the published archive. Empty is the goal. + 'patches' => array_values(array_map('strval', is_array($composer['patches'] ?? null) ? $composer['patches'] : [])), + ]; + } + + /** + * The patches this executable carries against the Composer release it ships. + * + * The bundled Composer is not always the published archive byte for byte: where a + * Composer bug stops the CLI working on one of its platforms, the build carries the + * minimum change needed. What was changed is recorded rather than left to be + * discovered, so `hyde info -v` can say so. + * + * @return list + */ + public function composerPatches(): array + { + return $this->composerManifest()['patches'] ?? []; + } + + /** The version of Composer this executable ships, if it ships one. */ + public function composerVersion(): ?string + { + return $this->composerManifest()['version'] ?? null; + } + + /** @throws \App\Launcher\LauncherException */ + private function extractComposer(): string + { + $manifest = $this->composerManifest(); + + if ($manifest === null || ! is_file($this->embeddedComposerPath())) { + throw new LauncherException(<<<'TEXT' + This executable does not bundle Composer. + + Released executables ship a Composer of their own, so this is either a build + made without one, or a source checkout. Install Composer, or use an official + release for your platform. + TEXT); + } + + return $this->extractResource( + $this->embeddedComposerPath(), + $this->composerCacheDirectory($manifest).'/'.$manifest['filename'], + $manifest['checksum'], + 'Composer' + ); + } + + /** + * Extract one gzipped resource out of the executable, or reuse a valid extraction. + * + * The checksum recorded at build time is of the decompressed file, so verification + * covers exactly the bytes that will be run. + * + * @throws \App\Launcher\LauncherException + */ + private function extractResource(string $source, string $target, string $checksum, string $label, bool $executable = false): string + { + $directory = dirname($target); clearstatcache(true, $target); - if ($this->isValid($target, $manifest['checksum'])) { - $this->ensureExecutable($target); + if ($this->isValid($target, $checksum)) { + if ($executable) { + $this->ensureExecutable($target); + } return $target; } $this->ensureDirectoryExists($directory); - $temporary = sprintf('%s/.%s.%s.%s', $directory, $manifest['filename'], (string) getmypid(), bin2hex(random_bytes(6))); + $temporary = sprintf('%s/.%s.%s.%s', $directory, basename($target), (string) getmypid(), bin2hex(random_bytes(6))); - $this->decompress($this->embeddedBinaryPath(), $temporary); + $this->decompress($source, $temporary, $label); - if (! $this->isValid($temporary, $manifest['checksum'])) { + if (! $this->isValid($temporary, $checksum)) { @unlink($temporary); - throw new LauncherException('The bundled PHP runtime failed its checksum verification. This executable may be corrupt or truncated; please reinstall it.'); + throw new LauncherException("The bundled $label failed its checksum verification. This executable may be corrupt or truncated; please reinstall it."); } - $this->ensureExecutable($temporary); + if ($executable) { + $this->ensureExecutable($temporary); + } - $this->install($temporary, $target, $manifest['checksum']); + $this->install($temporary, $target, $checksum, $label); return $target; } @@ -424,7 +557,7 @@ public function extract(): string * may additionally be locked by another Hyde process that is currently running * it, so the stale file is moved aside first and cleaned up opportunistically. */ - private function install(string $temporary, string $target, string $checksum): void + private function install(string $temporary, string $target, string $checksum, string $label = 'PHP runtime'): void { if (@rename($temporary, $target)) { return; @@ -454,7 +587,7 @@ private function install(string $temporary, string $target, string $checksum): v @unlink($temporary); - throw new LauncherException("Unable to install the bundled PHP runtime at $target. Another process may be using it, or the directory may be read-only."); + throw new LauncherException("Unable to install the bundled $label at $target. Another process may be using it, or the directory may be read-only."); } /** @@ -462,7 +595,7 @@ private function install(string $temporary, string $target, string $checksum): v * * @throws \App\Launcher\LauncherException */ - private function decompress(string $source, string $destination): void + private function decompress(string $source, string $destination, string $label = 'PHP runtime'): void { $in = @fopen($source, 'rb'); $out = @fopen($destination, 'wb'); @@ -476,7 +609,7 @@ private function decompress(string $source, string $destination): void fclose($out); } - throw new LauncherException("Unable to extract the bundled PHP runtime to $destination. Check that the directory is writable."); + throw new LauncherException("Unable to extract the bundled $label to $destination. Check that the directory is writable."); } // A window of 31 selects gzip framing rather than raw deflate. @@ -547,6 +680,19 @@ public function cacheDirectory(array $manifest): string return sprintf('%s/hyde/runtime/%s/%s', $this->cacheRoot(), $manifest['version'], $manifest['platform']); } + /** + * Where the bundled Composer is extracted to. + * + * Keyed by its own version, and not by the platform: a PHAR is the same file + * everywhere, and the runtime that runs it is chosen separately. + * + * @param array{version: string, filename: string, checksum: string} $manifest + */ + public function composerCacheDirectory(array $manifest): string + { + return sprintf('%s/hyde/composer/%s', $this->cacheRoot(), $manifest['version']); + } + /** The per-user cache root, following platform conventions and the XDG specification. */ public function cacheRoot(): string { @@ -590,6 +736,11 @@ public function embeddedBinaryPath(): string return $this->applicationRoot().'/'.self::RUNTIME_DIRECTORY.'/'.$this->platform->runtimeFilename().self::RUNTIME_SUFFIX; } + public function embeddedComposerPath(): string + { + return $this->applicationRoot().'/'.self::RUNTIME_DIRECTORY.'/'.self::COMPOSER_FILE.self::RUNTIME_SUFFIX; + } + public function manifestPath(): string { return $this->applicationRoot().'/'.self::RUNTIME_DIRECTORY.'/'.self::MANIFEST_FILE; diff --git a/app/Launcher/Subprocess.php b/app/Launcher/Subprocess.php new file mode 100644 index 00000000..72bd2f1d --- /dev/null +++ b/app/Launcher/Subprocess.php @@ -0,0 +1,120 @@ + $command The program, then its arguments, each unquoted. + * @param array|null $environment The child's environment, or null to inherit ours. + * + * @throws \App\Launcher\LauncherException If the process cannot be started. + */ + public static function run(array $command, ?string $workingDirectory = null, ?array $environment = null, ?string $failure = null): int + { + $process = @proc_open(array_values($command), self::descriptors(), $pipes, $workingDirectory, $environment); + + if ($process === false) { + throw new LauncherException($failure ?? 'Unable to start '.($command[0] ?? 'the requested program').'.'); + } + + return proc_close($process); + } + + /** + * Inherit the parent's standard streams so the child keeps its TTY. + * + * Passing the stream resources through gives the child the real file + * descriptors, which keeps interactive prompts, colours, and piping + * behaving exactly as they would if the program were run directly. + * + * @return array + */ + public static function descriptors(): array + { + if (defined('STDIN') && defined('STDOUT') && defined('STDERR')) { + return [STDIN, STDOUT, STDERR]; + } + + return [ + ['file', 'php://stdin', 'r'], + ['file', 'php://stdout', 'w'], + ['file', 'php://stderr', 'w'], + ]; + } + + /** + * Our own environment, with the directory holding the bundled PHP runtime at the + * front of the search path. + * + * This is the launcher doing its job rather than a fallback. The programs it starts + * shell out to a bare `php` for their own subprocesses — the realtime compiler's + * server and Composer's install scripts, most obviously — and on a machine with no + * PHP installed there would be nothing for them to find. + * + * @return array + */ + public static function environmentWith(string $runtime): array + { + $environment = getenv(); + + $key = self::searchPathKey($environment); + $directory = dirname($runtime); + $path = is_string($environment[$key] ?? null) ? $environment[$key] : ''; + + $environment[$key] = $path === '' ? $directory : $directory.PATH_SEPARATOR.$path; + + return $environment; + } + + /** + * Find the environment key holding the search path, whatever case the platform used. + * + * Windows stores it as `Path`, so the existing key is found rather than assumed: + * adding a second, differently cased one would leave the child with two search + * paths and no say in which of them is used. + * + * @param array $environment + */ + public static function searchPathKey(array $environment): string + { + foreach (array_keys($environment) as $name) { + if (strcasecmp((string) $name, 'PATH') === 0) { + return (string) $name; + } + } + + return 'PATH'; + } +} diff --git a/app/Providers/AppServiceProvider.php b/app/Providers/AppServiceProvider.php index 1fe87079..ce448e82 100644 --- a/app/Providers/AppServiceProvider.php +++ b/app/Providers/AppServiceProvider.php @@ -2,8 +2,10 @@ namespace App\Providers; +use App\Commands\PhpCommand; use App\Commands\InfoCommand; use App\Commands\ServeCommand; +use App\Commands\ComposerCommand; use App\Commands\NewProjectCommand; use App\Commands\SelfUpdateCommand; use App\Commands\Internal\Describer; @@ -22,6 +24,8 @@ public function register(): void InfoCommand::class, NewProjectCommand::class, SelfUpdateCommand::class, + PhpCommand::class, + ComposerCommand::class, ]); } diff --git a/app/Support/ComposerBinary.php b/app/Support/ComposerBinary.php index 6b8786b8..a20c5b80 100644 --- a/app/Support/ComposerBinary.php +++ b/app/Support/ComposerBinary.php @@ -4,6 +4,8 @@ namespace App\Support; +use App\Launcher\RuntimeManager; + use function getenv; use function is_file; use function explode; @@ -11,11 +13,15 @@ use function is_executable; /** - * Locates a Composer executable on the host. + * Works out which Composer to run, and how to run it. + * + * The executable bundles a Composer of its own, which is the one that gets used: + * it is the version this release was built and tested against, and it is there on + * a machine that has no Composer at all. The host's own Composer is the fallback, + * and is what a source checkout — which bundles nothing — actually uses. * - * The CLI never needs Composer for itself; this exists only so that - * `hyde new --composer` can fail cleanly, and before it writes anything, - * on a machine where Composer is not installed. + * The answer is a command rather than a path, because the bundled Composer is a + * PHAR: it is run by the bundled PHP runtime, not on its own. */ final class ComposerBinary { @@ -28,7 +34,47 @@ final class ComposerBinary /** @internal Test hook allowing the lookup to be forced to fail. */ public static bool $forceMissing = false; - /** Find a Composer executable, or null when the host has none. */ + /** + * The command that runs Composer, or null when there is no Composer to run. + * + * @return list|null + */ + public static function command(?RuntimeManager $runtime = null): ?array + { + if (self::$forceMissing) { + return null; + } + + if (self::$fake !== null) { + return [self::$fake]; + } + + $bundled = self::bundled($runtime); + + if ($bundled !== null) { + return $bundled; + } + + $host = self::locate(); + + return $host === null ? null : [$host]; + } + + /** + * The command that runs the Composer inside the executable, if it has one. + * + * A source checkout has none, and neither does a build made without one. + * + * @return list|null + */ + public static function bundled(?RuntimeManager $runtime = null): ?array + { + $runtime ??= RuntimeManager::make(); + + return $runtime->hasBundledComposer() ? [$runtime->path(), $runtime->composerPath()] : null; + } + + /** Find a Composer executable on the host, or null when it has none. */ public static function locate(): ?string { if (self::$forceMissing) { @@ -52,9 +98,10 @@ public static function locate(): ?string return null; } - public static function available(): bool + /** Is there a Composer to run at all, bundled or on the host? */ + public static function available(?RuntimeManager $runtime = null): bool { - return self::locate() !== null; + return self::command($runtime) !== null; } /** @return list */ diff --git a/bin/build-native.ps1 b/bin/build-native.ps1 index 3c132071..fa347c38 100644 --- a/bin/build-native.ps1 +++ b/bin/build-native.ps1 @@ -20,6 +20,8 @@ $Config = Join-Path $Root 'build\runtime.json' $runtime = Get-Content $Config -Raw | ConvertFrom-Json $phpVersion = $runtime.php +$composerVersion = $runtime.composer.version +$composerChecksum = $runtime.composer.sha256 # Windows PHP has never had pcntl or posix, and static-php-cli refuses to start a build # that asks for one. build/runtime.json records which extensions cannot exist here; @@ -30,10 +32,32 @@ $extensions = (($runtime.extensions.PSObject.Properties.Name) | Where-Object { $ Write-Host "==> HydeCLI native build" Write-Host " PHP version: $phpVersion" +Write-Host " Composer: $composerVersion" Write-Host " Extensions: $extensions" New-Item -ItemType Directory -Force -Path $Work | Out-Null +# Composer is bundled inside the executable, so that a machine with neither PHP nor +# Composer can still install a project's dependencies. It is pinned by version and +# verified on every build, cached copy included: a Composer that does not hash to +# what build/runtime.json records is not the Composer this release ships. +$composerPhar = Join-Path $Work "composer-$composerVersion.phar" + +if (-not (Test-Path $composerPhar)) { + Write-Host "==> Downloading Composer $composerVersion" + Invoke-WebRequest -Uri "https://getcomposer.org/download/$composerVersion/composer.phar" -OutFile "$composerPhar.download" + Move-Item "$composerPhar.download" $composerPhar +} + +$composerActual = (Get-FileHash -Algorithm SHA256 -Path $composerPhar).Hash.ToLower() + +if ($composerActual -ne $composerChecksum) { + Remove-Item $composerPhar -Force + throw "The downloaded Composer does not match the checksum in build/runtime.json.`n expected: $composerChecksum`n actual: $composerActual" +} + +Write-Host "==> Composer $composerVersion verified" + $spc = Join-Path $Work 'spc.exe' if (-not $SkipSpc) { @@ -82,7 +106,7 @@ Write-Host '==> Verifying the embedded dependency graph is v3' if ($LASTEXITCODE -ne 0) { throw 'The embedded dependency graph is not HydePHP v3' } Write-Host '==> Building the executable' -$arguments = @('-d', 'phar.readonly=0', (Join-Path $Root 'bin\build-phar.php'), "--micro=$micro", "--runtime=$php") +$arguments = @('-d', 'phar.readonly=0', (Join-Path $Root 'bin\build-phar.php'), "--micro=$micro", "--runtime=$php", "--composer=$composerPhar") if ($Build) { $arguments += "--build=$Build" } diff --git a/bin/build-native.sh b/bin/build-native.sh index a9111b59..273f5d4b 100755 --- a/bin/build-native.sh +++ b/bin/build-native.sh @@ -38,7 +38,13 @@ read_extensions() { php -r '$c = json_decode(file_get_contents($argv[1]), true); echo implode(",", array_diff(array_keys($c["extensions"]), $c["unsupported-extensions"][$argv[2]] ?? []));' "$CONFIG" "$1" } +read_composer() { + php -r '$c = json_decode(file_get_contents($argv[1]), true); echo $c["composer"][$argv[2]];' "$CONFIG" "$1" +} + PHP_VERSION="$(read_config php)" +COMPOSER_VERSION="$(read_composer version)" +COMPOSER_CHECKSUM="$(read_composer sha256)" case "$(uname -s)" in Darwin) TARGET="macos" ;; @@ -50,10 +56,37 @@ EXTENSIONS="$(read_extensions "$TARGET")" echo "==> HydeCLI native build" echo " PHP version: $PHP_VERSION" +echo " Composer: $COMPOSER_VERSION" echo " Extensions: $EXTENSIONS" mkdir -p "$WORK" +# Composer is bundled inside the executable, so that a machine with neither PHP nor +# Composer can still install a project's dependencies. It is pinned by version and +# verified on every build, cached copy included: a Composer that does not hash to +# what build/runtime.json records is not the Composer this release ships. +COMPOSER_PHAR="$WORK/composer-$COMPOSER_VERSION.phar" + +if [ ! -f "$COMPOSER_PHAR" ]; then + echo "==> Downloading Composer $COMPOSER_VERSION" + curl -fsSL -o "$COMPOSER_PHAR.download" "https://getcomposer.org/download/$COMPOSER_VERSION/composer.phar" + mv "$COMPOSER_PHAR.download" "$COMPOSER_PHAR" +fi + +# Hashed with PHP rather than with `sha256sum` or `shasum`, which are not the same +# command on every host this script runs on. PHP is already a requirement here. +COMPOSER_ACTUAL="$(php -r 'echo hash_file("sha256", $argv[1]);' "$COMPOSER_PHAR")" + +if [ "$COMPOSER_ACTUAL" != "$COMPOSER_CHECKSUM" ]; then + echo "The downloaded Composer does not match the checksum in build/runtime.json." >&2 + echo " expected: $COMPOSER_CHECKSUM" >&2 + echo " actual: $COMPOSER_ACTUAL" >&2 + rm -f "$COMPOSER_PHAR" + exit 1 +fi + +echo "==> Composer $COMPOSER_VERSION verified" + if [ "$SKIP_SPC" -eq 0 ]; then if [ ! -x "$WORK/spc" ]; then echo "==> Downloading static-php-cli" @@ -107,6 +140,7 @@ echo "==> Building the executable" php -d phar.readonly=0 "$ROOT/bin/build-phar.php" \ --micro="$MICRO" \ --runtime="$RUNTIME" \ + --composer="$COMPOSER_PHAR" \ ${BUILD_ID:+--build="$BUILD_ID"} echo "==> Restoring development dependencies" diff --git a/bin/build-phar.php b/bin/build-phar.php index c94a904f..33f45bed 100644 --- a/bin/build-phar.php +++ b/bin/build-phar.php @@ -18,7 +18,8 @@ | | Usage: | php -d phar.readonly=0 bin/build-phar.php \ -| --micro=path/to/micro.sfx --runtime=path/to/php [--output=builds/hyde] [--build=sha] +| --micro=path/to/micro.sfx --runtime=path/to/php --composer=path/to/composer.phar \ +| [--output=builds/hyde] [--build=sha] | */ @@ -64,7 +65,7 @@ function main(array $argv): int $options = parseOptions($argv); - foreach (['micro', 'runtime'] as $required) { + foreach (['micro', 'runtime', 'composer'] as $required) { if (! isset($options[$required]) || ! is_file($options[$required])) { fwrite(STDERR, "Missing or unreadable --$required.\n"); @@ -79,13 +80,18 @@ function main(array $argv): int info('Platform', $platform->slug()); info('Micro SAPI', $options['micro'].' ('.filesize($options['micro']).' bytes)'); info('PHP runtime', $options['runtime']); + info('Composer', $options['composer']); guardAgainstDevelopmentDependencies(); guardAgainstAPublishedFramework(); - $version = embedRuntime($options['runtime'], $options['micro'], $platform); + $version = embedRuntime($options['runtime'], $platform); + $composer = embedComposer($options['composer'], $options['runtime']); + + writeRuntimeManifest($version, $platform, $options['runtime'], $options['micro'], $composer); info('Runtime version', $version); + info('Composer version', $composer['version']); writeBuildMetadata($options['build'] ?? null); @@ -138,8 +144,8 @@ function guardAgainstAPublishedFramework(): void info('Framework', 'HydePHP v3 (develop@master)'); } -/** Copy the PHP CLI runtime into the source tree and describe it for the RuntimeManager. */ -function embedRuntime(string $runtime, string $micro, Platform $platform): string +/** Copy the PHP CLI runtime into the source tree, ready to be packed into the archive. */ +function embedRuntime(string $runtime, Platform $platform): string { $directory = ROOT.'/'.RuntimeManager::RUNTIME_DIRECTORY; @@ -153,23 +159,114 @@ function embedRuntime(string $runtime, string $micro, Platform $platform): strin fail("Unable to create $directory"); } - $target = $directory.'/'.$platform->runtimeFilename().RuntimeManager::RUNTIME_SUFFIX; - // The runtime is gzipped on the way in rather than compressed by the PHAR itself: // compressing several thousand small entries is slow and buys nothing, while // compressing this one large binary halves the size of the executable. - $source = fopen($runtime, 'rb'); - $compressed = fopen($target, 'wb'); + compress($runtime, $directory.'/'.$platform->runtimeFilename().RuntimeManager::RUNTIME_SUFFIX); + + return runtimeVersion($runtime); +} + +/** + * Copy Composer into the source tree, beside the runtime that will run it. + * + * Composer is what makes a Composer project usable on a machine that has none: the + * launcher's `hyde composer` runs this archive with the bundled PHP. The version is + * read by running it, rather than taken from the build configuration, so a build + * cannot record a Composer version that its own runtime is unable to start — and, + * since that happens after patching, cannot ship a patch that broke the archive. + * + * The archive handed to us is the published one, already verified against the pinned + * checksum. It is copied before anything is done to it, so what the build scripts + * cache stays byte-identical to what getcomposer.org served. + * + * @return array{version: string, filename: string, checksum: string, patches: list, upstream: array{sha256: string}} + */ +function embedComposer(string $composer, string $runtime): array +{ + $directory = ROOT.'/'.RuntimeManager::RUNTIME_DIRECTORY; + $shipped = ROOT.'/builds/'.RuntimeManager::COMPOSER_FILE; + + if (! copy($composer, $shipped)) { + fail("Unable to copy $composer to $shipped"); + } + + $patches = patchComposer($shipped); + + compress($shipped, $directory.'/'.RuntimeManager::COMPOSER_FILE.RuntimeManager::RUNTIME_SUFFIX); + + return [ + 'version' => composerVersion($shipped, $runtime), + 'filename' => RuntimeManager::COMPOSER_FILE, + + // The checksum of what is actually shipped, which the RuntimeManager verifies on + // every extraction, and the checksum of what upstream published, which is what + // the pin in build/runtime.json is about. Both, so neither claim is implied. + 'checksum' => hash_file('sha256', $shipped), + 'patches' => $patches, + 'upstream' => ['sha256' => hash_file('sha256', $composer)], + ]; +} + +/** + * Apply the patches in bin/lib/composer-patches.php to the archive we are about to ship. + * + * Composer signs its archive with SHA-512 rather than a key, so PHP can re-sign what it + * rewrites. A patch that no longer matches exactly once fails the build: a Composer + * version that moved the code must not be shipped silently unpatched, which is the + * one outcome worse than carrying the patch at all. + * + * @return list The names of the patches applied. + */ +function patchComposer(string $archive): array +{ + $patches = require ROOT.'/bin/lib/composer-patches.php'; + + if ($patches === []) { + info('Composer patches', 'none'); - stream_filter_append($compressed, 'zlib.deflate', STREAM_FILTER_WRITE, ['level' => 9, 'window' => 31]); - stream_copy_to_stream($source, $compressed); + return []; + } + + $phar = new Phar($archive); - fclose($source); - fclose($compressed); + $phar->startBuffering(); + + foreach ($patches as $name => $patch) { + if (! isset($phar[$patch['file']])) { + fail("The Composer patch $name expects {$patch['file']}, which this Composer does not have. See {$patch['issue']}."); + } + + $source = $phar[$patch['file']]->getContent(); + $found = substr_count($source, $patch['search']); + + if ($found !== 1) { + fail(sprintf( + "The Composer patch %s no longer applies: it matched %d times in %s, expected exactly 1. +". + "Check whether %s has been fixed upstream — if it has, delete the patch; if it has not, update it.", + $name, $found, $patch['file'], $patch['issue'] + )); + } - $version = runtimeVersion($runtime); + $phar[$patch['file']] = str_replace($patch['search'], $patch['replace'], $source); + + info('Composer patch', $name.' ('.$patch['summary'].')'); + } + + // Composer's own signature no longer covers what we changed, so the archive is + // re-signed with the algorithm it already used. + $phar->setSignatureAlgorithm(Phar::SHA512); + + $phar->stopBuffering(); + + return array_keys($patches); +} - file_put_contents($directory.'/'.RuntimeManager::MANIFEST_FILE, json_encode([ +/** Describe the embedded runtime, and the Composer beside it, for the RuntimeManager. */ +function writeRuntimeManifest(string $version, Platform $platform, string $runtime, string $micro, array $composer): void +{ + file_put_contents(ROOT.'/'.RuntimeManager::RUNTIME_DIRECTORY.'/'.RuntimeManager::MANIFEST_FILE, json_encode([ 'version' => $version, 'platform' => $platform->slug(), 'filename' => $platform->runtimeFilename(), @@ -178,9 +275,22 @@ function embedRuntime(string $runtime, string $micro, Platform $platform): strin // Where the application archive begins once it is concatenated onto the micro // SAPI binary. Verified against the archive's own marker at runtime. 'payload_offset' => filesize($micro), + + 'composer' => $composer, ], JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES)."\n"); +} - return $version; +/** Gzip a file into the runtime directory, inflated again by the RuntimeManager on first use. */ +function compress(string $source, string $target): void +{ + $in = fopen($source, 'rb'); + $out = fopen($target, 'wb'); + + stream_filter_append($out, 'zlib.deflate', STREAM_FILTER_WRITE, ['level' => 9, 'window' => 31]); + stream_copy_to_stream($in, $out); + + fclose($in); + fclose($out); } function runtimeVersion(string $runtime): string @@ -196,6 +306,18 @@ function runtimeVersion(string $runtime): string return $version; } +/** Run the bundled Composer with the bundled runtime, and read the version it reports. */ +function composerVersion(string $composer, string $runtime): string +{ + [$status, $output] = run([$runtime, $composer, '--version', '--no-ansi'], captureErrors: true); + + if ($status !== 0 || preg_match('/Composer version (\S+)/', $output, $matches) !== 1) { + fail("Unable to run the bundled Composer at $composer with the runtime at $runtime:\n".trim($output)); + } + + return $matches[1]; +} + /** * Run a program and capture what it printed, without going through a shell. * diff --git a/bin/lib/composer-patches.php b/bin/lib/composer-patches.php new file mode 100644 index 00000000..0fec76a5 --- /dev/null +++ b/bin/lib/composer-patches.php @@ -0,0 +1,69 @@ + Schannel + * + * with no `/version` after it. So the capture runs past the end of that line and + * on to the next `/` in the block — the one in `libSSH Version => libssh2/1.11.1` + * — and Composer builds a package called + * + * lib-curl-schannel\nZLib Version => 1.3.2\nlibSSH Version => libssh2 + * + * which fails its own name validation and aborts *every* dependency resolution. + * Every Windows build of static-php-cli uses Schannel: it compiles curl with + * `-DCURL_USE_SCHANNEL=ON -DCURL_USE_OPENSSL=OFF`, and that is not configurable. + * + * The fix is one character class: the library capture stops at a line boundary. The + * Schannel line then simply does not match, which is the correct outcome — it + * advertises no SSL library version, so there is no `lib-curl-` version to + * register. `ext-curl` and `lib-curl` are both registered before this code runs and + * are unaffected. + * + * The version capture is deliberately left as upstream wrote it. `.` already cannot + * cross a newline, so it was never part of the bug — and it absorbs the carriage + * return of a CRLF line, which is what lets `$` match there. Narrowing it to + * `[^\r\n]+` looks tidier and stops the pattern matching CRLF input at all. + * + * Reported as composer/composer#12615, which was closed by 47cde53 — a commit that + * fixed the *later* report on that issue (macOS SecureTransport with LibreSSL, + * where a slash is present and only the naming was wrong) and left the original + * Schannel case untouched. Still present on Composer's main branch. + */ + 'composer-12615' => [ + 'file' => 'src/Composer/Repository/PlatformRepository.php', + 'search' => '{^SSL Version => (?[^/]+)/(?.+)$}im', + 'replace' => '{^SSL Version => (?[^\r\n/]+)/(?.+)$}im', + 'issue' => 'https://github.com/composer/composer/issues/12615', + 'summary' => "curl's SSL backend is parsed across line boundaries, which breaks every install on Windows", + ], + +]; diff --git a/build/runtime.json b/build/runtime.json index dc85b74d..60ce6c98 100644 --- a/build/runtime.json +++ b/build/runtime.json @@ -15,13 +15,20 @@ "pcntl": "Required by symfony/process for signal handling when running `hyde serve` child processes. POSIX only.", "phar": "Required to run the embedded application: the executable is a micro SAPI bound to a PHAR.", "posix": "Required by symfony/console and laravel/prompts for TTY detection and interactive prompts. POSIX only.", + "session": "Required at install time by illuminate/session, which hyde/framework reaches through torchlight/torchlight-laravel and illuminate/http. The bundled Composer resolves platform requirements against the runtime it is running on, so without this every Hyde Composer project `hyde new --composer` creates is unresolvable.", "simplexml": "Required by hyde/framework for sitemap and RSS feed generation.", "tokenizer": "Required by the Blade compiler in illuminate/view.", "xml": "Required as the base extension for dom, simplexml, xmlreader and xmlwriter.", "xmlreader": "Required as a dependency of the XML extension family used by feed generation.", "xmlwriter": "Required by hyde/framework for sitemap and RSS feed generation.", + "zip": "Required by the bundled Composer to extract dist packages. Without it Composer falls back to an `unzip` binary, and then to `git`, neither of which a machine that installed Hyde to avoid installing PHP can be assumed to have: `hyde composer install` fails outright there.", "zlib": "Required to read the GZ-compressed PHAR that the executable embeds." }, + "composer": { + "$comment": "The Composer release bundled inside the executable, so that `hyde composer` and `hyde new --composer` work on a machine that has neither PHP nor Composer. Both build scripts download exactly this version and refuse to continue unless it hashes to the checksum below, which was taken from the one getcomposer.org publishes alongside that release. The executable distributes this, so keep it current: read the release notes before bumping it, and check a project can still be created with it. The archive is patched after verification and before it is embedded; see bin/lib/composer-patches.php for what is changed and why.", + "version": "2.10.2", + "sha256": "5ee7125f8a30a34d246cefdc0bc85b8a783b28f2aec968994118512350d28027" + }, "unsupported-extensions": { "$comment": "Extensions that cannot be built for a platform, dropped from that platform's build rather than left to fail the environment check. No PHP on Windows has ever had pcntl or posix, so nothing that runs there can be relying on them: symfony/process and symfony/console take their Windows code paths instead. Read by both build scripts, so neither can drift from the other on what it builds.", "windows": ["pcntl", "posix"] diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 2e980d6d..d177a670 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -92,20 +92,61 @@ called from the [`hyde`](../hyde) entry point **before** the embedded autoloader registered. The launcher classes are `require`d by explicit path for exactly that reason, and depend on nothing but the PHP standard library. -Three commands belong to the executable rather than to a project, and are answered even -inside a Composer project: +Some commands belong to the executable rather than to a project, and are answered even +inside a Composer project. They come in two kinds, and the difference is what answers them. + +**Answered by the embedded application** — `Launcher::LAUNCHER_COMMANDS`: | Command | Why | | --- | --- | -| `info` | It reports on the environment, including which project it found. | | `new` | It creates a project that does not exist yet. | +| `info` | It reports on the environment, including which project it found. | | `self-update` | It updates the executable itself. | +When one of these runs inside somebody else's Composer project, the embedded application is +pointed at a scratch directory rather than at the project root, so it can never read that +project's configuration or discover that project's packages. + +**Answered by the launcher, booting nothing** — `Launcher::RUNTIME_COMMANDS`: + +| Command | Why | +| --- | --- | +| `php` | It runs the PHP CLI bundled in the executable. | +| `composer` | It runs the Composer bundled in the executable, on that runtime. | + +Neither falls back to a program found on the search path: they mean *the ones Hyde +supplies*. A source checkout embeds neither, so `hyde php` there runs the PHP process +already running the code and `hyde composer` fails, telling the developer to run their +own. `hyde new --composer` is the one place that does fall back to a host Composer, +because it needs some Composer to create a project at all. + Everything else in a Composer project is dispatched into that project. -When a launcher-owned command runs inside somebody else's Composer project, the embedded -application is pointed at a scratch directory rather than at the project root, so it can -never read that project's configuration or discover that project's packages. +### The bundled programs + +The executable carries a complete PHP CLI and a Composer, because it needs both: one to +serve a site and to run a project's own entry point, the other to install a project's +dependencies. `hyde php` and `hyde composer` hand them to the user as well. + +[`App\Launcher\RuntimeDispatcher`](../app/Launcher/RuntimeDispatcher.php) runs them, and it +is the *launcher* that calls it, for two reasons that are not stylistic: + +- **Arguments have to arrive exactly as typed.** A console application claims `-v`, + `--version` and `--help` for itself before any command runs, so `hyde php -v` answered by + a console command could not print PHP's version. The launcher takes everything after the + command name — after the command, not after the program, so `hyde -v php -r '...'` does + not leak the CLI's own option into PHP. +- **`hyde composer install` has to work where nothing else does.** A Composer project with + no `vendor/` is the state the launcher refuses to run anything in, and this is the + command that repairs it. So it is answered *before* the project is detected at all. + +`App\Commands\PhpCommand` and `App\Commands\ComposerCommand` are registered all the same, +so `hyde list` and `hyde help php` have something to describe. They are not the shipped +execution path, and the tests exercise the launcher rather than them. + +The command list renders these, and the launcher-owned commands above, in a section of +their own, with membership read from `Launcher::ownedCommands()` — the list cannot come to +disagree with the routing it describes. ### Self-dispatch @@ -130,7 +171,8 @@ hyde = micro.sfx ++ hyde.phar ├── vendor/ (the embedded dependency graph) └── runtime/ ├── php.gz (a full static PHP CLI, gzipped) - └── runtime.json (version, platform, checksum, offset) + ├── composer.phar.gz (Composer, patched and gzipped) + └── runtime.json (versions, platform, checksums, patches, offset) ``` `bin/build-native.sh` (POSIX) and `bin/build-native.ps1` (Windows) drive static-php-cli; @@ -160,6 +202,95 @@ aside rather than overwritten in place. besides the embedded one is the CLI process that is already running the code, which only happens in a source checkout, where no embedded runtime exists. +### Why Composer is embedded too + +The CLI could always *run* a Composer project on a machine with no PHP. Installing that +project's dependencies still needed a Composer that machine did not have, which left the +promise one step short. So the build embeds one. + +`build/runtime.json` pins the version to bundle and the SHA-256 it must hash to, taken +from the checksum getcomposer.org publishes alongside that release. Both build scripts +download that version and verify it against the pinned checksum before it goes anywhere +near the archive — on every build, cached download included, so the pin rather than the +download is what decides. It is then extracted like the runtime, verified against the +checksum of the decompressed file, and cached under its own version rather than under +the platform: + +``` +~/.cache/hyde/composer//composer.phar +``` + +Composer is a PHAR, so it is never executed on its own: the bundled runtime is named as the +program and Composer as its first argument. That is also what decides which PHP an install +runs against — the one this executable ships, never whatever a shebang would find. + +Two extensions are in the runtime for this, and the acceptance suites assert both are +present in the artifact rather than trusting the build configuration: + +- `ext-zip`. Without it Composer falls back to an `unzip` binary and then to `git`, and a + machine that installed Hyde to avoid installing PHP cannot be assumed to have either: + `composer install` fails outright there. +- `ext-session`. Composer resolves platform requirements against the runtime it is + running on, and `illuminate/session` — reached from `hyde/framework` through + `hydephp/torchlight-commonmark`, `torchlight/torchlight-laravel` and `illuminate/http` + — requires it. Without it every project `hyde new --composer` creates is unresolvable. + +**The executable distributes Composer, so the pin is a supply-chain decision, not a +convenience.** Keep it current with upstream's security releases; read the release notes +and confirm a project can still be created before bumping it. + +### The patch carried against Composer + +The bundled archive is not always the published one byte for byte. +[`bin/lib/composer-patches.php`](../bin/lib/composer-patches.php) carries the minimum +change needed where a Composer bug stops the CLI working on one of its platforms; the +build applies it *after* verifying the download against the pinned checksum, and records +the result: + +``` +published composer.phar ──verified against the pin──▶ patched ──▶ hashed into runtime.json + (provenance) (what runs, verified + on every extraction) +``` + +So the manifest carries both checksums — upstream's and ours — and `hyde info -v` names +every patch applied. Nothing about what is shipped is left to be discovered. + +One patch exists today. Composer parses curl's SSL backend with `[^/]+`, which matches +newlines, so a curl that reports `SSL Version => Schannel` — with no `/version` after it, +which is every static-php-cli Windows build — makes the capture run past the end of the +line to the next `/` further down the block. Composer then builds a platform package +called `lib-curl-schannel +ZLib Version => 1.3.2 +libSSH Version => libssh2`, rejects it +as an invalid name, and aborts **every** dependency resolution. `hyde composer install` +and `hyde new --composer` were both dead on Windows; the Windows acceptance run is what +caught it. + +The fix is one character class, so the library capture stops at a line boundary and the +Schannel line simply does not match — the right answer, since it advertises no version to +register. `ext-curl` and `lib-curl` are registered before that code runs and are +untouched. The version capture is left exactly as upstream wrote it: `.` cannot cross a +newline, so it was never part of the bug, and it absorbs the carriage return that lets +`$` match on a CRLF host. + +Reported as [composer/composer#12615](https://github.com/composer/composer/issues/12615), +which was closed by a commit that fixed the *later* report on that issue — macOS +SecureTransport with LibreSSL, where a slash is present and only the naming was wrong — +and left the original Schannel case untouched. + +**A patch is a liability, and is meant to be deleted.** The build fails if one no longer +matches exactly once, so a Composer bump cannot silently ship unpatched; the day a release +contains the fix, the entry goes and `tests/Unit/ComposerPatchTest.php` goes with it. +That test needs no Composer, no network and no artifact: it runs the pattern against the +extension info the platforms actually report, including the SecureTransport case upstream +did fix, which the patch must not take back away. + +`hyde composer self-update` is refused. The bundled Composer is versioned with the +executable and verified on every run, so an update to the extracted copy would be +repaired away by the next command that needed it — silently, since repairing a copy that +fails verification is exactly what the extraction does. + ### Finding the archive inside the executable A plain PHP CLI cannot open the combined binary as a PHAR — the PHAR extension rejects the @@ -269,9 +400,10 @@ and the tests that need it generate it or fail loudly rather than skipping. ### Runtime version and extensions -`build/runtime.json` is the single build configuration. It pins **PHP 8.4** and lists -every extension with the reason it is present. Nothing is compiled that is not needed: -static-php-cli's default extension set is not used. +`build/runtime.json` is the single build configuration. It pins **PHP 8.4**, lists every +extension with the reason it is present, and pins the Composer release to bundle along +with the checksum it must hash to. Nothing is compiled that is not needed: static-php-cli's +default extension set is not used. PHP 8.4 rather than 8.5 because 8.5 emits deprecation notices from dependencies in the current release line (`PDO::MYSQL_ATTR_SSL_CA` in `laravel-zero/foundation`), and the @@ -290,7 +422,8 @@ is what every PHP on Windows has always done. | --- | --- | | Bundled PHP runtime | 8.4.x | | PHP required on the user's machine | none | -| Composer required on the user's machine | none, except for `hyde new --composer` | +| Bundled Composer | the release pinned in `build/runtime.json` | +| Composer required on the user's machine | none | | Framework, Portable projects | the version embedded in the executable | | Framework, Composer projects | whatever the project declares | | Running the CLI from source | PHP 8.2 – 8.4 with the extensions in `build/runtime.json` | diff --git a/hyde b/hyde index e966548f..f5201532 100755 --- a/hyde +++ b/hyde @@ -27,7 +27,9 @@ require_once __DIR__.'/app/Launcher/Platform.php'; require_once __DIR__.'/app/Launcher/ComposerManifest.php'; require_once __DIR__.'/app/Launcher/ProjectDetector.php'; require_once __DIR__.'/app/Launcher/RuntimeManager.php'; +require_once __DIR__.'/app/Launcher/Subprocess.php'; require_once __DIR__.'/app/Launcher/ProjectDispatcher.php'; +require_once __DIR__.'/app/Launcher/RuntimeDispatcher.php'; require_once __DIR__.'/app/Launcher/Launcher.php'; $launcher = new App\Launcher\Launcher(); diff --git a/tests/Feature/AppServiceProviderTest.php b/tests/Feature/AppServiceProviderTest.php index 93d59a4b..c0836385 100644 --- a/tests/Feature/AppServiceProviderTest.php +++ b/tests/Feature/AppServiceProviderTest.php @@ -2,8 +2,10 @@ declare(strict_types=1); +use App\Commands\PhpCommand; use App\Commands\InfoCommand; use App\Commands\ServeCommand; +use App\Commands\ComposerCommand; use App\Commands\NewProjectCommand; use App\Commands\SelfUpdateCommand; use Tests\Support\TemporaryProject; @@ -13,12 +15,25 @@ $commands = $this->registeredCommands(); - expect($commands)->toHaveKeys(['info', 'new', 'self-update', 'serve']) + expect($commands)->toHaveKeys(['info', 'new', 'self-update', 'serve', 'php', 'composer']) ->and($commands['info'])->toBeInstanceOf(InfoCommand::class) ->and($commands['new'])->toBeInstanceOf(NewProjectCommand::class) ->and($commands['self-update'])->toBeInstanceOf(SelfUpdateCommand::class); }); +it('describes the programs it bundles, so they can be listed and helped', function () { + $this->boot(TemporaryProject::portable()); + + $commands = $this->registeredCommands(); + + // These are answered by the launcher, before the application exists. They are + // registered so `hyde list` and `hyde help php` have something to describe. + expect($commands['php'])->toBeInstanceOf(PhpCommand::class) + ->and($commands['composer'])->toBeInstanceOf(ComposerCommand::class) + ->and($commands['php']->getDescription())->toContain('bundled PHP') + ->and($commands['composer']->getDescription())->toContain('bundled Composer'); +}); + it('overrides the realtime compiler serve command with its own', function () { $this->boot(TemporaryProject::portable()); diff --git a/tests/Feature/CommandListTest.php b/tests/Feature/CommandListTest.php new file mode 100644 index 00000000..7d1bda6e --- /dev/null +++ b/tests/Feature/CommandListTest.php @@ -0,0 +1,84 @@ +boot(TemporaryProject::portable()); + + expect($this->runCommand('list'))->toBe(0); + + $this->rendered = $this->consoleOutput(); +}); + +it('gives the commands the executable owns a section of their own', function () { + expect($this->rendered) + ->toContain('HYDE CLI') + ->toContain('the executable itself, in any directory') + ->toContain('PROJECT') + ->toContain('the Hyde site in the current directory'); +}); + +it('lists every command the launcher answers in that section', function () { + [$cli] = listSections($this->rendered); + + foreach (Launcher::ownedCommands() as $command) { + expect($cli)->toContain($command); + } +}); + +it('lists the programs the executable bundles with the commands it owns', function () { + [$cli] = listSections($this->rendered); + + expect($cli) + ->toContain('Run the bundled PHP CLI.') + ->toContain('Run the bundled Composer.'); +}); + +it('leaves the project commands out of the CLI section', function () { + [$cli, $project] = listSections($this->rendered); + + expect($cli)->not->toContain('build') + ->and($project)->toContain('build') + ->and($project)->toContain('make:page') + ->and($project)->toContain('route:list'); +}); + +it('puts the command that creates a project first', function () { + // `new` creates the project the rest of the list acts on, so it leads. + [$cli] = listSections($this->rendered); + + expect(strpos($cli, 'new'))->toBeLessThan(strpos($cli, 'self-update')); +}); + +it('names the program in the usage line', function () { + expect($this->rendered)->toContain('USAGE: '.ARTISAN_BINARY.' '); +}); + +it('does not list the list command itself', function () { + // It is the command being run, and the configuration hides it. `route:list` is + // matched by the same words, so this asks about the start of a line. + expect($this->rendered)->not->toMatch('/^ list\s/m'); +}); diff --git a/tests/Feature/NewProjectCommandTest.php b/tests/Feature/NewProjectCommandTest.php index a72b4d65..e3b4f5eb 100644 --- a/tests/Feature/NewProjectCommandTest.php +++ b/tests/Feature/NewProjectCommandTest.php @@ -3,6 +3,7 @@ declare(strict_types=1); use App\Support\ComposerBinary; +use App\Launcher\RuntimeManager; use Tests\Support\TemporaryProject; /* @@ -106,6 +107,60 @@ ->and(scandir($workspace))->toBe(['.', '..']); }); +/* +|-------------------------------------------------------------------------- +| The Composer that gets used +|-------------------------------------------------------------------------- +*/ + +it('creates the project with the composer bundled in the executable', function () { + $workspace = TemporaryProject::directory('workspace'); + + // An executable carrying a Composer, and no PHP binary of its own: the runtime then + // resolves to the process already running, exactly as a source checkout does. + $root = TemporaryProject::directory('bundled'); + + mkdir($root.'/'.RuntimeManager::RUNTIME_DIRECTORY); + + $composer = ' PHP_VERSION, + 'checksum' => '', + 'composer' => ['version' => '2.8.12', 'filename' => RuntimeManager::COMPOSER_FILE, 'checksum' => hash('sha256', $composer)], + ])); + + $this->boot($workspace); + + $this->app->instance(RuntimeManager::class, new RuntimeManager(null, $root)); + + expect($this->runCommand('new', ['name' => 'my-site', '--composer' => true, '--no-interaction' => true]))->toBe(0) + ->and($this->consoleOutput()) + ->toContain('Using the Composer bundled with this executable (2.8.12)') + ->toContain('the bundled composer ran') + ->and($workspace.'/my-site/composer.json')->toBeFile(); +}); + +it('falls back to the host composer when the executable bundles none', function () { + $workspace = TemporaryProject::directory('workspace'); + $fake = TemporaryProject::directory('fake-composer'); + + file_put_contents($fake.'/composer', "#!/bin/sh\nmkdir -p \"$3\"\necho 'the host composer ran'\nexit 0\n"); + + chmod($fake.'/composer', 0755); + + ComposerBinary::$fake = $fake.'/composer'; + + $this->boot($workspace); + + expect($this->runCommand('new', ['name' => 'my-site', '--composer' => true, '--no-interaction' => true]))->toBe(0) + ->and($this->consoleOutput()) + ->toContain('the host composer ran') + ->not->toContain('Using the Composer bundled'); +})->skipOnWindows(); + /* |-------------------------------------------------------------------------- | Propagating a Composer failure diff --git a/tests/Integration/NativeExecutableTest.php b/tests/Integration/NativeExecutableTest.php index 5ccdaa62..91adbbcc 100644 --- a/tests/Integration/NativeExecutableTest.php +++ b/tests/Integration/NativeExecutableTest.php @@ -143,21 +143,38 @@ ->and($parent.'/my-site/vendor')->not->toBeDirectory(); }); -it('refuses to create a composer project when Composer is missing, without touching the filesystem', function () { - $parent = TemporaryProject::directory('workspace'); +it('runs the bundled Composer on a machine that has none', function () { + $directory = TemporaryProject::directory('workspace'); $empty = TemporaryProject::directory('empty-bin'); - $result = Executable::run( - ['new', 'my-site', '--composer', '--no-ansi', '--no-interaction'], - $parent, - ['PATH' => $empty] - ); + $result = Executable::run(['composer', '--version', '--no-ansi'], $directory, ['PATH' => $empty]); - expect($result['status'])->not->toBe(0) - ->and($result['output']) - ->toContain('Creating a Composer project requires Composer.') - ->toContain('hyde new NAME --portable') - ->and($parent.'/my-site')->not->toBeDirectory(); + expect($result['status'])->toBe(0) + ->and($result['output'])->toContain('Composer version'); +}); + +it('runs the bundled PHP runtime as a command', function () { + $directory = TemporaryProject::directory('workspace'); + $empty = TemporaryProject::directory('empty-bin'); + + $result = Executable::run(['php', '-r', 'echo "runtime ", PHP_SAPI;'], $directory, ['PATH' => $empty]); + + expect($result['status'])->toBe(0) + // Not the micro SAPI the executable itself runs on: a real CLI came out of it. + ->and($result['output'])->toContain('runtime cli'); +}); + +it('answers a bundled program inside a composer project it refuses to build', function () { + // The state invariant 2 exists for is the state `hyde composer install` repairs, so + // the command has to be answerable there. `hyde build` still must not be. + $path = TemporaryProject::composer(['_pages/index.md' => "# Broken\n"], withAutoloader: false); + + expect(Executable::run(['build', '--no-ansi'], $path)['status'])->not->toBe(0); + + $result = Executable::run(['composer', '--version', '--no-ansi'], $path); + + expect($result['status'])->toBe(0) + ->and($result['output'])->toContain('Composer version'); }); /* diff --git a/tests/Integration/NewComposerProjectTest.php b/tests/Integration/NewComposerProjectTest.php index 896d66b5..0c2626c9 100644 --- a/tests/Integration/NewComposerProjectTest.php +++ b/tests/Integration/NewComposerProjectTest.php @@ -2,7 +2,6 @@ declare(strict_types=1); -use App\Support\ComposerBinary; use Tests\Support\Executable; use Tests\Support\ProjectTemplate; use Tests\Support\TemporaryProject; @@ -17,10 +16,12 @@ | no evidence of v3 support at all, so the command is pointed at a local v3 | source through `HYDE_PROJECT_SOURCE` and the result is inspected. | -| This runs the real executable and a real Composer install. Only the absence of -| a built artifact skips it, which is what the whole Integration suite does; a -| missing template or a missing Composer is a hard failure, because a green run -| that quietly left this out would be worse than a red one. +| This runs the real executable and a real Composer install, on the Integration +| suite's scrubbed search path: no PHP and no Composer. The Composer that does +| the work is therefore the one inside the executable. Only the absence of a +| built artifact skips it, which is what the whole Integration suite does; a +| missing template is a hard failure, because a green run that quietly left +| this out would be worse than a red one. | */ @@ -28,10 +29,6 @@ if (Executable::path() === null) { $this->markTestSkipped(Executable::missingMessage()); } - - // Not a skip: the suite is run with Composer-installed dev dependencies, so a host - // without Composer is a broken environment rather than an unsupported one. - expect(ComposerBinary::locate())->not->toBeNull('Composer is required to run the integration suite.'); }); it('creates a project running the v3 development dependency graph', function () { @@ -41,11 +38,14 @@ $result = Executable::run( ['new', 'my-site', '--composer', '--no-ansi', '--no-interaction'], $workspace, - ['HYDE_PROJECT_SOURCE' => $template, 'PATH' => getenv('PATH')], + ['HYDE_PROJECT_SOURCE' => $template], ); expect($result['status'])->toBe(0) - ->and($result['output'])->toContain('Created a Hyde Composer project'); + ->and($result['output']) + ->toContain('Created a Hyde Composer project') + // On this search path there is no Composer to find, so this is the bundled one. + ->toContain('Using the Composer bundled with this executable'); $lock = json_decode((string) file_get_contents($workspace.'/my-site/composer.lock'), true); @@ -75,7 +75,7 @@ Executable::run( ['new', 'my-site', '--composer', '--no-ansi', '--no-interaction'], $workspace, - ['HYDE_PROJECT_SOURCE' => $template, 'PATH' => getenv('PATH')], + ['HYDE_PROJECT_SOURCE' => $template], ); file_put_contents($workspace.'/my-site/_pages/index.md', <<<'MD' diff --git a/tests/System/acceptance.ps1 b/tests/System/acceptance.ps1 index 830d4256..87fb37cc 100644 --- a/tests/System/acceptance.ps1 +++ b/tests/System/acceptance.ps1 @@ -17,6 +17,10 @@ if (-not (Test-Path $Hyde)) { exit 2 } +# Almost every check runs the executable from inside a directory it just made, so a +# relative path has to be resolved before the first one of those changes location. +$Hyde = (Resolve-Path $Hyde).Path + $script:Checks = 0 $script:Failures = 0 @@ -79,6 +83,83 @@ try { $version = Invoke-Hyde $work @('--version', '--no-ansi') Assert-Contains 'hyde --version works' $version.Output 'HydePHP' + Write-Host '==> The bundled PHP runtime' + + # The point of these checks is that they run on a host with no PHP: the interpreter + # they exercise can only be the one inside the executable. + + $phpVersion = Invoke-Hyde $work @('php', '-v') + Assert-Contains 'hyde php -v reports a PHP version' $phpVersion.Output 'PHP 8.' + Assert-Contains 'the runtime is the static build' $phpVersion.Output 'static-php-cli' + + # No quotes inside the snippet: how PowerShell passes an embedded quote to a native + # program depends on which PowerShell is running it, and the check is not about that. + $phpEval = Invoke-Hyde $work @('php', '-r', 'echo 40 + 2;') + Assert-Contains 'hyde php -r evaluates code' $phpEval.Output '42' + + Set-Content -Path (Join-Path $work 'probe.php') -Value ' The bundled Composer' + + $composerVersion = Invoke-Hyde $work @('composer', '--version', '--no-ansi') + Assert-Contains 'hyde composer runs the bundled Composer' $composerVersion.Output 'Composer version' + + # A manifest with nothing in it: enough to prove Composer runs and writes an install, + # without making this suite depend on the network or on any package staying published. + $install = Join-Path $work 'install' + New-Item -ItemType Directory -Force -Path $install | Out-Null + Set-Content -Path (Join-Path $install 'composer.json') -Value '{"name":"acme/empty","require":{}}' + + $installResult = Invoke-Hyde $install @('composer', 'install', '--no-interaction', '--no-ansi') + + if ($installResult.Status -eq 0) { + Pass 'hyde composer install succeeds' + } else { + Fail 'hyde composer install succeeds' $installResult.Output + } + + if (Test-Path (Join-Path $install 'vendor\autoload.php')) { + Pass 'hyde composer install writes an autoloader' + } else { + Fail 'hyde composer install writes an autoloader' + } + + # The bundled Composer is versioned with the executable, and is checksum-verified on + # every run: an update to the extracted copy would be repaired away by the next + # command that needs it. + $selfUpdate = Invoke-Hyde $work @('composer', 'self-update', '--no-ansi') + + if ($selfUpdate.Status -eq 0) { + Fail 'hyde composer self-update is refused' 'it reported success' + } else { + Pass 'hyde composer self-update is refused' + } + + Assert-Contains 'the refusal says what to do instead' $selfUpdate.Output 'hyde self-update' + Write-Host '==> Portable project' $site = Join-Path $work 'site' @@ -188,19 +269,21 @@ try { $newBuild = Invoke-Hyde (Join-Path $workspace 'my-site') @('build', '--no-ansi') Assert-Contains 'the new project builds immediately' $newBuild.Output 'Your static site has been built!' - Write-Host '==> hyde new --composer without Composer' + Write-Host '==> hyde new --composer with no Composer on the host' + # The command no longer needs a Composer on the machine: it runs the one inside the + # executable. What that Composer then resolves depends on the network and on what is + # published, so this checks which Composer was used rather than what it installed. $composerAttempt = Invoke-Hyde $workspace @('new', 'composer-site', '--composer', '--no-ansi', '--no-interaction') - if ($composerAttempt.Status -eq 0) { - Fail 'hyde new --composer fails without Composer' 'it reported success' + Assert-Contains 'hyde new --composer uses the bundled Composer' $composerAttempt.Output 'Using the Composer bundled with this executable' + + if ($composerAttempt.Output -like '*Creating a Composer project requires Composer.*') { + Fail 'hyde new --composer no longer needs a Composer on the host' } else { - Pass 'hyde new --composer fails without Composer' + Pass 'hyde new --composer no longer needs a Composer on the host' } - Assert-Contains 'the failure explains what to do' $composerAttempt.Output 'Creating a Composer project requires Composer.' - Assert-Missing 'no directory is left behind' (Join-Path $workspace 'composer-site') - Write-Host '==> Project detection' $unrelated = Join-Path $work 'unrelated' @@ -227,6 +310,12 @@ try { Assert-Contains 'the failure names composer install' $brokenBuild.Output 'composer install' Assert-Missing 'nothing was built' (Join-Path $broken '_site') + # The command that repairs that project has to be answerable *in* it. It is handled + # before the project is detected at all, so the state that stops `hyde build` does + # not stop the command that fixes it. + $brokenComposer = Invoke-Hyde $broken @('composer', '--version', '--no-ansi') + Assert-Contains 'hyde composer answers inside a project with no vendor' $brokenComposer.Output 'Composer version' + Write-Host '==> Serving' $listener = [System.Net.Sockets.TcpListener]::new([System.Net.IPAddress]::Loopback, 0) diff --git a/tests/System/acceptance.sh b/tests/System/acceptance.sh index c2a5f7dc..a0aab52f 100755 --- a/tests/System/acceptance.sh +++ b/tests/System/acceptance.sh @@ -18,6 +18,14 @@ if [ -z "$HYDE" ] || [ ! -x "$HYDE" ]; then exit 2 fi +# Almost every check runs the executable from inside a directory it just made, so a +# relative path — `builds/hyde-linux-x86_64`, as it would be typed — has to be resolved +# before the first one of those changes directory out from under it. +case "$HYDE" in + /*) ;; + *) HYDE="$(cd "$(dirname "$HYDE")" && pwd)/$(basename "$HYDE")" ;; +esac + WORK="$(mktemp -d)" FAILURES=0 CHECKS=0 @@ -96,6 +104,79 @@ fi VERSION_OUTPUT="$("$HYDE" --version --no-ansi 2>&1)" assert_contains "hyde --version works" "$VERSION_OUTPUT" "HydePHP" +echo "==> The bundled PHP runtime" + +# The point of these checks is that they run on a host with no PHP: the interpreter +# they exercise can only be the one inside the executable. + +PHP_VERSION_OUTPUT="$("$HYDE" php -v 2>&1)" +assert_contains "hyde php -v reports a PHP version" "$PHP_VERSION_OUTPUT" "PHP 8." +assert_contains "the runtime is the static build" "$PHP_VERSION_OUTPUT" "static-php-cli" + +PHP_EVAL_OUTPUT="$("$HYDE" php -r 'echo "evaluated ", 1 + 1;' 2>&1)" +assert_contains "hyde php -r evaluates code" "$PHP_EVAL_OUTPUT" "evaluated 2" + +printf ' "$WORK/probe.php" + +PHP_SCRIPT_OUTPUT="$(cd "$WORK" && "$HYDE" php probe.php 2>&1)" +assert_contains "hyde php runs a script relative to the working directory" "$PHP_SCRIPT_OUTPUT" "script ran in $(basename "$WORK")" + +"$HYDE" php -r 'exit(3);' >/dev/null 2>&1 && PHP_STATUS=0 || PHP_STATUS=$? +if [ "$PHP_STATUS" -eq 3 ]; then + pass "hyde php propagates the exit status" +else + fail "hyde php propagates the exit status" "expected 3, got $PHP_STATUS" +fi + +# The extensions the bundled Composer cannot work without, asked of the runtime that is +# actually inside this artifact. Without zip, Composer falls back to an `unzip` binary +# this machine has no reason to have; without session, every Hyde project it resolves +# is unresolvable. A build that lost either would otherwise still pass every check +# above, and fail on the first real install a user attempted. +"$HYDE" php -r 'exit(extension_loaded("zip") && extension_loaded("session") ? 0 : 1);' >/dev/null 2>&1 && EXT_STATUS=0 || EXT_STATUS=$? +if [ "$EXT_STATUS" -eq 0 ]; then + pass "the runtime carries the extensions Composer needs" +else + fail "the runtime carries the extensions Composer needs" "$("$HYDE" php -r 'echo "zip: ", var_export(extension_loaded("zip"), true), ", session: ", var_export(extension_loaded("session"), true);' 2>&1)" +fi + +echo "==> The bundled Composer" + +COMPOSER_VERSION_OUTPUT="$("$HYDE" composer --version --no-ansi 2>&1)" +assert_contains "hyde composer runs the bundled Composer" "$COMPOSER_VERSION_OUTPUT" "Composer version" + +# A manifest with nothing in it: enough to prove Composer runs and writes an install, +# without making this suite depend on the network or on any package staying published. +INSTALL="$WORK/install" +mkdir -p "$INSTALL" +printf '{"name":"acme/empty","require":{}}' > "$INSTALL/composer.json" + +INSTALL_OUTPUT="$(cd "$INSTALL" && "$HYDE" composer install --no-interaction --no-ansi 2>&1)" && INSTALL_STATUS=0 || INSTALL_STATUS=$? + +if [ "$INSTALL_STATUS" -eq 0 ]; then + pass "hyde composer install succeeds" +else + fail "hyde composer install succeeds" "$INSTALL_OUTPUT" +fi + +if [ -f "$INSTALL/vendor/autoload.php" ]; then + pass "hyde composer install writes an autoloader" +else + fail "hyde composer install writes an autoloader" +fi + +# The bundled Composer is versioned with the executable, and is checksum-verified on +# every run: an update to the extracted copy would be repaired away by the next command. +"$HYDE" composer self-update --no-ansi >/dev/null 2>&1 && SELF_UPDATE_STATUS=0 || SELF_UPDATE_STATUS=$? + +if [ "$SELF_UPDATE_STATUS" -eq 0 ]; then + fail "hyde composer self-update is refused" "it reported success" +else + pass "hyde composer self-update is refused" +fi + +assert_contains "the refusal says what to do instead" "$("$HYDE" composer self-update --no-ansi 2>&1 || true)" "hyde self-update" + echo "==> Portable project" SITE="$WORK/site" @@ -134,6 +215,9 @@ assert_contains "info reports a v3 framework version" "$INFO_OUTPUT" "3.0.0-dev" LIST_OUTPUT="$(cd "$SITE" && "$HYDE" list --no-ansi 2>&1)" +assert_contains "the list separates the CLI's own commands" "$LIST_OUTPUT" "HYDE CLI" +assert_contains "the list separates the project's commands" "$LIST_OUTPUT" "PROJECT" + if printf '%s' "$LIST_OUTPUT" | grep -q 'rebuild'; then fail "the rebuild command v3 removed is absent" else @@ -174,18 +258,21 @@ assert_missing "the new project has no vendor directory" "$NEW/my-site/vendor" NEW_BUILD="$(cd "$NEW/my-site" && "$HYDE" build --no-ansi 2>&1)" assert_contains "the new project builds immediately" "$NEW_BUILD" "Your static site has been built!" -echo "==> hyde new --composer without Composer" +echo "==> hyde new --composer with no Composer on the host" -COMPOSER_OUTPUT="$(cd "$NEW" && "$HYDE" new composer-site --composer --no-ansi --no-interaction 2>&1)" && COMPOSER_STATUS=0 || COMPOSER_STATUS=$? +# The command no longer needs a Composer on the machine: it runs the one inside the +# executable. What that Composer then resolves depends on the network and on what is +# published, so this checks which Composer was used rather than what it installed. +COMPOSER_OUTPUT="$(cd "$NEW" && "$HYDE" new composer-site --composer --no-ansi --no-interaction 2>&1)" || true -if [ "$COMPOSER_STATUS" -eq 0 ]; then - fail "hyde new --composer fails without Composer" "it reported success" -else - pass "hyde new --composer fails without Composer" -fi +assert_contains "hyde new --composer uses the bundled Composer" "$COMPOSER_OUTPUT" "Using the Composer bundled with this executable" -assert_contains "the failure explains what to do" "$COMPOSER_OUTPUT" "Creating a Composer project requires Composer." -assert_missing "no directory is left behind" "$NEW/composer-site" +case "$COMPOSER_OUTPUT" in + *"Creating a Composer project requires Composer."*) + fail "hyde new --composer no longer needs a Composer on the host" ;; + *) + pass "hyde new --composer no longer needs a Composer on the host" ;; +esac echo "==> Project detection" @@ -213,6 +300,12 @@ fi assert_contains "the failure names composer install" "$BROKEN_OUTPUT" "composer install" assert_missing "nothing was built" "$BROKEN/_site" +# The command that repairs that project has to be answerable *in* it. It is handled +# before the project is detected at all, so the state that stops `hyde build` does +# not stop the command that fixes it. +BROKEN_COMPOSER_OUTPUT="$(cd "$BROKEN" && "$HYDE" composer --version --no-ansi 2>&1)" +assert_contains "hyde composer answers inside a project with no vendor" "$BROKEN_COMPOSER_OUTPUT" "Composer version" + echo "==> Serving" PORT=8${$} diff --git a/tests/Unit/ComposerPatchTest.php b/tests/Unit/ComposerPatchTest.php new file mode 100644 index 00000000..d73d058c --- /dev/null +++ b/tests/Unit/ComposerPatchTest.php @@ -0,0 +1,125 @@ + */ +function composerPatches(): array +{ + return require dirname(__DIR__, 2).'/bin/lib/composer-patches.php'; +} + +/** + * The platform package Composer would name, given a pattern and a block of `php --ri curl`. + * + * This is what PlatformRepository does with the match: the library goes into the + * package name, lowercased. + */ +function curlSslPackage(string $pattern, string $info): ?string +{ + return preg_match($pattern, $info, $matches) === 1 + ? 'lib-curl-'.strtolower($matches['library']) + : null; +} + +/** + * What a curl built against Schannel reports. Every static-php-cli Windows build does: + * it compiles curl with `-DCURL_USE_SCHANNEL=ON -DCURL_USE_OPENSSL=OFF`. + * + * Line endings are what PHP actually emits here, which the URL in the upstream report + * confirms: `lib-curl-schannel%0Azlib version` — one LF, no carriage return. + */ +function schannelCurlInfo(string $break = "\n"): string +{ + return implode($break, [ + 'cURL support => enabled', + 'cURL Information => 8.15.0', + 'Age => 11', + 'Host => x86_64-pc-win32', + 'SSL Version => Schannel', + 'ZLib Version => 1.3.2', + 'libSSH Version => libssh2/1.11.1', + '', + ]); +} + +it('is a patch against a Composer bug that is still present upstream', function () { + $patch = composerPatches()['composer-12615']; + + expect($patch['file'])->toBe('src/Composer/Repository/PlatformRepository.php') + ->and($patch['issue'])->toContain('composer/composer/issues/12615'); +}); + +it('changes one character class and nothing else', function () { + // The narrowest patch that fixes the bug is the one to carry: every character that + // differs from the release is a character somebody has to re-check on every bump. + $patch = composerPatches()['composer-12615']; + + expect(str_replace('[^\r\n/]', '[^/]', $patch['replace']))->toBe($patch['search']); +}); + +it('writes the escapes into the patched source rather than the characters they stand for', function () { + // A double-quoted string here would put a real carriage return inside Composer's + // source. It would even work, and it would be indefensible to read. + $patch = composerPatches()['composer-12615']; + + expect($patch['replace'])->toContain('\r\n') + ->and($patch['replace'])->not->toContain("\r") + ->and($patch['replace'])->not->toContain("\n"); +}); + +it('names a package out of three lines of curl info before the patch', function () { + // The bug, stated as the thing it produces. Composer rejects this name as invalid + // and aborts the resolution, so no install of any kind can run on Windows. + $package = curlSslPackage(composerPatches()['composer-12615']['search'], schannelCurlInfo()); + + expect($package)->toBe("lib-curl-schannel\nzlib version => 1.3.2\nlibssh version => libssh2"); +}); + +it('names no ssl backend package at all after the patch', function (string $break) { + // Schannel advertises no version, so there is no `lib-curl-` version to + // register. Not matching is the right answer: `ext-curl` and `lib-curl` are both + // registered earlier and are untouched by this. + $package = curlSslPackage(composerPatches()['composer-12615']['replace'], schannelCurlInfo($break)); + + expect($package)->toBeNull(); +})->with(['unix line endings' => ["\n"], 'windows line endings' => ["\r\n"]]); + +it('still reads an ssl backend that reports one', function (string $line, string $expected) { + $info = "cURL support => enabled\nHost => x86_64-linux-gnu\n$line\nZLib Version => 1.3.1\nlibSSH Version => libssh2/1.11.1\n"; + + expect(curlSslPackage(composerPatches()['composer-12615']['replace'], $info))->toBe($expected); +})->with([ + 'openssl' => ['SSL Version => OpenSSL/3.5.4', 'lib-curl-openssl'], + 'libressl' => ['SSL Version => LibreSSL/3.3.6', 'lib-curl-libressl'], + // The case upstream did fix, in the commit that closed the issue this patch is for. + // The patch must not take it back away. + 'securetransport with libressl' => ['SSL Version => (SecureTransport) LibreSSL/2.8.3', 'lib-curl-(securetransport) libressl'], +]); + +it('reads the same version the unpatched pattern reads', function (string $break) { + // The version capture is untouched, so whatever upstream made of a working platform + // is what this still makes of it — carriage return and all, on a CRLF host. + $patch = composerPatches()['composer-12615']; + $info = "SSL Version => OpenSSL/3.5.4{$break}ZLib Version => 1.3.1{$break}"; + + preg_match($patch['search'], $info, $before); + preg_match($patch['replace'], $info, $after); + + expect($after['version'])->toBe($before['version']) + ->and(trim($after['version']))->toBe('3.5.4'); +})->with(['unix line endings' => ["\n"], 'windows line endings' => ["\r\n"]]); diff --git a/tests/Unit/Launcher/LauncherTest.php b/tests/Unit/Launcher/LauncherTest.php index 4587849e..7e8478df 100644 --- a/tests/Unit/Launcher/LauncherTest.php +++ b/tests/Unit/Launcher/LauncherTest.php @@ -7,6 +7,7 @@ use App\Launcher\ProjectType; use App\Launcher\ProjectDetector; use App\Launcher\ProjectDispatcher; +use App\Launcher\RuntimeDispatcher; use Tests\Support\TemporaryProject; /* @@ -29,6 +30,24 @@ [['hyde', '-v', 'route:list', '--json'], 'route:list'], ]); +it('does not mistake the value of a global option for the command', function (array $argv, ?string $expected) { + // `--env` takes its value as the next token, so that token is not a command name. + // Reading it as one would route the call to a command nobody typed. + expect((new Launcher())->commandName($argv))->toBe($expected); +})->with([ + 'separate value' => [['hyde', '--env', 'development', 'php'], 'php'], + 'joined value' => [['hyde', '--env=development', 'php'], 'php'], + 'separate value, then a project command' => [['hyde', '--env', 'development', 'build'], 'build'], + 'among other options' => [['hyde', '--no-ansi', '--env', 'development', '-v', 'composer'], 'composer'], + 'nothing after the value' => [['hyde', '--env', 'development'], null], + 'no value to take' => [['hyde', '--env'], null], +]); + +it('takes what follows a bare double dash as the command', function () { + expect((new Launcher())->commandName(['hyde', '--', 'php']))->toBe('php') + ->and((new Launcher())->commandName(['hyde', '--']))->toBeNull(); +}); + it('keeps the CLI-owned commands for the executable', function (string $command) { expect((new Launcher())->isLauncherCommand(['hyde', $command]))->toBeTrue(); })->with(['info', 'new', 'self-update']); @@ -37,6 +56,26 @@ expect((new Launcher())->isLauncherCommand(['hyde', $command]))->toBeFalse(); })->with(['build', 'serve', 'route:list', 'make:page', 'test:addon']); +it('keeps the bundled programs for the executable', function (string $command) { + expect((new Launcher())->isRuntimeCommand(['hyde', $command]))->toBeTrue(); +})->with(Launcher::RUNTIME_COMMANDS); + +it('does not treat a project command as a bundled program', function (string $command) { + expect((new Launcher())->isRuntimeCommand(['hyde', $command]))->toBeFalse(); +})->with(['build', 'serve', 'info', 'new', 'self-update']); + +it('forwards everything typed after the command name', function (array $argv, array $expected) { + expect((new Launcher())->argumentsFor($argv))->toBe($expected); +})->with([ + 'nothing to forward' => [['hyde', 'php'], []], + 'an option for the program' => [['hyde', 'php', '-v'], ['-v']], + 'a script and its arguments' => [['hyde', 'php', 'script.php', '--flag', 'value'], ['script.php', '--flag', 'value']], + 'an option for the CLI is not forwarded' => [['hyde', '-v', 'php', '-r', 'echo 1;'], ['-r', 'echo 1;']], + 'nor is one that took a value' => [['hyde', '--env', 'development', 'php', '-r', 'echo 1;'], ['-r', 'echo 1;']], + 'nor is one that carried its value' => [['hyde', '--env=development', 'php', '-v'], ['-v']], + 'no command at all' => [['hyde', '--version'], []], +]); + /* |-------------------------------------------------------------------------- | Dispatching @@ -115,6 +154,49 @@ public function dispatch(Project $project, array $arguments = []): int ->and($dispatcher->called)->toBeFalse(); }); +it('answers a bundled program without dispatching, even in a broken composer project', function () { + // This is the whole point of answering these before detection: `hyde composer install` + // has to work in the one project state the launcher otherwise refuses to run in. + $path = TemporaryProject::composer(withAutoloader: false); + + $dispatcher = new class() extends ProjectDispatcher + { + public bool $called = false; + + public function dispatch(Project $project, array $arguments = []): int + { + $this->called = true; + + return 0; + } + }; + + $runtime = new class() extends RuntimeDispatcher + { + public ?string $command = null; + + /** @var list */ + public array $arguments = []; + + public function run(string $command, array $arguments = []): int + { + $this->command = $command; + $this->arguments = $arguments; + + return 4; + } + }; + + putenv("HYDE_WORKING_DIR=$path"); + + $status = (new Launcher(new ProjectDetector(), $dispatcher, $runtime))->run(['hyde', 'php', '-r', 'echo 1;']); + + expect($status)->toBe(4) + ->and($runtime->command)->toBe('php') + ->and($runtime->arguments)->toBe(['-r', 'echo 1;']) + ->and($dispatcher->called)->toBeFalse(); +}); + it('refuses to dispatch into itself', function () { // The CLI's own checkout is a Hyde Composer project whose entry point is the very // file that is running, and dispatching into it would recurse forever. diff --git a/tests/Unit/Launcher/RuntimeDispatcherTest.php b/tests/Unit/Launcher/RuntimeDispatcherTest.php new file mode 100644 index 00000000..ad027128 --- /dev/null +++ b/tests/Unit/Launcher/RuntimeDispatcherTest.php @@ -0,0 +1,145 @@ +php(['-r', 'exit(0);']); + + expect($status)->toBe(0); +}); + +it('returns the exit status the runtime returned', function () { + expect((new RuntimeDispatcher())->php(['-r', 'exit(9);']))->toBe(9); +}); + +it('forwards the arguments to the runtime untouched', function () { + $directory = TemporaryProject::directory('runtime'); + + file_put_contents($file = $directory.'/arguments.php', 'php([$file, $written = $directory.'/arguments.txt', '--flag', 'a b', '-v']); + + expect(file_get_contents($written))->toBe('--flag|a b|-v'); +}); + +it('runs the runtime through the `php` command name', function () { + expect((new RuntimeDispatcher())->run('php', ['-r', 'exit(5);']))->toBe(5); +}); + +it('runs the bundled composer with the bundled runtime', function () { + // An application root carrying a Composer but no PHP binary of its own: the runtime + // then resolves to the CLI process already running, exactly as a source checkout + // does, and the Composer is the real extraction path rather than a stand-in. + $root = TemporaryProject::directory('bundled'); + + mkdir($root.'/'.RuntimeManager::RUNTIME_DIRECTORY); + + $composer = ' PHP_VERSION, + 'checksum' => '', + 'composer' => ['version' => '2.8.12', 'filename' => RuntimeManager::COMPOSER_FILE, 'checksum' => hash('sha256', $composer)], + ])); + + $dispatcher = new RuntimeDispatcher(new RuntimeManager(null, $root)); + + expect($dispatcher->composer(['install', '--no-dev']))->toBe(6); +}); + +it('says so when asked for a composer the executable does not have', function () { + $root = TemporaryProject::directory('bundled'); + + mkdir($root.'/'.RuntimeManager::RUNTIME_DIRECTORY); + + // A complete runtime manifest, describing no Composer: a build made without one. + file_put_contents($root.'/'.RuntimeManager::RUNTIME_DIRECTORY.'/'.RuntimeManager::MANIFEST_FILE, json_encode([ + 'version' => PHP_VERSION, + 'checksum' => '', + ])); + + expect(fn () => (new RuntimeDispatcher(new RuntimeManager(null, $root)))->composer(['install'])) + ->toThrow(LauncherException::class, 'does not bundle Composer'); +}); + +/* +|-------------------------------------------------------------------------- +| Composer updating itself +|-------------------------------------------------------------------------- +| +| The bundled Composer is versioned with the executable. Letting it replace its +| own extracted copy would appear to work and then be undone by the next run, +| which verifies what it finds against the checksum from the build. +| +*/ + +it('refuses to let the bundled composer update itself', function (array $arguments) { + $dispatcher = new class() extends RuntimeDispatcher + { + public bool $started = false; + + public function composer(array $arguments = []): int + { + $refused = $this->refuseSelfUpdate($arguments); + + if ($refused !== null) { + return $refused; + } + + $this->started = true; + + return 0; + } + }; + + // The status is what a script acts on, and it is not success. The explanation goes + // to standard error, where it cannot be mistaken for the output of a command. + expect($dispatcher->composer($arguments))->toBe(1) + ->and($dispatcher->started)->toBeFalse(); +})->with([ + 'the command' => [['self-update']], + 'its alias' => [['selfupdate']], + 'behind an option' => [['--no-ansi', 'self-update']], + 'with its own options' => [['self-update', '--rollback']], +]); + +it('lets every other composer command through', function (array $arguments) { + $dispatcher = new class() extends RuntimeDispatcher + { + public function refusal(array $arguments): ?int + { + return $this->refuseSelfUpdate($arguments); + } + }; + + expect($dispatcher->refusal($arguments))->toBeNull(); +})->with([ + 'install' => [['install']], + 'update' => [['update']], + 'nothing at all' => [[]], + 'only options' => [['--version']], + 'a package that reads like it' => [['require', 'acme/self-update']], + 'an option that took a separate value' => [['-d', '/some/path', 'install']], +]); + +it('refuses a program it does not bundle', function () { + expect(fn () => (new RuntimeDispatcher())->run('perl')) + ->toThrow(LauncherException::class, 'bundles no `perl` program'); +}); diff --git a/tests/Unit/Launcher/RuntimeManagerTest.php b/tests/Unit/Launcher/RuntimeManagerTest.php index ee8c30a5..ba35a4ff 100644 --- a/tests/Unit/Launcher/RuntimeManagerTest.php +++ b/tests/Unit/Launcher/RuntimeManagerTest.php @@ -21,7 +21,7 @@ /** * @return array{0: RuntimeManager, 1: string, 2: string} The manager, its application root, and the cache root. */ -function runtimeFixture(string $contents = "#!/bin/sh\necho runtime\n", ?string $checksum = null, ?Platform $platform = null): array +function runtimeFixture(string $contents = "#!/bin/sh\necho runtime\n", ?string $checksum = null, ?Platform $platform = null, string|false $composer = false): array { $root = TemporaryProject::directory('runtime-root'); $cache = TemporaryProject::directory('runtime-cache'); @@ -34,19 +34,96 @@ function runtimeFixture(string $contents = "#!/bin/sh\necho runtime\n", ?string file_put_contents($binary, gzencode($contents)); - file_put_contents($root.'/'.RuntimeManager::RUNTIME_DIRECTORY.'/'.RuntimeManager::MANIFEST_FILE, json_encode([ + $manifest = [ 'version' => '8.4.24', 'platform' => $platform->slug(), 'filename' => $platform->runtimeFilename(), 'checksum' => $checksum ?? hash('sha256', $contents), 'payload_offset' => 1024, - ])); + ]; + + if ($composer !== false) { + file_put_contents($root.'/'.RuntimeManager::RUNTIME_DIRECTORY.'/'.RuntimeManager::COMPOSER_FILE.RuntimeManager::RUNTIME_SUFFIX, gzencode($composer)); + + $manifest['composer'] = [ + 'version' => '2.8.12', + 'filename' => RuntimeManager::COMPOSER_FILE, + 'checksum' => hash('sha256', $composer), + ]; + } + + file_put_contents($root.'/'.RuntimeManager::RUNTIME_DIRECTORY.'/'.RuntimeManager::MANIFEST_FILE, json_encode($manifest)); putenv("HYDE_CACHE_DIR=$cache"); return [new RuntimeManager($platform, $root), $root, $cache]; } +/* +|-------------------------------------------------------------------------- +| The bundled Composer +|-------------------------------------------------------------------------- +| +| Composer is embedded and extracted exactly like the runtime, but it is a PHAR +| rather than a binary: it is keyed by its own version rather than by platform, +| and it is never marked executable, since the runtime is what runs it. +| +*/ + +it('extracts the bundled composer into a cache directory of its own', function () { + [$manager, , $cache] = runtimeFixture(composer: 'composerPath(); + + expect($path)->toBe($cache.'/hyde/composer/2.8.12/composer.phar') + ->and(file_get_contents($path))->toBe('composerPath(); + + expect($manager->composerPath())->toBe($first); +}); + +it('reports whether composer is bundled', function () { + [$with] = runtimeFixture(composer: 'hasBundledComposer())->toBeTrue() + ->and($with->composerVersion())->toBe('2.8.12') + ->and($without->hasBundledComposer())->toBeFalse() + ->and($without->composerVersion())->toBeNull(); +}); + +it('says so when the executable bundles no composer', function () { + [$manager] = runtimeFixture(); + + expect(fn () => $manager->composerPath()) + ->toThrow(LauncherException::class, 'does not bundle Composer'); +}); + +it('refuses a composer that does not match its checksum', function () { + [$manager, $root] = runtimeFixture(composer: ' $manager->composerPath()) + ->toThrow(LauncherException::class, 'The bundled Composer failed its checksum verification'); +}); + +it('repairs a composer extraction that was corrupted', function () { + [$manager, $root] = runtimeFixture(composer: 'composerPath(), 'corrupted'); + + // A later run of the CLI is a new process, and verifies what it finds on disk. + $fresh = new RuntimeManager(new Platform('Linux', 'x86_64'), $root); + + expect(file_get_contents($fresh->composerPath()))->toBe('toBe(0) + ->and(Subprocess::run([PHP_BINARY, '-r', 'exit(3);']))->toBe(3); +}); + +it('runs the program in the given working directory', function () { + $directory = realpath(sys_get_temp_dir()); + + $status = Subprocess::run([PHP_BINARY, '-r', 'exit(realpath(getcwd()) === realpath($argv[1]) ? 0 : 1);', $directory], $directory); + + expect($status)->toBe(0); +}); + +it('gives the program the environment it was handed', function () { + $status = Subprocess::run([PHP_BINARY, '-r', 'exit(getenv("HYDE_SUBPROCESS_PROBE") === "set" ? 0 : 1);'], null, ['HYDE_SUBPROCESS_PROBE' => 'set']); + + expect($status)->toBe(0); +}); + +/* +|-------------------------------------------------------------------------- +| The search path +|-------------------------------------------------------------------------- +*/ + +it('puts the bundled runtime at the front of the search path', function () { + $environment = Subprocess::environmentWith('/opt/hyde/runtime/php'); + + expect($environment[Subprocess::searchPathKey($environment)]) + ->toStartWith('/opt/hyde/runtime'.PATH_SEPARATOR); +}); + +it('finds the search path key whatever case the platform used', function () { + expect(Subprocess::searchPathKey(['Path' => 'C:\\Windows', 'HOME' => 'C:\\Users\\emma']))->toBe('Path') + ->and(Subprocess::searchPathKey(['PATH' => '/usr/bin']))->toBe('PATH') + ->and(Subprocess::searchPathKey(['path' => '/usr/bin']))->toBe('path') + ->and(Subprocess::searchPathKey(['HOME' => '/home/emma']))->toBe('PATH'); +}); diff --git a/tests/Unit/Support/ComposerBinaryTest.php b/tests/Unit/Support/ComposerBinaryTest.php index 7446adf0..cf5aa883 100644 --- a/tests/Unit/Support/ComposerBinaryTest.php +++ b/tests/Unit/Support/ComposerBinaryTest.php @@ -3,8 +3,15 @@ declare(strict_types=1); use App\Support\ComposerBinary; +use App\Launcher\RuntimeManager; use Tests\Support\TemporaryProject; +/** A runtime that bundles nothing, standing in for a source checkout. */ +function bareRuntime(): RuntimeManager +{ + return new RuntimeManager(null, TemporaryProject::directory('bare-runtime')); +} + afterEach(function () { ComposerBinary::$fake = null; ComposerBinary::$forceMissing = false; @@ -45,19 +52,82 @@ } })->skipOnWindows()->skip(fn () => posix_geteuid() === 0, 'Root can execute anything.'); -it('reports no composer when the search path is empty', function () { +it('reports no composer when the search path is empty and none is bundled', function () { $original = getenv('PATH'); putenv('PATH='); try { expect(ComposerBinary::locate())->toBeNull() - ->and(ComposerBinary::available())->toBeFalse(); + ->and(ComposerBinary::available(bareRuntime()))->toBeFalse() + ->and(ComposerBinary::command(bareRuntime()))->toBeNull(); } finally { putenv("PATH=$original"); } }); +/* +|-------------------------------------------------------------------------- +| Which Composer gets used +|-------------------------------------------------------------------------- +| +| The one inside the executable, whenever there is one: it is the release this +| build was made against, and it is there on a machine that has no Composer. +| +*/ + +it('prefers the bundled composer over one on the search path', function () { + $directory = TemporaryProject::directory('bin'); + + file_put_contents($directory.'/composer', "#!/bin/sh\nexit 0\n"); + chmod($directory.'/composer', 0755); + + $root = TemporaryProject::directory('bundled'); + + mkdir($root.'/'.RuntimeManager::RUNTIME_DIRECTORY); + + file_put_contents($root.'/'.RuntimeManager::RUNTIME_DIRECTORY.'/'.RuntimeManager::COMPOSER_FILE.RuntimeManager::RUNTIME_SUFFIX, gzencode(' PHP_VERSION, + 'checksum' => '', + 'composer' => ['version' => '2.8.12', 'filename' => RuntimeManager::COMPOSER_FILE, 'checksum' => hash('sha256', 'toBe([$runtime->path(), $runtime->composerPath()]) + ->and(ComposerBinary::bundled($runtime))->toBe(ComposerBinary::command($runtime)); + } finally { + putenv("PATH=$original"); + } +})->skipOnWindows(); + +it('falls back to the host composer when nothing is bundled', function () { + $directory = TemporaryProject::directory('bin'); + + file_put_contents($directory.'/composer', "#!/bin/sh\nexit 0\n"); + chmod($directory.'/composer', 0755); + + $original = getenv('PATH'); + + putenv("PATH=$directory"); + + try { + expect(ComposerBinary::bundled(bareRuntime()))->toBeNull() + ->and(ComposerBinary::command(bareRuntime()))->toBe([$directory.DIRECTORY_SEPARATOR.'composer']); + } finally { + putenv("PATH=$original"); + } +})->skipOnWindows(); + it('can be forced to report composer as missing', function () { ComposerBinary::$forceMissing = true; diff --git a/tests/Unit/Support/DescriberTest.php b/tests/Unit/Support/DescriberTest.php deleted file mode 100644 index 01a07bac..00000000 --- a/tests/Unit/Support/DescriberTest.php +++ /dev/null @@ -1,47 +0,0 @@ -assertSame(['new', 'aaa', 'bbb'], commandNames($commands)); -}); - -it('sorts the commands properly with different starting order', function () { - $commands = createCommandMocks(['new', 'aaa', 'bbb']); - - TestDescriber::sortCommandsInGroup($commands); - - $this->assertSame(['new', 'aaa', 'bbb'], commandNames($commands)); -}); - -function createCommandMocks(array $names): array -{ - return array_map(function (string $name): MockObject { - $command = test()->getMockBuilder(Command::class) - ->disableOriginalConstructor() - ->getMock(); - - $command->method('getName')->willReturn($name); - - return $command; - }, $names); -} - -function commandNames(array $commands): array -{ - return array_map(fn (Command $command): string => $command->getName(), $commands); -} - -class TestDescriber extends Describer -{ - public static function sortCommandsInGroup(array &$commands): void - { - parent::sortCommandsInGroup($commands); - } -}