diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 042bc85b..f99d55dd 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -7,3 +7,24 @@ name: 'Lint, Unit & Integration Tests' jobs: lint-unit: uses: test-kitchen/.github/.github/workflows/lint-unit.yml@main + + integration: + name: 'Integration / Ruby ${{ matrix.ruby }}' + runs-on: ubuntu-latest + needs: lint-unit + strategy: + fail-fast: false + matrix: + ruby: + - '3.1' + - '3.4' + steps: + - uses: actions/checkout@v7 + - uses: ruby/setup-ruby@v1 + with: + ruby-version: ${{ matrix.ruby }} + bundler-cache: true + # Drives the driver's real create/destroy code through Test Kitchen + # against fog-openstack's mock backend, so it needs no credentials and + # no OpenStack deployment. See CONTRIBUTING.md. + - run: bundle exec rake integration diff --git a/AGENTS.md b/AGENTS.md index 7b0b2071..36e85ac4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -28,12 +28,13 @@ kitchen-openstack is a Test Kitchen driver for OpenStack. It provisions and dest ```bash bundle install -bundle exec rake # runs tests + style (default) -bundle exec rake test # unit tests only (RSpec) -bundle exec rake style # Cookstyle lint -bundle exec rake quality # style -bundle exec rake yard # render YARD docs to doc/ (not CI-gated) -bundle exec rake yard_stats # list undocumented methods +bundle exec rake # runs tests + style (default) +bundle exec rake test # unit tests only (RSpec) +bundle exec rake integration # Test Kitchen suites against fog-openstack mocks +bundle exec rake style # Cookstyle lint +bundle exec rake quality # style +bundle exec rake yard # render YARD docs to doc/ (not CI-gated) +bundle exec rake yard_stats # list undocumented methods ``` ## Conventions @@ -47,4 +48,5 @@ bundle exec rake yard_stats # list undocumented methods - `verify_partial_doubles` is on. Fog *model* classes take `instance_double`; Fog *service* objects cannot, because Fog defines their methods dynamically at instantiation - Unit tests never sleep, hit the network, or read outside a `Dir.mktmpdir`. The one exemption is `openstack_version_spec.rb`, which reads the gemspec and the Release Please manifest to catch version drift, and skips when they are absent. `ENV` is replaced with an `OS_*`-free hash, and the clouds.yaml specs additionally pin `Dir.pwd`, `Dir.home` and `/etc/openstack`, so neither a developer's OpenStack environment nor their real `clouds.yaml`/`secure.yaml` can leak in - SimpleCov reports to `coverage/` with no enforced threshold +- Integration suites live in `kitchen.yml` and run against fog-openstack's mock backend; `test/fog_mock.rb` must be required before the `kitchen` executable (`ruby -Itest -r fog_mock -S kitchen test`). No credentials or cloud needed - Release automation via Release Please — version bumps go in `lib/kitchen/driver/openstack_version.rb` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f653a0e8..5c866f5a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -29,14 +29,17 @@ bundle install ## Rake tasks ```bash -bundle exec rake # tests + lint (default) -bundle exec rake test # unit tests only -bundle exec rake style # Cookstyle lint -bundle exec rake yard # render docs to doc/ -bundle exec rake yard_stats # list undocumented methods +bundle exec rake # tests + lint (default) +bundle exec rake test # unit tests only +bundle exec rake integration # Test Kitchen suites against fog's mocks +bundle exec rake style # Cookstyle lint +bundle exec rake yard # render docs to doc/ +bundle exec rake yard_stats # list undocumented methods ``` -`rake test` runs the `unit` task, and `rake quality` runs `style`. +`rake test` runs the `unit` task, and `rake quality` runs `style`. The default +task deliberately leaves `integration` out so that the common case stays fast; +CI runs it as its own job. To run a single spec file: @@ -77,9 +80,44 @@ undocumented. Neither is enforced in CI, but new methods should come with docs. When you add or change a configuration option, update the configuration reference in `README.md` as well. +## Integration tests + +`bundle exec rake integration` runs the suites in `kitchen.yml` through Test +Kitchen itself. They need no credentials and no OpenStack deployment: +`test/fog_mock.rb` puts Fog into mock mode before the driver is loaded, so +`create` and `destroy` run their real code against fog-openstack's in-memory +Nova, Neutron and Cinder. + +Fog has to be mocked before Test Kitchen instantiates the driver, which is why +the rake task requires the file ahead of the `kitchen` executable rather than +doing it from `kitchen.yml`: + +```bash +bundle exec ruby -Itest -r fog_mock -S kitchen test +bundle exec ruby -Itest -r fog_mock -S kitchen test network-ref-cirros +bundle exec ruby -Itest -r fog_mock -S kitchen diagnose --all +``` + +What this catches that the unit tests cannot: the driver failing to load or +register, a Driver API version mismatch, a file missing from the gemspec, a +config key that no longer survives `finalize_config!`, and any exception on the +real create/destroy path — each of which the unit tests can miss because they +never go through Test Kitchen's own plugin loading and instance lifecycle. + +What it cannot catch: anything only a real cloud decides. Quotas, scheduling, +whether the image actually boots, real networking, and SSH are all outside a +mock. There is no way to run those in hosted CI — they need a live OpenStack — +so they stay a manual step, below. + +A few paths are not covered as suites because fog's mocks do not model them: +allocating a *new* floating IP (the mock's response carries no address), +Cinder volumes (mock volumes never reach `available`), and reading an address +back off a named network (mock servers are not attached to the mock networks). +Those are covered by the unit tests, and by the manual pass. + ## Manual testing -The unit tests never contact a cloud, so changes that touch instance creation, +The tests above never contact a cloud, so changes that touch instance creation, networking, or credential resolution should also be exercised against a real OpenStack deployment. diff --git a/Rakefile b/Rakefile index 18cf778b..45e7f6d4 100644 --- a/Rakefile +++ b/Rakefile @@ -6,6 +6,13 @@ RSpec::Core::RakeTask.new(:unit) desc "Run all test suites" task test: [:unit] +desc "Run the Test Kitchen integration suites against fog-openstack's mocks" +task :integration do + # test/fog_mock.rb has to be loaded before Test Kitchen instantiates the + # driver, hence -r rather than anything inside kitchen.yml. + sh "bundle exec ruby -Itest -r fog_mock -S kitchen test" +end + desc "Run the unit tests with coverage reporting to coverage/" task :coverage do ENV["COVERAGE"] = "1" diff --git a/kitchen.yml b/kitchen.yml new file mode 100644 index 00000000..4801b78d --- /dev/null +++ b/kitchen.yml @@ -0,0 +1,104 @@ +--- +# Test Kitchen integration suites for this driver. +# +# These run against fog-openstack's mock backend rather than a real cloud, so +# they need no credentials and no OpenStack deployment. Put Fog into mock mode +# by requiring test/fog_mock.rb ahead of the kitchen executable: +# +# bundle exec rake integration +# bundle exec ruby -Itest -r fog_mock -S kitchen test network-ref-cirros +# +# Every suite drives the driver's real create and destroy code through Test +# Kitchen itself. What that catches, and what it cannot, is written up in +# CONTRIBUTING.md -- in short, anything a live cloud would reject (quotas, +# scheduling, real networking, SSH) still needs the manual pass described +# there. +# +# The image and flavor names below are the ones fog-openstack's mock Nova +# ships with. + +driver: + name: openstack + image_ref: cirros-0.3.0-x86_64-blank + flavor_ref: 512 server + openstack_auth_url: http://keystone.example.com:5000/v3 + openstack_username: kitchen + openstack_api_key: kitchen + openstack_domain_id: default + glance_cache_wait_timeout: 60 + +provisioner: + name: dummy + +transport: + name: dummy + +verifier: + name: dummy + +platforms: + - name: cirros + +suites: + # Boot and tear down with nothing but an image and a flavor set. + - name: default + + # image_ref and flavor_ref given as /regex/ rather than as exact names, + # which takes a different path through find_matching. + - name: image-flavor-regex + driver: + image_ref: /^cirros/ + flavor_ref: /512/ + + # image_id and flavor_id, which are used verbatim with no lookup at all. + # These are the ids the mock Nova assigns to the image and flavor named + # above; they are fixed constants in fog-openstack's mock data. + - name: image-flavor-id + driver: + image_ref: null + flavor_ref: null + image_id: 0e09fbd6-43c5-448a-83e9-0d3d05f9747e + flavor_id: '2' + + # A network resolved by name and turned into a nics entry. + - name: network-ref + driver: + network_ref: network_1 + + # Takes an already-allocated, unattached address out of a pool and + # associates it, then reads it back as the hostname. + - name: floating-ip-pool + driver: + floating_ip_pool: nova + + # A specific address, which short-circuits address resolution entirely. + - name: floating-ip + driver: + floating_ip: 192.168.27.130 + + # Server naming: a user-supplied prefix plus a random suffix. + - name: server-name-prefix + driver: + server_name_prefix: kitchen-ci + + # The optional server settings, all of which are resolved by optional_config. + - name: server-options + driver: + availability_zone: nova + security_groups: + - default + metadata: + kitchen: integration + config_drive: true + + # Inline cloud-config, which the driver renders to user_data itself. + - name: cloud-config + driver: + cloud_config: + packages: + - htop + + # user_data read from a file on disk. + - name: user-data + driver: + user_data: test/integration/cloud-init.yml diff --git a/test/fog_mock.rb b/test/fog_mock.rb new file mode 100644 index 00000000..2126c198 --- /dev/null +++ b/test/fog_mock.rb @@ -0,0 +1,32 @@ +# frozen_string_literal: true + +# +# Copyright:: (C) 2026, Oregon State University +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Puts Fog into mock mode before Test Kitchen loads the driver, so the +# integration suites in kitchen.yml run the driver's real create/destroy code +# against fog-openstack's in-memory Nova, Neutron and Cinder instead of a live +# cloud. +# +# Required ahead of the `kitchen` executable rather than from kitchen.yml, so +# that kitchen.yml stays an ordinary example file: +# +# bundle exec ruby -Itest -r fog_mock -S kitchen test +# +# See CONTRIBUTING.md for what this does and does not cover. + +require "fog/openstack" + +Fog.mock! diff --git a/test/integration/cloud-init.yml b/test/integration/cloud-init.yml new file mode 100644 index 00000000..dd286775 --- /dev/null +++ b/test/integration/cloud-init.yml @@ -0,0 +1,6 @@ +#cloud-config +# Fixture for the user-data integration suite. The mock Nova accepts the +# contents unread; the point of the suite is that the driver finds the file, +# reads it, and puts it on the server definition. +packages: + - htop