From 5fd8f085dc725c89f42bf5708fb504e11304b1ce Mon Sep 17 00:00:00 2001 From: Kurt Garloff Date: Thu, 23 Jan 2025 10:18:52 +0100 Subject: [PATCH 1/2] Explain SCS DigiSov and Certs. How to get into compliance. - Explain the motivation for the SCS certification levels rooted in the levels of digital sovereignty. - Give a practical example how to achieve the SCS certification by running the tests. - Hints on how to get listed, why we have the HM there and how to react to failed runs. Editorial note: - This was written up in January 2025 - It was not merged, as the description how to get certified is better done separately, elsewhere - Resurrect it, as the explanation of digital sovereigtny relative to SCS certification is useful - ToDo: Drop the duplicate pieces on testing and certification process Signed-off-by: Kurt Garloff --- .lintstagedrc.json | 2 +- sidebarsStandards.js | 5 +- standards/certification/digisov-and-cert.md | 121 +++++++++++++ .../getting-scs-compatible-certified.md | 113 ++++++++++++ .../certification/test-and-adapt-example.md | 165 ++++++++++++++++++ 5 files changed, 404 insertions(+), 2 deletions(-) create mode 100644 standards/certification/digisov-and-cert.md create mode 100644 standards/certification/getting-scs-compatible-certified.md create mode 100644 standards/certification/test-and-adapt-example.md diff --git a/.lintstagedrc.json b/.lintstagedrc.json index fc5bcf3c97..160cc6e73b 100644 --- a/.lintstagedrc.json +++ b/.lintstagedrc.json @@ -1,4 +1,4 @@ { "*": "prettier --ignore-unknown --write", - "*.md": "markdownlint-cli2-fix" + "*.md": "markdownlint-cli2" } diff --git a/sidebarsStandards.js b/sidebarsStandards.js index 27ec888089..68eb8f55ab 100644 --- a/sidebarsStandards.js +++ b/sidebarsStandards.js @@ -7,7 +7,7 @@ const sidebars = { label: 'Certification', link: { type: 'doc', - id: 'certification/overview' + id: 'certification/digisov-and-cert' }, items: [ { @@ -24,6 +24,9 @@ const sidebars = { label: 'Compliance Check Pipeline', id: 'certification/pipeline' } + 'certification/getting-scs-compatible-certified', + 'certification/test-and-adapt-example', + 'certification/overview' ] }, { diff --git a/standards/certification/digisov-and-cert.md b/standards/certification/digisov-and-cert.md new file mode 100644 index 0000000000..b2ccaa3226 --- /dev/null +++ b/standards/certification/digisov-and-cert.md @@ -0,0 +1,121 @@ +# Digital Sovereignty and SCS certification + +## The taxonomy of digital sovereignty + +As published in [DuD](https://rdcu.be/cWdBJ) (German, English version in +[the cloud report](https://the-report.cloud/why-digital-sovereignty-is-more-than-mere-legal-compliance/)) +and being summarized nicely in a [cloudahead article](https://www.cloudahead.de/der-freiheitskampf-des-sovereign-cloud-stacks), +we differentiate between several levels of digital sovereignty. +We'll skip stage 0, introduced by Gregor Schuhmacher in his description, which +specifies using a cloud at all as the pre-step to be taken. This has relevance, +as some companies continue to call solutions that are not on-demand, not +self-service API driven, not metered +(see [NIST definition of cloud](https://nvlpubs.nist.gov/nistpubs/legacy/sp/nistspecialpublication800-145.pdf)) +to be (private) clouds. We talk about real clouds, where deployment of infrastructure +is API-driven, unlocking DevOps teams productivity. + +The levels as seen by the SCS movement are: + +1. Control over data and data sharing and ability to fulfill regulatory requirements (GDPR) +2. Capability to chose between *highly compatible* operators, this way enabling a provider + switch or using several providers in a federated fashion. This also includes the + possibility to run your infrastructure in a *highly compatible* manner. +3. Capability to influence and shape the infrastructure, enabling innovation at the + infrastructure layer. +4. Transparency over operational aspects of running infrastructure, this way supporting + to overcome a skill gap to being able to operate infrastructure in a highly reliable + manner. + +These aspects of sovereignty drive the work from the SCS team. + +Level number 1 is sometimes referred to as "data sovereignty". Achieving it does require +cloud infrastructure and cloud operations that can not be interfered with by actors that +are outside of the respective jurisdiction. For Europeans that need to observe GDPR, this +excludes using US clouds for personally identifiable information, expecting that the +adequacy decisions for the US do not fully address the risks. The SCS project does not +have deep legal expertise and refers to the work from [noyb](https://noyb.eu/) +and [ENISA](https://www.enisa.europa.eu/) here. + +In order to achieve level 2, +the SCS community has worked on standards that define the APIs and the infrastructure +behavior, so application developers and application operators can deploy the same application +using the same automation and rely on the same infrastructure behavior to operate the +application in a resilient way. The standards allow for switching providers or to use +several providers in a federated way. Operating own infrastructure according to the same +standards is also possible, allowing for hybrid cloud setups without technical barriers. + +Level 3 drives the work on a comprehensive openly developed open source software stack, +allowing operators to use, study, change and redistribute the software according to the +[Four Freedoms](https://en.wikipedia.org/wiki/The_Free_Software_Definition) of free software. We are requiring +a complete stack that uses real open source licenses (as defined by [OSI](https://opensource.org/)) +as to ensure that users have the four freedoms, the right to use, study, modify, (re)distribute +the software that drives the cloud stack. To ensure that this does not require extensive +and expensive forking, we further require the [Four Opens](https://openinfra.dev/four-opens/) +of the Open Infra Foundation here. The software can be used to provide cloud services +for others (public cloud) or just for your own community (community cloud) or +internal (private cloud) needs. + +Level 4 addresses the skills and transparency aspects. Operating highly dynamic distributed +systems in a reliable manner requires knowledge and experience - engineers with these skills +are scarce. To address this, the SCS team networks operations staff from providers and helps +to share and distill common knowledge that help everyone to be more successful. SCS has +thus been driving the [Open Operations](https://openoperations.org) initiative. + +Levels 2 and 3 are sometimes related to the term "technological sovereignty", indicating +that the ability to control and shape the technology. + +## The SCS certification levels + +Corresponding to the levels of digital sovereignty in the SCS taxonomy, SCS defines +SCS certification levels + +1. (Defined outside of the SCS scope) +2. SCS-compatible +3. SCS-open +4. SCS-sovereign + +### Why no SCS certification for GDPR? + +SCS significantly lowers the bar to offer real cloud services. These can be used internally +(private cloud) or to offer services for your community, your region or country. The vision +is to have a network of providers. We expect most if not all of them to be operated in ways +that fulfill the European GDPR regulation; it is also possible to operate clouds that fulfill +special regulation, e.g. in the banking or insurance sector. + +SCS is not in a position to judge this and thus defines no own label / certificate to +vouch for regulatory compliance. We typically refer to the ENISA for GDPR considerations +and also recommend to take the Gaia-X labels into account here. + +## Status of SCS certification for cloud operators + +As of September 2024, we have not yet formalized the requirements for SCS-open and SCS-sovereign +certification. + +The technical compatibility validation corresponding to the SCS-compatible certification does +exist since more than a year. There are certificates for two layers of the SCS architecture +stack: + +* The virtualization layer: SCS-compatible IaaS +* The container layer: SCS-compatible KaaS + +For each of these, technical tests are being run to test service offerings for compliance. +The standards and the corresponding tests are versioned. The SCS-compatible certification +for a specific layer (currently IaaS or KaaS) and version is called a *certification scope*. +Please see [Scopes and Versions](scopes-versions.md) for detailed definitions. + +As of September 2024, the latest SCS-compatible certification scope on the IaaS layer is +SCS-compatible IaaS v4. For November 2024, SCS-compatible IaaS v5 and the first Kaas +scope SCS-compatible KaaS v1 are planned. + +## Certification for non-operators + +Software can deliver infrastructure components for operators to provide SCS-compatible +IaaS or KaaS; it is planned that infrastructure software can also receive SCS certification. + +Likewise, applications can be developed in a way that they will work without any changes on +all SCS-compatible IaaS or on all SCS-compatible KaaS (or may require both). It is planned +that such software can also be certified. + +Implementation partners from the SCS ecosystem may support operators (CSPs) to build +and operate SCS-compatible infrastructure. A certification program that certifies the +skills and experience of such partners is planned as well. diff --git a/standards/certification/getting-scs-compatible-certified.md b/standards/certification/getting-scs-compatible-certified.md new file mode 100644 index 0000000000..22bb2f633f --- /dev/null +++ b/standards/certification/getting-scs-compatible-certified.md @@ -0,0 +1,113 @@ +# Getting SCS-compatible certification (for operators) + +## Process overview + +The SCS Certification is a technical certification: +The Operator needs to fulfill technical requirements, such as providing certain +APIs and guaranteeing certain platform behavior in order to be certifiable. + +These requirements are meant to provide guarantees to their customers, allowing +them to rely on certain features to be available and on certain system behavior +that lets their applications run in a reliable way. + +The SCS certification process typically consists of a few simple steps: + +1. Running the SCS compliance test suite and adjusting the infrastructure until it passes. +2. Any additional declarations (for non-testable aspects) are written and passed to the SCS Forum. +3. The operator must be a member ("shaper" or "advisor" level) of the Forum SCS-Standards in the + OSB Alliance (a non-profit) and pay the respective membership fees. Alternatively fees can + be paid without becoming a member. +4. The cloud can be listed on the SCS pages as SCS-compatible with a compatibility status that is + updated on a daily basis. SCS then tests the infrastructure on a daily basis. + +## Self-testing and technical adjustments + +In order for a cloud service offering to obtain a certificate, it has to +conform to all standards of the respective scope, which will be tested at +regular intervals, and the results of these tests will be made available +publicly. For more details on how to become certified, please consult the +corresponding [document](/standards/scs-0004-v1-achieving-certification). + +The best approach to get your cloud into compliance is by installing the +test suite locally. Have a look at the [example](/standards/test-and-adapt-example). + +A description how SCS-compatible IaaS compliance can be achieved on environments that use different +OpenStack implementations is written up in a blog article +[Cost of making an OpenStack Cluster SCS compliant](https://scs.community/de/2024/05/13/cost-of-making-an-openstack-cluster-scs-compliant/). + +## Declarations + +For the SCS-compatible IaaS v5 standard, the providers must - if they implement availability zones +at all (which is optional) - guarantee certain levels of independence for these. This can not +be fully tested by an automated test. The process thus envisions that providers must create some +documentation on the physical infrastructure and how it maps to availability zones and declare that +this documentation reflects the truth. SCS will review the docs and judge whether they meet the +criteria. In case of doubt, audits can be performed. + +## Forum SCS-Standards @ OSBA + +The SCS brand belongs to the Open Source Business Alliance e.V. (OSBA), an non-profit organization and +association for the Open Source Industry in Germany. The OSBA creates the Forum SCS-Standards +which performs the work to evolve the SCS standards, develops the tests and perform the certification +process. + +Members of the OSBA can become also member of the Forum SCS-Standards for an additional membership +fee, providing the financial resources for the Forum to do its work. Membership in the OSBA is +open to any organization that supports the goals of the OSBA. +Alternatively, a certification fee can be paid without any membership; the fee corresponds to the +lower tier membership fee. + +## Getting listed and tested + +When all tests are passing, all needed declarations are done, fees for the certification or the +membership in the Forum SCS-Standards at the OSBA have been paid, the infrastructure service +can become officially certified. + +The SCS team will add the cloud to the [list of certified clouds](https://docs.scs.community/standards/certification/overview) +on the SCS docs page. This can be used to prove to customers that the cloud is SCS compliant. +Note that there will be a nightly job that tests the cloud for compliance, which will be +triggered by SCS infrastructure (zuul). For this, access to a tenant on the cloud needs +to be provided free of charge. (This only requires very low quota, one VM is created for a minute +in one of the tests.) +The list of certified clouds will be replaced by the +[compliance monitor](https://compliance.sovereignit.cloud/page/table) soon. + +For clouds not being accessible from the outside, a VPN tunnel or a local monitoring +job (with result upload) can be used. + +Next to the addition into the list, we also plan to create an SCS-certified badge that +operators will be allowed to use in their marketing material as long as they fulfill the +certification conditions. + +Note that for almost all certified clouds in the list of certified clouds, we also +have a health monitor running (currently still +[openstack-health-monitor](https://github.com/SovereignCloudStack/openstack-health-monitor/), +but soon the new [health-monitor](https://scs.community/de/tech/2024/09/06/vp12-scs-health-monitor-tech-preview/)), +which exposes information on the performance and error rate of each cloud. +This provides some transparency on the state of the clouds by constantly running +scenario tests against them and is tremendously helpful for both the cloud operations +teams and their customers. Strictly speaking, it is *not* a requirement for the +SCS-compatible certification, just best practice. It will be part of an +SCS-sovereign certification though, where transparency on operational aspects +is included. + +## Staying compliant + +Once your cloud is listed in the list of certified clouds or in the compliance manager, it +will enjoy the nightly tests. These might fail for a number of reasons: + +* There is a new version of the SCS standards in effect and you need to adjust things. +* Your cloud was unreachable or otherwise had intermittent issues. +* You have done changes to your cloud that break SCS-compatible compliance. +* The test automation engine (github actions / zuul) is in trouble. +* The tests have a bug. + +In either case, this need proper analysis to determine what should be done. +In the list of certified clouds, the tests are performed by github actions. +These are executed from the +[github SCS standards repository](https://github.com/SovereignCloudStack/standards). +By looking at the logs from the github actions, you can typically see why the failure +happened. You could of course also do a local test again to see if the issue can +be reproduced. +In the compliance manager, we will add links to the log files directly on the table, +so it will be even easier to find the relevant log. diff --git a/standards/certification/test-and-adapt-example.md b/standards/certification/test-and-adapt-example.md new file mode 100644 index 0000000000..8ab3a8a468 --- /dev/null +++ b/standards/certification/test-and-adapt-example.md @@ -0,0 +1,165 @@ +# Example: Testing adjustment for SCS IaaS-compatible compliance + +## Run the tests + +Get the test suite by cloning [the SCS standards repo](https://github.com/SovereignCloudStack/standards/). +In order to run the tests, you need to have normal customer (tenant) access to the cloud or +container infrastructure that you want to test. (This is by design; we explicitly do not +require nor recommend admin level access for normal compliance testing.) + +You can run the test suite from any machine that has a working `python3-openstacksdk` (for the +IaaS tests) or working `python3`, `kubectl` and `helm` (for the KaaS tests). Go to the +checked out tree into the `Tests/` directory to run tests. Check that the tooling works, +e.g. by issuing a command like `openstack --os-cloud=MYCLOUD catalog list` or +`KUBECONFIG=~/.kube/MYCLUSTER.yaml kubectl get nodes -o wide`. + +Let's do a run against a sample environment: + +```bash +garloff@framekurt(//):/casa/src/SCS/standards/Tests [1]$ ./scs-compliance-check.py -V v4 -s CIAB -a os_cloud=ciab-test scs-compatible-iaas.yaml +INFO: module opc-v2022.11 missing checks or test cases +DEBUG: Fetching flavors from cloud 'ciab-test' +DEBUG: Checking 28 flavor specs against 18 flavors +WARNING: Flavor 'SCS-4V-16' found via name only, missing property 'scs:name-v2' +ERROR: Flavor 'SCS-4V-16' violating property constraints: scs:cpu-type: None should be 'shared-core'; scs:name-v1: None should be 'SCS-4V:16'; scs:name-v2: None should be 'SCS-4V-16' +WARNING: Flavor 'SCS-8V-32' found via name only, missing property 'scs:name-v2' +ERROR: Flavor 'SCS-8V-32' violating property constraints: scs:cpu-type: None should be 'shared-core'; scs:name-v1: None should be 'SCS-8V:32'; scs:name-v2: None should be 'SCS-8V-32' +WARNING: Missing recommended flavor 'SCS-1V-4-10' +WARNING: Missing recommended flavor 'SCS-2V-8-20' +WARNING: Missing recommended flavor 'SCS-4V-16-50' +WARNING: Missing recommended flavor 'SCS-8V-32-100' +WARNING: Missing recommended flavor 'SCS-1V-2-5' +WARNING: Missing recommended flavor 'SCS-2V-4-10' +WARNING: Missing recommended flavor 'SCS-4V-8-20' +WARNING: Missing recommended flavor 'SCS-8V-16-50' +WARNING: Missing recommended flavor 'SCS-16V-32-100' +WARNING: Missing recommended flavor 'SCS-1V-8-20' +WARNING: Missing recommended flavor 'SCS-2V-16-50' +WARNING: Missing recommended flavor 'SCS-4V-32-100' +WARNING: Missing recommended flavor 'SCS-1L-1-5' +DEBUG: Total critical / error / info: 0 / 2 / 0 +DEBUG: Fetching image list from cloud 'ciab-test' +DEBUG: Images present: Cirros 0.6.1, Cirros 0.6.2, Debian 12, EVIL2, EVIL3, Ubuntu 22.04 Minimal, disk-bf.qcow2, disk-vmdk.vmdk, openSUSE 15.5, openSUSE 15.6 +DEBUG: Checking 6 image specs against 10 images +ERROR: Missing mandatory image 'Ubuntu 22.04' +WARNING: Missing recommended image 'ubuntu-capi-image' +DEBUG: Missing optional image 'Ubuntu 20.04' +DEBUG: Missing optional image 'Debian 11' +DEBUG: Missing optional image 'Debian 10' +DEBUG: Total critical / error / warning: 0 / 1 / 1 +******************************************************************************** +CIAB SCS-compatible IaaS v4 (effective): +- main: FAIL (3 passed, 2 failed) + - FAILED: + - standard-flavors-check: + > Must fulfill all requirements of https://docs.scs.community/standards/scs-0103-v1-standard-flavors + - standard-images-check: + > Must fulfill all requirements of https://docs.scs.community/standards/scs-0104-v1-standard-images +``` + +So we run the SCS-compatible IaaS tests defined in `scs-compatible-iaas.yaml` in version `v4`; without option `-V`, +the latest effective version would have been used. We further define the cloud to be named `CIAB` (short for +Cloud-in-a-Box) in the report. And we set the parameter `os_cloud` to `ciab-test`. This references the +name of the cloud as configured in OpenStack `clouds.yaml` and `secure.yaml` which contain the configuration +and credentials to access the cloud as tenant user via the API (SDK or CLI). + +Let's have a look at the results: + +* We seem to have all 15 mandatory compute flavors, but two of them miss mandatory properties (`extra_specs`). + We also receive 13 warnings for not having recommended flavors, we can ignore them for now. +* On the images side, the mandatory image `Ubuntu 22.04` is not registered. +* The end result is that we passed three tests and failed to comply with two specs: + +```yaml + - standard-flavors-check: + > Must fulfill all requirements of https://docs.scs.community/standards/scs-0103-v1-standard-flavors + - standard-images-check: + > Must fulfill all requirements of https://docs.scs.community/standards/scs-0104-v1-standard-images +``` + +With option `-v`, we can make the test suite more verbose to e.g. see that we pass the flavor naming test, +the entropy test and the image metadata test. + +## Address issues + +To fix the failures, we will thus need to: + +* Add properties to the two flavors where they are missing. +* Register the `Ubuntu 22.04` image (with the appropriate metadata). + +Neither is difficult to do manually, but a more systematic and automated process is preferable. +For the first issue, there is a [blog article on flavor metadata](https://scs.community/de/tech/2024/08/20/flavor-extra-specs-compliance/). +For the image registration, the [OpenStack Image Manager](https://github.com/osism/openstack-image-manager) can be used. + +For adjusting the environment, we of course do need admin access to the cloud. +We use the tools referenced above: + +```shell +garloff@framekurt(//):/casa/src/SCS/standards/Tests [3]$ OS_CLOUD=ciab-admin ./iaas/flavor-naming/flavor-add-extra-specs.py -a apply +INFO: Flavor SCS-8V-32: SET scs:cpu-type=shared-core +INFO: Flavor SCS-8V-32: SET scs:name-v1=SCS-8V:32 +INFO: Flavor SCS-8V-32: SET scs:name-v2=SCS-8V-32 +INFO: Flavor SCS-4V-16: SET scs:cpu-type=shared-core +INFO: Flavor SCS-4V-16: SET scs:name-v1=SCS-4V:16 +INFO: Flavor SCS-4V-16: SET scs:name-v2=SCS-4V-16 +INFO: Processed 15 flavors, 6 changes +``` + +and as this is a OSISM-based SCS system, we can on the manager just run the image manager: + +```shell +dragon@manager:~$ osism manage images --cloud admin --filter "Ubuntu 22.04" +2024-09-23 13:21:43 | INFO | Processing image 'Ubuntu 22.04 (20240705)' +2024-09-23 13:21:43 | INFO | Tested URL https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-images/ubuntu-22.04/20240705-ubuntu-22.04.qcow2: 200 +2024-09-23 13:21:43 | INFO | Importing image Ubuntu 22.04 (20240705) +2024-09-23 13:21:43 | INFO | Importing from URL https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-images/ubuntu-22.04/20240705-ubuntu-22.04.qcow2 +2024-09-23 13:21:44 | INFO | Waiting for image to leave queued state... +2024-09-23 13:21:46 | INFO | Waiting for import to complete... +2024-09-23 13:21:56 | INFO | Waiting for import to complete... +2024-09-23 13:22:06 | INFO | Waiting for import to complete... +2024-09-23 13:22:16 | INFO | Import of 'Ubuntu 22.04 (20240705)' successfully completed, reloading images +2024-09-23 13:22:17 | INFO | Checking parameters of 'Ubuntu 22.04 (20240705)' +2024-09-23 13:22:17 | INFO | Setting internal_version = 20240705 +2024-09-23 13:22:17 | INFO | Setting image_original_user = ubuntu +2024-09-23 13:22:17 | INFO | Adding tag os:ubuntu +2024-09-23 13:22:17 | INFO | Setting property architecture: x86_64 +2024-09-23 13:22:17 | INFO | Setting property hw_disk_bus: scsi +2024-09-23 13:22:17 | INFO | Setting property hw_rng_model: virtio +2024-09-23 13:22:17 | INFO | Setting property hw_scsi_model: virtio-scsi +2024-09-23 13:22:17 | INFO | Setting property hw_watchdog_action: reset +2024-09-23 13:22:17 | INFO | Setting property hypervisor_type: qemu +2024-09-23 13:22:17 | INFO | Setting property os_distro: ubuntu +2024-09-23 13:22:18 | INFO | Setting property os_version: 22.04 +2024-09-23 13:22:18 | INFO | Setting property replace_frequency: quarterly +2024-09-23 13:22:18 | INFO | Setting property uuid_validity: last-3 +2024-09-23 13:22:18 | INFO | Setting property provided_until: none +2024-09-23 13:22:18 | INFO | Setting property image_description: Ubuntu 22.04 +2024-09-23 13:22:18 | INFO | Setting property image_name: Ubuntu 22.04 +2024-09-23 13:22:18 | INFO | Setting property internal_version: 20240705 +2024-09-23 13:22:18 | INFO | Setting property image_original_user: ubuntu +2024-09-23 13:22:18 | INFO | Setting property image_source: https://cloud-images.ubuntu.com/jammy/20240705/jammy-server-cloudimg-amd64.img +2024-09-23 13:22:18 | INFO | Setting property image_build_date: 2024-07-05 +2024-09-23 13:22:18 | INFO | Checking status of 'Ubuntu 22.04 (20240705)' +2024-09-23 13:22:18 | INFO | Checking visibility of 'Ubuntu 22.04 (20240705)' +2024-09-23 13:22:18 | INFO | Setting visibility of 'Ubuntu 22.04 (20240705)' to 'public' +2024-09-23 13:22:19 | INFO | Renaming Ubuntu 22.04 (20240705) to Ubuntu 22.04 +2024-09-23 13:22:19 | INFO | Processing image 'Ubuntu 22.04 Minimal (20240701)' +dragon@manager:~$ +``` + +A description how SCS-compatible IaaS compliance can be achieved on environments that use different +OpenStack implementations is written up in a blog article +[Cost of making an OpenStack Cluster SCS compliant](https://scs.community/de/2024/05/13/cost-of-making-an-openstack-cluster-scs-compliant/). + +## Rerun tests + +We now succeed: + +```shell +garloff@framekurt(//):/casa/src/SCS/standards/Tests [130]$ ./scs-compliance-check.py -V v4 -s CIAB -a os_cloud=ciab-test scs-compatible-iaas.yaml +INFO: module opc-v2022.11 missing checks or test cases +CIAB SCS-compatible IaaS v4 (effective): +- main: PASS (5 passed) +``` + +If you don't pass the tests yet, you'll need further adjustments. From 37de60a2c0a0eb3e625fb322effd2acb94426bf7 Mon Sep 17 00:00:00 2001 From: Kurt Garloff Date: Fri, 10 Jul 2026 01:06:12 +0200 Subject: [PATCH 2/2] Revert .lintstagedrc, drop duplicated process docs. Signed-off-by: Kurt Garloff --- sidebarsStandards.js | 2 - .../getting-scs-compatible-certified.md | 113 ------------ .../certification/test-and-adapt-example.md | 165 ------------------ 3 files changed, 280 deletions(-) delete mode 100644 standards/certification/getting-scs-compatible-certified.md delete mode 100644 standards/certification/test-and-adapt-example.md diff --git a/sidebarsStandards.js b/sidebarsStandards.js index 68eb8f55ab..3e6f8e9f84 100644 --- a/sidebarsStandards.js +++ b/sidebarsStandards.js @@ -24,8 +24,6 @@ const sidebars = { label: 'Compliance Check Pipeline', id: 'certification/pipeline' } - 'certification/getting-scs-compatible-certified', - 'certification/test-and-adapt-example', 'certification/overview' ] }, diff --git a/standards/certification/getting-scs-compatible-certified.md b/standards/certification/getting-scs-compatible-certified.md deleted file mode 100644 index 22bb2f633f..0000000000 --- a/standards/certification/getting-scs-compatible-certified.md +++ /dev/null @@ -1,113 +0,0 @@ -# Getting SCS-compatible certification (for operators) - -## Process overview - -The SCS Certification is a technical certification: -The Operator needs to fulfill technical requirements, such as providing certain -APIs and guaranteeing certain platform behavior in order to be certifiable. - -These requirements are meant to provide guarantees to their customers, allowing -them to rely on certain features to be available and on certain system behavior -that lets their applications run in a reliable way. - -The SCS certification process typically consists of a few simple steps: - -1. Running the SCS compliance test suite and adjusting the infrastructure until it passes. -2. Any additional declarations (for non-testable aspects) are written and passed to the SCS Forum. -3. The operator must be a member ("shaper" or "advisor" level) of the Forum SCS-Standards in the - OSB Alliance (a non-profit) and pay the respective membership fees. Alternatively fees can - be paid without becoming a member. -4. The cloud can be listed on the SCS pages as SCS-compatible with a compatibility status that is - updated on a daily basis. SCS then tests the infrastructure on a daily basis. - -## Self-testing and technical adjustments - -In order for a cloud service offering to obtain a certificate, it has to -conform to all standards of the respective scope, which will be tested at -regular intervals, and the results of these tests will be made available -publicly. For more details on how to become certified, please consult the -corresponding [document](/standards/scs-0004-v1-achieving-certification). - -The best approach to get your cloud into compliance is by installing the -test suite locally. Have a look at the [example](/standards/test-and-adapt-example). - -A description how SCS-compatible IaaS compliance can be achieved on environments that use different -OpenStack implementations is written up in a blog article -[Cost of making an OpenStack Cluster SCS compliant](https://scs.community/de/2024/05/13/cost-of-making-an-openstack-cluster-scs-compliant/). - -## Declarations - -For the SCS-compatible IaaS v5 standard, the providers must - if they implement availability zones -at all (which is optional) - guarantee certain levels of independence for these. This can not -be fully tested by an automated test. The process thus envisions that providers must create some -documentation on the physical infrastructure and how it maps to availability zones and declare that -this documentation reflects the truth. SCS will review the docs and judge whether they meet the -criteria. In case of doubt, audits can be performed. - -## Forum SCS-Standards @ OSBA - -The SCS brand belongs to the Open Source Business Alliance e.V. (OSBA), an non-profit organization and -association for the Open Source Industry in Germany. The OSBA creates the Forum SCS-Standards -which performs the work to evolve the SCS standards, develops the tests and perform the certification -process. - -Members of the OSBA can become also member of the Forum SCS-Standards for an additional membership -fee, providing the financial resources for the Forum to do its work. Membership in the OSBA is -open to any organization that supports the goals of the OSBA. -Alternatively, a certification fee can be paid without any membership; the fee corresponds to the -lower tier membership fee. - -## Getting listed and tested - -When all tests are passing, all needed declarations are done, fees for the certification or the -membership in the Forum SCS-Standards at the OSBA have been paid, the infrastructure service -can become officially certified. - -The SCS team will add the cloud to the [list of certified clouds](https://docs.scs.community/standards/certification/overview) -on the SCS docs page. This can be used to prove to customers that the cloud is SCS compliant. -Note that there will be a nightly job that tests the cloud for compliance, which will be -triggered by SCS infrastructure (zuul). For this, access to a tenant on the cloud needs -to be provided free of charge. (This only requires very low quota, one VM is created for a minute -in one of the tests.) -The list of certified clouds will be replaced by the -[compliance monitor](https://compliance.sovereignit.cloud/page/table) soon. - -For clouds not being accessible from the outside, a VPN tunnel or a local monitoring -job (with result upload) can be used. - -Next to the addition into the list, we also plan to create an SCS-certified badge that -operators will be allowed to use in their marketing material as long as they fulfill the -certification conditions. - -Note that for almost all certified clouds in the list of certified clouds, we also -have a health monitor running (currently still -[openstack-health-monitor](https://github.com/SovereignCloudStack/openstack-health-monitor/), -but soon the new [health-monitor](https://scs.community/de/tech/2024/09/06/vp12-scs-health-monitor-tech-preview/)), -which exposes information on the performance and error rate of each cloud. -This provides some transparency on the state of the clouds by constantly running -scenario tests against them and is tremendously helpful for both the cloud operations -teams and their customers. Strictly speaking, it is *not* a requirement for the -SCS-compatible certification, just best practice. It will be part of an -SCS-sovereign certification though, where transparency on operational aspects -is included. - -## Staying compliant - -Once your cloud is listed in the list of certified clouds or in the compliance manager, it -will enjoy the nightly tests. These might fail for a number of reasons: - -* There is a new version of the SCS standards in effect and you need to adjust things. -* Your cloud was unreachable or otherwise had intermittent issues. -* You have done changes to your cloud that break SCS-compatible compliance. -* The test automation engine (github actions / zuul) is in trouble. -* The tests have a bug. - -In either case, this need proper analysis to determine what should be done. -In the list of certified clouds, the tests are performed by github actions. -These are executed from the -[github SCS standards repository](https://github.com/SovereignCloudStack/standards). -By looking at the logs from the github actions, you can typically see why the failure -happened. You could of course also do a local test again to see if the issue can -be reproduced. -In the compliance manager, we will add links to the log files directly on the table, -so it will be even easier to find the relevant log. diff --git a/standards/certification/test-and-adapt-example.md b/standards/certification/test-and-adapt-example.md deleted file mode 100644 index 8ab3a8a468..0000000000 --- a/standards/certification/test-and-adapt-example.md +++ /dev/null @@ -1,165 +0,0 @@ -# Example: Testing adjustment for SCS IaaS-compatible compliance - -## Run the tests - -Get the test suite by cloning [the SCS standards repo](https://github.com/SovereignCloudStack/standards/). -In order to run the tests, you need to have normal customer (tenant) access to the cloud or -container infrastructure that you want to test. (This is by design; we explicitly do not -require nor recommend admin level access for normal compliance testing.) - -You can run the test suite from any machine that has a working `python3-openstacksdk` (for the -IaaS tests) or working `python3`, `kubectl` and `helm` (for the KaaS tests). Go to the -checked out tree into the `Tests/` directory to run tests. Check that the tooling works, -e.g. by issuing a command like `openstack --os-cloud=MYCLOUD catalog list` or -`KUBECONFIG=~/.kube/MYCLUSTER.yaml kubectl get nodes -o wide`. - -Let's do a run against a sample environment: - -```bash -garloff@framekurt(//):/casa/src/SCS/standards/Tests [1]$ ./scs-compliance-check.py -V v4 -s CIAB -a os_cloud=ciab-test scs-compatible-iaas.yaml -INFO: module opc-v2022.11 missing checks or test cases -DEBUG: Fetching flavors from cloud 'ciab-test' -DEBUG: Checking 28 flavor specs against 18 flavors -WARNING: Flavor 'SCS-4V-16' found via name only, missing property 'scs:name-v2' -ERROR: Flavor 'SCS-4V-16' violating property constraints: scs:cpu-type: None should be 'shared-core'; scs:name-v1: None should be 'SCS-4V:16'; scs:name-v2: None should be 'SCS-4V-16' -WARNING: Flavor 'SCS-8V-32' found via name only, missing property 'scs:name-v2' -ERROR: Flavor 'SCS-8V-32' violating property constraints: scs:cpu-type: None should be 'shared-core'; scs:name-v1: None should be 'SCS-8V:32'; scs:name-v2: None should be 'SCS-8V-32' -WARNING: Missing recommended flavor 'SCS-1V-4-10' -WARNING: Missing recommended flavor 'SCS-2V-8-20' -WARNING: Missing recommended flavor 'SCS-4V-16-50' -WARNING: Missing recommended flavor 'SCS-8V-32-100' -WARNING: Missing recommended flavor 'SCS-1V-2-5' -WARNING: Missing recommended flavor 'SCS-2V-4-10' -WARNING: Missing recommended flavor 'SCS-4V-8-20' -WARNING: Missing recommended flavor 'SCS-8V-16-50' -WARNING: Missing recommended flavor 'SCS-16V-32-100' -WARNING: Missing recommended flavor 'SCS-1V-8-20' -WARNING: Missing recommended flavor 'SCS-2V-16-50' -WARNING: Missing recommended flavor 'SCS-4V-32-100' -WARNING: Missing recommended flavor 'SCS-1L-1-5' -DEBUG: Total critical / error / info: 0 / 2 / 0 -DEBUG: Fetching image list from cloud 'ciab-test' -DEBUG: Images present: Cirros 0.6.1, Cirros 0.6.2, Debian 12, EVIL2, EVIL3, Ubuntu 22.04 Minimal, disk-bf.qcow2, disk-vmdk.vmdk, openSUSE 15.5, openSUSE 15.6 -DEBUG: Checking 6 image specs against 10 images -ERROR: Missing mandatory image 'Ubuntu 22.04' -WARNING: Missing recommended image 'ubuntu-capi-image' -DEBUG: Missing optional image 'Ubuntu 20.04' -DEBUG: Missing optional image 'Debian 11' -DEBUG: Missing optional image 'Debian 10' -DEBUG: Total critical / error / warning: 0 / 1 / 1 -******************************************************************************** -CIAB SCS-compatible IaaS v4 (effective): -- main: FAIL (3 passed, 2 failed) - - FAILED: - - standard-flavors-check: - > Must fulfill all requirements of https://docs.scs.community/standards/scs-0103-v1-standard-flavors - - standard-images-check: - > Must fulfill all requirements of https://docs.scs.community/standards/scs-0104-v1-standard-images -``` - -So we run the SCS-compatible IaaS tests defined in `scs-compatible-iaas.yaml` in version `v4`; without option `-V`, -the latest effective version would have been used. We further define the cloud to be named `CIAB` (short for -Cloud-in-a-Box) in the report. And we set the parameter `os_cloud` to `ciab-test`. This references the -name of the cloud as configured in OpenStack `clouds.yaml` and `secure.yaml` which contain the configuration -and credentials to access the cloud as tenant user via the API (SDK or CLI). - -Let's have a look at the results: - -* We seem to have all 15 mandatory compute flavors, but two of them miss mandatory properties (`extra_specs`). - We also receive 13 warnings for not having recommended flavors, we can ignore them for now. -* On the images side, the mandatory image `Ubuntu 22.04` is not registered. -* The end result is that we passed three tests and failed to comply with two specs: - -```yaml - - standard-flavors-check: - > Must fulfill all requirements of https://docs.scs.community/standards/scs-0103-v1-standard-flavors - - standard-images-check: - > Must fulfill all requirements of https://docs.scs.community/standards/scs-0104-v1-standard-images -``` - -With option `-v`, we can make the test suite more verbose to e.g. see that we pass the flavor naming test, -the entropy test and the image metadata test. - -## Address issues - -To fix the failures, we will thus need to: - -* Add properties to the two flavors where they are missing. -* Register the `Ubuntu 22.04` image (with the appropriate metadata). - -Neither is difficult to do manually, but a more systematic and automated process is preferable. -For the first issue, there is a [blog article on flavor metadata](https://scs.community/de/tech/2024/08/20/flavor-extra-specs-compliance/). -For the image registration, the [OpenStack Image Manager](https://github.com/osism/openstack-image-manager) can be used. - -For adjusting the environment, we of course do need admin access to the cloud. -We use the tools referenced above: - -```shell -garloff@framekurt(//):/casa/src/SCS/standards/Tests [3]$ OS_CLOUD=ciab-admin ./iaas/flavor-naming/flavor-add-extra-specs.py -a apply -INFO: Flavor SCS-8V-32: SET scs:cpu-type=shared-core -INFO: Flavor SCS-8V-32: SET scs:name-v1=SCS-8V:32 -INFO: Flavor SCS-8V-32: SET scs:name-v2=SCS-8V-32 -INFO: Flavor SCS-4V-16: SET scs:cpu-type=shared-core -INFO: Flavor SCS-4V-16: SET scs:name-v1=SCS-4V:16 -INFO: Flavor SCS-4V-16: SET scs:name-v2=SCS-4V-16 -INFO: Processed 15 flavors, 6 changes -``` - -and as this is a OSISM-based SCS system, we can on the manager just run the image manager: - -```shell -dragon@manager:~$ osism manage images --cloud admin --filter "Ubuntu 22.04" -2024-09-23 13:21:43 | INFO | Processing image 'Ubuntu 22.04 (20240705)' -2024-09-23 13:21:43 | INFO | Tested URL https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-images/ubuntu-22.04/20240705-ubuntu-22.04.qcow2: 200 -2024-09-23 13:21:43 | INFO | Importing image Ubuntu 22.04 (20240705) -2024-09-23 13:21:43 | INFO | Importing from URL https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-images/ubuntu-22.04/20240705-ubuntu-22.04.qcow2 -2024-09-23 13:21:44 | INFO | Waiting for image to leave queued state... -2024-09-23 13:21:46 | INFO | Waiting for import to complete... -2024-09-23 13:21:56 | INFO | Waiting for import to complete... -2024-09-23 13:22:06 | INFO | Waiting for import to complete... -2024-09-23 13:22:16 | INFO | Import of 'Ubuntu 22.04 (20240705)' successfully completed, reloading images -2024-09-23 13:22:17 | INFO | Checking parameters of 'Ubuntu 22.04 (20240705)' -2024-09-23 13:22:17 | INFO | Setting internal_version = 20240705 -2024-09-23 13:22:17 | INFO | Setting image_original_user = ubuntu -2024-09-23 13:22:17 | INFO | Adding tag os:ubuntu -2024-09-23 13:22:17 | INFO | Setting property architecture: x86_64 -2024-09-23 13:22:17 | INFO | Setting property hw_disk_bus: scsi -2024-09-23 13:22:17 | INFO | Setting property hw_rng_model: virtio -2024-09-23 13:22:17 | INFO | Setting property hw_scsi_model: virtio-scsi -2024-09-23 13:22:17 | INFO | Setting property hw_watchdog_action: reset -2024-09-23 13:22:17 | INFO | Setting property hypervisor_type: qemu -2024-09-23 13:22:17 | INFO | Setting property os_distro: ubuntu -2024-09-23 13:22:18 | INFO | Setting property os_version: 22.04 -2024-09-23 13:22:18 | INFO | Setting property replace_frequency: quarterly -2024-09-23 13:22:18 | INFO | Setting property uuid_validity: last-3 -2024-09-23 13:22:18 | INFO | Setting property provided_until: none -2024-09-23 13:22:18 | INFO | Setting property image_description: Ubuntu 22.04 -2024-09-23 13:22:18 | INFO | Setting property image_name: Ubuntu 22.04 -2024-09-23 13:22:18 | INFO | Setting property internal_version: 20240705 -2024-09-23 13:22:18 | INFO | Setting property image_original_user: ubuntu -2024-09-23 13:22:18 | INFO | Setting property image_source: https://cloud-images.ubuntu.com/jammy/20240705/jammy-server-cloudimg-amd64.img -2024-09-23 13:22:18 | INFO | Setting property image_build_date: 2024-07-05 -2024-09-23 13:22:18 | INFO | Checking status of 'Ubuntu 22.04 (20240705)' -2024-09-23 13:22:18 | INFO | Checking visibility of 'Ubuntu 22.04 (20240705)' -2024-09-23 13:22:18 | INFO | Setting visibility of 'Ubuntu 22.04 (20240705)' to 'public' -2024-09-23 13:22:19 | INFO | Renaming Ubuntu 22.04 (20240705) to Ubuntu 22.04 -2024-09-23 13:22:19 | INFO | Processing image 'Ubuntu 22.04 Minimal (20240701)' -dragon@manager:~$ -``` - -A description how SCS-compatible IaaS compliance can be achieved on environments that use different -OpenStack implementations is written up in a blog article -[Cost of making an OpenStack Cluster SCS compliant](https://scs.community/de/2024/05/13/cost-of-making-an-openstack-cluster-scs-compliant/). - -## Rerun tests - -We now succeed: - -```shell -garloff@framekurt(//):/casa/src/SCS/standards/Tests [130]$ ./scs-compliance-check.py -V v4 -s CIAB -a os_cloud=ciab-test scs-compatible-iaas.yaml -INFO: module opc-v2022.11 missing checks or test cases -CIAB SCS-compatible IaaS v4 (effective): -- main: PASS (5 passed) -``` - -If you don't pass the tests yet, you'll need further adjustments.