Support CPD 5.3.1 - Helm-based install for WML, WSL, Spark, Cognos Analytics and SPSS - #555
Open
Sakshi-Singhroha1 wants to merge 72 commits into
Open
Support CPD 5.3.1 - Helm-based install for WML, WSL, Spark, Cognos Analytics and SPSS#555Sakshi-Singhroha1 wants to merge 72 commits into
Sakshi-Singhroha1 wants to merge 72 commits into
Conversation
…Jobs for WML, Spark, SPSS
…stead of lt(semver,semver)
…fore namespace-scoped helm install
… for Helm 5.3.1+ installs
…I groups for Helm install
- Gate OperandRegistry/OperandConfig to < 5.3.1 to prevent OLM resource drift when services are installed via Helm - Gate spark-hb-control-plane patch job to < 5.3.1; OLM-managed deployment does not exist in Helm install model - Add CRD establishment wait inside WSL install_component() between cluster-scoped apply and namespace-scoped helm upgrade to prevent race condition - Extend Helm Job --wait timeout from 30m to 60m for WML, SPSS and Spark to match WSL and avoid premature timeout on loaded clusters
* Added CPD 5.3.1 gitops support * [minor] Fix CatalogSources template - add CPD 5.3.1 block (MASCORE-15507) * For CPD 5.3.1 services are installed via Helm * MASCORE-15507: Fix WSL Helm Job - install deps in sequence; add Helm Jobs for WML, Spark, SPSS * MASCORE-15507: Pass cpd_product_version to WSL, Spark, SPSS ArgoCD Applications * MASCORE-15507: Fix heredoc in WSL Helm Job - use printf to avoid YAML/shell conflict * Fix semver comparison - use toString to handle unquoted cpd_product_version value * MASCORE-15507: skip OLM CCS/OpenSearch post-sync patching for CPD 5.3.1+ * MASCORE-15507: bump job version to v2 to force new job name after script change * minor update * changes undo * MASCORE-15507: gate CCS/datarefinery/ws-runtimes OLM subscriptions behind < 5.3.1 * MASCORE-15507: fix semver comparison syntax - use .Compare pattern instead of lt(semver,semver) * MASCORE-15507: install cluster-scoped chart first to register CRDs before namespace-scoped helm install * MASCORE-15507: grant cpd-sa cluster-level CRD/ClusterRole permissions for Helm 5.3.1+ installs * MASCORE-15507: expand cpd-sa ClusterRole with escalate, roles, CPD API groups for Helm install * MASCORE-15507: pin cli image to digest instead of :latest in Helm install jobs * adding temp changes for facilities * undone postbackflow.sh script * MASCORE-15507: fix CPD 5.3.1 Helm service install correctness - Gate OperandRegistry/OperandConfig to < 5.3.1 to prevent OLM resource drift when services are installed via Helm - Gate spark-hb-control-plane patch job to < 5.3.1; OLM-managed deployment does not exist in Helm install model - Add CRD establishment wait inside WSL install_component() between cluster-scoped apply and namespace-scoped helm upgrade to prevent race condition - Extend Helm Job --wait timeout from 30m to 60m for WML, SPSS and Spark to match WSL and avoid premature timeout on loaded clusters --------- Co-authored-by: Sakshi Singhroha <Sakshi.Singhroha@ibm.com> Co-authored-by: Nikita Kumble <nikitakumble@gmail.com>
… Jobs Static-named ArgoCD sync hook Jobs (install-wsl-helm, install-wml-helm, install-spss-helm, install-spark-helm) were failing with: 'spec.template: Invalid value ... field is immutable' This happens when an old Job with the same name exists from a prior sync and ArgoCD tries to patch it. Since Job spec.template is immutable, the patch is rejected. Fix: add HookFailed to hook-delete-policy on all four Jobs so ArgoCD deletes the Job on both success AND failure, ensuring no stale Job exists when the next sync runs.
The wait loop was polling for: opensearchclusters.opensearch.cloudpackopen.ibm.com (does not exist) The actual CRD name registered by the IBM OpenSearch Helm chart is: clusters.opensearch.cloudpackopen.ibm.com This caused the WSL Helm Job to always timeout after 5 minutes at the OpenSearch step, blocking CCS, DataRefinery, WS-Runtimes and WSL from ever installing.
…f oc wait oc wait exits immediately with error if the resource does not exist. The CCS CR (ccs-cr) is created by the CCS operator after Helm installs it, so it is not immediately present. Replace oc wait with a polling loop using --ignore-not-found, same pattern used for all other CRD waits in this script. Polls every 30s up to 60 minutes.
…fter CCS Helm install
…us for CCS CR readiness
…analytics helm install job
…odeler chart with CPD 5.3.1 Helm support
…3.0.4), bump job to v2
…adata (spss/canvasbase 12.1.0, cognos 29.1.0)
Sakshi-Singhroha1
requested review from
mnivedithaa,
nehasangwai-ibm and
prajeshjain
August 11, 2026 06:54
…ion semver check in SPSS
…version semver checks
…stance-root README, add Job to WML README
…rn and $_cli_image_digest to WSL and Spark Helm install jobs
mnivedithaa
reviewed
Aug 12, 2026
| @@ -0,0 +1,19 @@ | |||
| {{- if lt (semver "5.3.1" | (semver (.Values.cpd_product_version | default "0.0.0" | toString)).Compare) 0 }} | |||
Contributor
There was a problem hiding this comment.
cognos analytics is required for cpd version >= 5.3.1 right? Can you check the condition if that is the case? Verify in all cognos related resource
Contributor
Author
There was a problem hiding this comment.
Cognos Analytics was not deployed via this GitOps repo before this story. It was newly added as part of MASCORE-15507 for CPD 5.3.1+ only via Helm. The OLM Subscription and CognosAnalytics CR have been removed. No version gate is needed — the Helm install always applies.
| - -c | ||
| - | | ||
| set -e | ||
| helm repo add ibm-charts https://raw.githubusercontent.com/IBM/charts/master/repo/ibm-helm |
Contributor
There was a problem hiding this comment.
helm charts are installed using k8s job. Is there any particular reason to use this way instead of using helm chart as source directly? How does this behave whenever the job re-runs due to cli image update?
Cognos Analytics is only available on CPD >= 5.3.1 and is installed via Helm only. There is no OLM path for Cognos - it was never available on CPD < 5.3.1. The Subscription file was incorrect.
Cognos Analytics is only available on CPD >= 5.3.1 via Helm. There is no OLM/CR path for older versions. Removed the incorrect version gate and the CognosAnalytics CR block that was gating on CPD < 5.3.1.
This reverts commit 759bdd9.
This reverts commit c702932.
Cognos Analytics was never deployed via this GitOps repo before this story. It was newly added as part of MASCORE-15507 for CPD 5.3.1+ only via Helm. No OLM path is needed. - Remove 00-ibm-cognos-analytics_Subscription.yaml entirely - Remove version gate and CognosAnalytics CR block from 01-ibm-cognos-analytics-cr.yaml — Helm install job always applies
The Ibmcpd CR spec.imagePullSecret was set to 'ibm_entitlement_key' (underscores) instead of the actual Kubernetes Secret name 'ibm-entitlement-key' (dashes). This caused the CP4D operator's Ansible playbook to fail to pull Common Services images, resulting in: 'Fail to wait for commonservice CR to reach Succeeded status' Fixes MASCORE-15507
… OutOfSync loop
When only an ArgoCD Hook Job exists in a chart (no non-hook resources),
ArgoCD enters an OutOfSync/Progressing loop after the job deletes itself
(HookSucceeded,HookFailed delete policy). The app shows OutOfSync because
ArgoCD sees zero managed resources.
Add a stable ConfigMap (sync-wave 091, before the hook job at 092) to
WML, Spark, SPSS, Cognos Analytics and WSL 5.3.1 Helm branches so
ArgoCD always has a persistent resource to track.
Fixes: wml.noble8.inst02, spark.noble8.inst02, spss.noble8.inst02
cognos.noble8.inst02, wsl.noble8.inst02 OutOfSync/Progressing
… WSL CI verify_chart_readme_tables.py requires every resource kind in a chart to be documented in the README. The persistent ConfigMap added to the CPD 5.3.1 Helm branches was not yet listed. Fixes CI lint failure: 'resource kind not documented in README table: ConfigMap'
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue
https://jsw.ibm.com/browse/MASCORE-15507
https://jsw.ibm.com/browse/MASCORE-15752
https://jsw.ibm.com/browse/MASCORE-16163
Description
Updates GitOps automation to support CPD 5.3.1 in addition to existing CPD 5.2 and 5.3 support. For CPD 5.3.1+, OLM-based installation is replaced with Helm-based installation using IBM Helm charts.
Summary of changes
CPD 5.3.1 OLM bootstrap (
101-ibm-sync-jobs-cp4d, 110-ibm-cp4d)WML (
120-ibm-wml)WSL (
120-ibm-wsl)Spark (
120-ibm-spark)Cognos Analytics (
120-ibm-cognos-analytics)SPSS Modeler (
120-ibm-spss) — re-added with CPD 5.3.1 supportRe-added chart with OLM Subscription + SPSSModeler CR for CPD < 5.3.1
Added Helm-based install job for CPD 5.3.1+ installing canvasbase (v12.1.0, dependency) then spss (v12.1.0)
Added ArgoCD Application template 120-ibm-spss-app.yaml
Post-sync job (
121-ibm-post-sync-job-cp4d-services)Skip OLM-specific CCS/OpenSearch post-sync patching for CPD 5.3.1+
Bumped $_job_version to v2
RBAC
Test Results
Tested on noble8 cluster with CPD 5.3.1 (inst02):
wml.noble8.inst02 → WmlBase/wml-cr Completed 100%
wsl.noble8.inst02 → Synced Healthy
spark.noble8.inst02 → Synced Healthy
cognos.noble8.inst02 → CAService/ca-addon-cr Completed 100%
spss.noble8.inst02 → canvasbase + spss Helm charts installed, canvasbase Completed 100%