Skip to content

db: pin utf8mb4_general_ci as the default connection collation - #13962

Draft
winterhazel wants to merge 5 commits into
apache:mainfrom
winterhazel:pin-default-collation
Draft

db: pin utf8mb4_general_ci as the default connection collation#13962
winterhazel wants to merge 5 commits into
apache:mainfrom
winterhazel:pin-default-collation

Conversation

@winterhazel

@winterhazel winterhazel commented Aug 25, 2026

Copy link
Copy Markdown
Member

Description

This change is related to Ubuntu 26.04 support.

The latest version of MariaDB for Ubuntu 26.04 is 11.8; on version 11.6, the default collation for the UTF-8 character set was changed from utf8mb4_general_ci to utf8mb4_0900_ai_ci. This results in some queries throwing exceptions due to the collation mix. For instance, the call to listTemplates by the instance deployment wizard throws the exception below.

2026-08-24 23:20:26,478 ERROR [c.c.a.q.d.TemplateJoinDaoImpl] (qtp888557915-28:[ctx-e2e22867, ctx-64ef36f1]) (logid:56439487) DB Exception on: HikariProxyPreparedStatement@1786128789 wrapping com.mysql.cj.jdbc.ServerPreparedStatement[186]: SELECT template_view.uuid, template_view.unique_name, template_view.name, template_view.format, template_view.public, template_view.featured, template_view.type, template_view.url, template_view.hvm, template_view.arch, template_view.bits, template_view.created, template_view.created_on_store, template_view.removed, template_view.checksum, template_view.display_text, template_view.enable_password, template_view.dynamically_scalable, template_view.guest_os_id, template_view.guest_os_uuid, template_view.guest_os_name, template_view.guest_os_category_id, template_view.bootable, template_view.prepopulate, template_view.cross_zones, template_view.hypervisor_type, template_view.extractable, template_view.source_template_id, template_view.source_template_uuid, template_view.template_tag, template_view.sort_key, template_view.enable_sshkey, template_view.account_id, template_view.account_uuid, template_view.account_name, template_view.account_type, template_view.domain_id, template_view.domain_uuid, template_view.domain_name, template_view.domain_path, template_view.project_id, template_view.project_uuid, template_view.project_name, template_view.data_center_id, template_view.data_center_uuid, template_view.data_center_name, template_view.store_scope, template_view.store_id, template_view.download_state, template_view.download_pct, template_view.error_str, template_view.size, template_view.physical_size, template_view.template_state, template_view.destroyed, template_view.lp_account_id, template_view.parent_template_id, template_view.parent_template_uuid, template_view.detail_name, template_view.detail_value, template_view.state, template_view.temp_zone_pair, template_view.direct_download, template_view.deploy_as_is, template_view.for_cks, template_view.user_data_id, template_view.user_data_uuid, template_view.user_data_name, template_view.user_data_policy, template_view.user_data_params, template_view.extension_id, template_view.extension_uuid, template_view.extension_name, template_view.id, template_view.tag_id, template_view.tag_uuid, template_view.tag_key, template_view.tag_value, template_view.tag_domain_id, template_view.tag_account_id, template_view.tag_resource_id, template_view.tag_resource_uuid, template_view.tag_resource_type, template_view.tag_customer, template_view.tag_account_name, template_view.tag_domain_uuid, template_view.tag_domain_name FROM template_view WHERE template_view.template_state IN ('Active','UploadAbandoned','UploadError','NotUploaded','UploadInProgress')  AND template_view.temp_zone_pair=x'3230345f31' ORDER BY template_view.sort_key ASC , template_view.temp_zone_pair ASC java.sql.SQLException: Illegal mix of collations (utf8mb4_general_ci,COERCIBLE) and (utf8mb4_0900_ai_ci,COERCIBLE) for operation '='

This PR pins TransactionLegacy to use utf8mb4_general_ci by default, unless specified in db.<schema>.url.params or db.<schema>.uri.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Some tests are still pending; so far, I've:

  • Upgraded an all-in-one VM containing both the Management Server and the database from Ubuntu 24.04 to 26.04, and then installed the 4.23 packages.
    • Before the changes, I would get an exception in some queries.
    • After the changes, I did not get the exception anymore.

I will also test a fresh deployment without/with these changes.

@winterhazel

Copy link
Copy Markdown
Member Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

@codecov

codecov Bot commented Aug 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.41%. Comparing base (4e4a26b) to head (8383db0).

❗ There is a different number of reports uploaded between BASE (4e4a26b) and HEAD (8383db0). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (4e4a26b) HEAD (8383db0)
unittests 1 0
Additional details and impacted files
@@              Coverage Diff              @@
##               main   #13962       +/-   ##
=============================================
- Coverage     19.73%    3.41%   -16.33%     
=============================================
  Files          6371      487     -5884     
  Lines        575765    41863   -533902     
  Branches      70478     7912    -62566     
=============================================
- Hits         113642     1429   -112213     
+ Misses       449766    40234   -409532     
+ Partials      12357      200    -12157     
Flag Coverage Δ
uitests 3.41% <ø> (ø)
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18974

@weizhouapache

Copy link
Copy Markdown
Member

@winterhazel
could you try with the following mariadb settings ?

# cat /etc/mysql/my.cnf 
#
# The MariaDB/MySQL configuration file.
#

!includedir /etc/mysql/mariadb.conf.d/

# cat /etc/mysql/mariadb.conf.d/99-cloudstack.cnf 
[mysqld]
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci

@winterhazel winterhazel removed this from the 4.23.0 milestone Aug 25, 2026
@DaanHoogland DaanHoogland moved this from Backlog to conflict/waiting in CloudStack Testing Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: conflict/waiting

Development

Successfully merging this pull request may close these issues.

4 participants