-
Notifications
You must be signed in to change notification settings - Fork 879
Fix collation/ctype query error for non-default LC_COLLATE (#9798) #10040
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
dpage
wants to merge
3
commits into
pgadmin-org:master
Choose a base branch
from
dpage:fix-9798-collation-ctypes
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
16 changes: 8 additions & 8 deletions
16
...in/browser/server_groups/servers/databases/templates/databases/sql/16_plus/get_ctypes.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,8 @@ | ||
| SELECT CASE WHEN datlocprovider = 'i' THEN | ||
| (SELECT daticulocale as cname FROM pg_catalog.pg_database WHERE datname = current_database()) | ||
| ELSE | ||
| (SELECT datcollate as cname FROM pg_catalog.pg_database WHERE datname = current_database() | ||
| UNION | ||
| SELECT datctype as cname FROM pg_catalog.pg_database WHERE datname = current_database()) | ||
| END | ||
| FROM pg_catalog.pg_database WHERE datname = current_database(); | ||
| SELECT daticulocale AS cname FROM pg_catalog.pg_database | ||
| WHERE datname = current_database() AND datlocprovider = 'i' | ||
| UNION | ||
| SELECT datcollate AS cname FROM pg_catalog.pg_database | ||
| WHERE datname = current_database() AND datlocprovider <> 'i' | ||
| UNION | ||
| SELECT datctype AS cname FROM pg_catalog.pg_database | ||
| WHERE datname = current_database() AND datlocprovider <> 'i'; |
19 changes: 11 additions & 8 deletions
19
...in/browser/server_groups/servers/databases/templates/databases/sql/17_plus/get_ctypes.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,11 @@ | ||
| SELECT CASE WHEN datlocprovider = 'i' THEN | ||
| (SELECT datlocale as cname FROM pg_catalog.pg_database WHERE datname = current_database()) | ||
| ELSE | ||
| (SELECT datcollate as cname FROM pg_catalog.pg_database WHERE datname = current_database() | ||
| UNION | ||
| SELECT datctype as cname FROM pg_catalog.pg_database WHERE datname = current_database()) | ||
| END | ||
| FROM pg_catalog.pg_database WHERE datname = current_database(); | ||
| {# Both the ICU and the builtin provider keep the locale in datlocale; only a | ||
| libc database collates according to datcollate and datctype, which a | ||
| builtin database merely inherits from its template. #} | ||
| SELECT datlocale AS cname FROM pg_catalog.pg_database | ||
| WHERE datname = current_database() AND datlocprovider IN ('i', 'b') | ||
| UNION | ||
| SELECT datcollate AS cname FROM pg_catalog.pg_database | ||
| WHERE datname = current_database() AND datlocprovider = 'c' | ||
| UNION | ||
| SELECT datctype AS cname FROM pg_catalog.pg_database | ||
| WHERE datname = current_database() AND datlocprovider = 'c'; |
152 changes: 152 additions & 0 deletions
152
web/pgadmin/browser/server_groups/servers/databases/tests/test_db_get_ctypes.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,152 @@ | ||
| ########################################################################## | ||
| # | ||
| # pgAdmin 4 - PostgreSQL Tools | ||
| # | ||
| # Copyright (C) 2013 - 2026, The pgAdmin Development Team | ||
| # This software is released under the PostgreSQL Licence | ||
| # | ||
| ########################################################################## | ||
|
|
||
| """Tests for the get_ctypes.sql templates (#9798). | ||
|
|
||
| The query feeding the collation and character type fields of the Database | ||
| dialog has to report the locale the database actually uses, which depends on | ||
| its locale provider: datcollate and datctype for libc, and datlocale for both | ||
| ICU and the builtin provider introduced in PostgreSQL 17. A builtin database | ||
| still carries the collation it inherited from its template, so reading | ||
| datcollate for it reports a locale that is not in use. | ||
|
|
||
| The query is executed here through the versioned template loader, so the test | ||
| also covers the server version picking the right template bucket. | ||
| """ | ||
|
|
||
| import uuid | ||
|
|
||
| from flask import render_template | ||
|
|
||
| from pgadmin.utils.route import BaseTestGenerator | ||
| from regression.python_test_utils import test_utils as utils | ||
|
|
||
| DEFAULTS = ('C', 'POSIX') | ||
|
|
||
|
|
||
| class GetCtypesTestCase(BaseTestGenerator): | ||
| """get_ctypes.sql must report the locale in use, per locale provider.""" | ||
|
|
||
| scenarios = [ | ||
| ('libc reports the collation', dict( | ||
| provider='libc', swapped=False)), | ||
| ('libc reports the character type', dict( | ||
| provider='libc', swapped=True)), | ||
| ('The builtin provider reports its own locale', dict( | ||
| provider='builtin', swapped=False)), | ||
| ('ICU reports the ICU locale', dict( | ||
| provider='icu', swapped=False)), | ||
| ] | ||
|
|
||
| def setUp(self): | ||
| # No pgAdmin server registration needed: this exercises the SQL, so | ||
| # it talks to the server directly. | ||
| self.db_name = None | ||
| self.connection = self._connect('postgres') | ||
| cursor = self.connection.cursor() | ||
| cursor.execute("SELECT current_setting('server_version_num')::int") | ||
| self.server_version = cursor.fetchone()[0] | ||
| cursor.execute("SELECT datcollate, pg_encoding_to_char(encoding) " | ||
| "FROM pg_catalog.pg_database WHERE datname = " | ||
| "'template0'") | ||
| self.template_collate, self.template_encoding = cursor.fetchone() | ||
|
|
||
| def _connect(self, db_name): | ||
| return utils.get_db_connection(db_name, | ||
| self.server['username'], | ||
| self.server['db_password'], | ||
| self.server['host'], | ||
| self.server['port'], | ||
| self.server['sslmode']) | ||
|
|
||
| def _create_database(self, options): | ||
| self.db_name = 'test_ctypes_%s' % str(uuid.uuid4())[1:8] | ||
| old_isolation_level = self.connection.isolation_level | ||
| utils.set_isolation_level(self.connection, 0) | ||
| cursor = self.connection.cursor() | ||
| cursor.execute('CREATE DATABASE "%s" TEMPLATE template0 %s' % ( | ||
| self.db_name, options)) | ||
| utils.set_isolation_level(self.connection, old_isolation_level) | ||
| self.connection.commit() | ||
|
|
||
| def _reported_locales(self): | ||
| """Run the versioned get_ctypes.sql against the new database.""" | ||
| template_path = 'databases/sql/#{0}#'.format(self.server_version) | ||
| with self.app.app_context(): | ||
| sql = render_template("/".join([template_path, 'get_ctypes.sql'])) | ||
|
|
||
| connection = self._connect(self.db_name) | ||
| try: | ||
| cursor = connection.cursor() | ||
| cursor.execute(sql) | ||
| return [row[0] for row in cursor.fetchall()] | ||
| finally: | ||
| connection.close() | ||
|
|
||
| def _skip_unless_distinct_template_locale(self): | ||
| if self.template_collate in DEFAULTS: | ||
| self.skipTest( | ||
| "template0 uses the '%s' locale, so a database with a " | ||
| "collation distinguishable from the defaults cannot be " | ||
| "created here." % self.template_collate) | ||
|
|
||
| def runTest(self): | ||
| if self.provider == 'libc': | ||
| self._skip_unless_distinct_template_locale() | ||
| # Set exactly one of the two to the template locale, so that the | ||
| # value proves which of datcollate and datctype was reported. | ||
| collate, ctype = ('C', self.template_collate) if self.swapped \ | ||
| else (self.template_collate, 'C') | ||
| self._create_database( | ||
| "LC_COLLATE '%s' LC_CTYPE '%s'" % (collate, ctype)) | ||
| self.assertIn(self.template_collate, self._reported_locales()) | ||
| return | ||
|
|
||
| if self.provider == 'builtin': | ||
| if self.server_version < 170000: | ||
| self.skipTest('The builtin locale provider requires ' | ||
| 'PostgreSQL 17 or later.') | ||
| if self.template_encoding != 'UTF8': | ||
| self.skipTest("template0 is %s encoded, so a UTF-8 builtin " | ||
| "locale cannot be used here." | ||
| % self.template_encoding) | ||
| if self.template_collate == 'C.UTF-8': | ||
| self.skipTest("template0 already uses the 'C.UTF-8' locale, " | ||
| "the only one a UTF-8 builtin database can " | ||
| "use, so a distinguishable one cannot be " | ||
| "created here.") | ||
| self._skip_unless_distinct_template_locale() | ||
| self._create_database("LOCALE_PROVIDER builtin " | ||
| "BUILTIN_LOCALE 'C.UTF-8' ENCODING UTF8") | ||
|
|
||
| reported = self._reported_locales() | ||
| self.assertIn('C.UTF-8', reported) | ||
| # datcollate is inherited from the template and is not the locale | ||
| # this database collates with, so it must not be offered. | ||
| self.assertNotIn(self.template_collate, reported) | ||
| return | ||
|
|
||
| # ICU | ||
| if self.server_version < 150000: | ||
| self.skipTest('LOCALE_PROVIDER icu requires PostgreSQL 15 or ' | ||
| 'later.') | ||
| cursor = self.connection.cursor() | ||
| cursor.execute("SELECT 1 FROM pg_catalog.pg_collation " | ||
| "WHERE collprovider = 'i' LIMIT 1") | ||
| if cursor.fetchone() is None: | ||
| self.skipTest('This server was built without ICU support.') | ||
| self._create_database( | ||
| "LOCALE_PROVIDER icu ICU_LOCALE 'en-GB' LC_COLLATE '%s' " | ||
| "LC_CTYPE '%s'" % (self.template_collate, self.template_collate)) | ||
| self.assertIn('en-GB', self._reported_locales()) | ||
|
|
||
| def tearDown(self): | ||
| if self.db_name: | ||
| utils.drop_database(self.connection, self.db_name) | ||
| self.connection.close() | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.