Skip to content

Optimize username lookups with measured MySQL benchmarks - #362

Open
erikrozi wants to merge 2 commits into
DevOpsfrom
devin/1793936664-mysql-query-performance
Open

erikrozi wants to merge 2 commits into
DevOpsfrom
devin/1793936664-mysql-query-performance

Conversation

@erikrozi

@erikrozi erikrozi commented Nov 6, 2026 •

Copy link
Copy Markdown

Summary

AccountRepository.findByUsername scanned all accounts during login, dashboard/history and transfer recipient lookup. Add one non-unique account(username) index as an explicit versioned SQL migration, preserving duplicate usernames, nulls and collation. No Java application, query, security, ordering or response-size changes. The existing transaction foreign-key index was verified in real MySQL and left alone.

Applying the SQL is required: application startup does not automatically run these migration files. This PR does not deploy or connect to an existing database.

Measurement

MySQL 8.0.40, 100,000 synthetic accounts, 1,000,006 transactions, 80,000 active accounts; one history contains 50,000 rows. Deterministic skew, fixed dates, decimal and null/duplicate boundary fixtures. Same disposable server, dataset and connection in before/after/rollback phases; 5 warm-ups + 21 samples per query per phase.

Host: Linux x86-64 / Intel Xeon / 8 visible CPUs / approximately 31 GiB RAM / no swap. Database constrained to 2 CPUs, 2 GiB RAM, 256 MiB buffer pool. Java 17.0.19. Image ID: sha256:d58ac93387f644e4e040c636b8f50494e78e5afc27ca0a87348b2f577da2b7ff.

Query Rows Median DB execution before → after Median JDBC before → after DB after rollback
Username user-0050000 1 22.3 → 0.00719 ms 17.574 → 0.347 ms 23.8 ms
Username absent-user 0 19.0 → 0.00247 ms 14.224 → 0.151 ms 37.0 ms
Username duplicate 2 19.1 → 0.00543 ms 14.169 → 0.190 ms 19.0 ms
History, account 1 50,000 42.3 → 43.1 ms 72.247 → 68.981 ms 41.8 ms

DB timing is the instrumented root iterator last-row time from EXPLAIN ANALYZE; JDBC includes preparation, transport and all-column/all-row consumption, not HTTP. Username lookup improved; no meaningful transaction-history SQL improvement was demonstrated, so no transaction index/rewrite was added.

Separately, in-process MockMvc response medians: /dashboard 35.620 → 2.432 ms; /transactions 37.652 → 3.655 ms for an ordinary account. These include controller/repository/template work but exclude TCP, browser rendering and password hashing. They are not end-user login latency or an application-wide speedup claim.

Generated SQL and plans

-- parameter: user-0050000; one result row, no ORDER BY
select a1_0.id,a1_0.balance,a1_0.password,a1_0.username
from account a1_0 where a1_0.username=?;

-- parameter: 1; all 50,000 result rows, no LIMIT or ORDER BY
select t1_0.id,t1_0.account_id,t1_0.amount,t1_0.timestamp,t1_0.type
from transaction t1_0 where t1_0.account_id=?;

Baseline SHOW INDEX: account PRIMARY(id) only; transaction PRIMARY(id) and InnoDB-created FK6g20fcr3bhr6bihgy24rq1r1b(account_id). After: the sole additional index is idx_account_username(username), Non_unique=1.

Username before:
-> Filter: (a1_0.username = 'user-0050000')  (cost=10132 rows=9931) (actual time=11.5..23.6 rows=1 loops=1)
    -> Table scan on a1_0  (cost=10132 rows=99314) (actual time=0.017..15.2 rows=100000 loops=1)

Username after:
-> Index lookup on a1_0 using idx_account_username (username='user-0050000')  (cost=0.35 rows=1) (actual time=0.00943..0.0103 rows=1 loops=1)

After rollback:
-> Filter: (a1_0.username = 'user-0050000')  (cost=10132 rows=9931) (actual time=20.1..43.3 rows=1 loops=1)
    -> Table scan on a1_0  (cost=10132 rows=99314) (actual time=0.0119..32.7 rows=100000 loops=1)

History before:
-> Index lookup on t1_0 using FK6g20fcr3bhr6bihgy24rq1r1b (account_id=1)  (cost=14064 rows=101456) (actual time=0.11..64.3 rows=50000 loops=1)

History after:
-> Index lookup on t1_0 using FK6g20fcr3bhr6bihgy24rq1r1b (account_id=1)  (cost=14064 rows=101456) (actual time=0.143..64.8 rows=50000 loops=1)

The harness archives full plans, SQL, parameters, exact counts, schema/index definitions, storage sizes, machine metadata, raw samples and median summaries. The plans above are captured examples before warm-up, not the median timing samples in the table.

Equivalence and validation

  • Real-MySQL integration scenario passed at both 100,000/1,000,000 and 10,000/50,000 scales, plus the unchanged H2 context smoke test. H2 is not used as MySQL evidence.
  • Full ordered-row SHA-256 digests + counts match before = after = rollback for all selected fields/cases. No paging or truncation. Reapply is verified too.
  • Repository assertions cover empty and 50,000-row history, duplicate usernames (same exception), null fields/FK, empty/255-character names, accent/case collation, duplicate values/timestamps, zero/negative cents, maximum DECIMAL(38,2), leap-day microseconds and 1970/2038 boundaries.
  • Authenticated controller tests in both states verify transfer balances and both history entries, missing recipient, insufficient funds, anonymous redirects, and that a supplied foreign accountId cannot replace the authenticated username. Valid history HTML matches before/after.
  • GitHub Actions uses disposable MySQL and archives JUnit reports/plans even on failure. It has no latency threshold, publishing step or write permissions. Existing Jenkins/publishing/deployment files and security gates are unchanged and were not invoked.
  • GitHub Actions real-MySQL correctness passed; reports and plans are archived on the run. All six PR checks passed. The initial invalid setup-java action pin was corrected to the verified upstream v4.7.1 commit; no runtime code changed.
  • Recorded browser verification passed after explicit migration on synthetic MySQL accounts: login/history/transfers, separate recipient login, rejection cases, logout and anonymous redirects. Evidence and recording. PR Add Cypress end-to-end smoke suite, GitHub Actions workflow and Jenkins stage #361 scenarios/selectors were reused; this was manual browser verification, not execution of its Cypress suite. A cosmetic logged-out login-logo issue was observed; UI/security code is unchanged.
  • Devin Review completed on the final commit with zero findings. No remaining requested checks are pending.

Exact reproduction and rollback

# Java 17, Maven, Docker; creates and removes its own MySQL instance.
scripts/performance/benchmark.sh
# Equivalent full-size Maven command:
mvn -B -Pmysql-performance verify -Dperf.accounts=100000 \
  -Dperf.transactions=1000000 -Dperf.samples=21 -Dperf.warmup=5
# Small correctness entry point for a non-publishing Jenkins job:
mvn -B -Pmysql-performance verify -Dperf.accounts=10000 \
  -Dperf.transactions=50000 -Dperf.samples=5
# Parameterized local fixture:
PERF_ACCOUNTS=25000 PERF_TRANSACTIONS=250000 scripts/performance/benchmark.sh

Local commands used JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 and appended -s /home/ubuntu/maven-settings.xml because Maven Central returned HTTP 429; that VM-only, credential-free file mirrors central to https://maven-central.storage-download.googleapis.com/maven2. It is unnecessary on normal runners. No repository mirror/security changes were made.

Preserve target/mysql-performance/**, target/surefire-reports/**, and target/failsafe-reports/**. See methodology, Jenkins archive example, commands and limitations.

For a separately owned disposable database, inspect SHOW INDEX first and do not add an equivalent existing leading-column index. Apply sql/performance/V001__account_username_index.sql once; rollback with sql/performance/U001__account_username_index.sql using a MySQL client with an interactive password prompt. The harness executes both files and compares results automatically. There is no migration-history framework or automatic application on startup.

Index allocation grew by 3,686,400 bytes (3.52 MiB) on this dataset. Inserts/deletes/username updates maintain another B-tree and consume buffer/redo/I/O; balance-only changes do not alter the username key. ALGORITHM=INPLACE, LOCK=NONE still requires metadata locks and temporary build resources. No write-throughput claim is made. Rollback drops only this index, requires no data or Java rollback, and restores the original scan.

Untested / limits: production data or access; concurrent load/writes or migration contention; cold caches; other server versions; end-user/network latency; null-valued transaction rendering in the browser; write-throughput overhead. Measurements concern this application and synthetic MySQL dataset—not Oracle 19c or a 10-TB deployment. Existing transactionality/security/unbounded-history behavior is unchanged. No changes to PR #361.

Prompt

Repository: COG-GTM/Springboot-BankApp
URL: https://github.com/COG-GTM/Springboot-BankApp
Base branch: DevOps

Investigate and improve SQL query performance in the existing MySQL-backed application. Demonstrate a measured before/after improvement without changing query results or application behavior. Do not migrate databases.

1. Inspect the entities, repository queries, generated SQL, schema and existing indexes. Select one or two expensive queries used by real application flows. Do not assume an index is missing: inspect the actual database, including indexes created for foreign keys.

2. Start a disposable real MySQL instance matching the application's supported version. Generate deterministic synthetic data with realistic account/transaction distributions and skew. Make the volume configurable; aim for approximately one million transactions locally and a smaller fixture in CI. Do not use H2 as evidence of MySQL performance.

3. Capture the baseline SQL, parameters, row counts, indexes and EXPLAIN ANALYZE output where supported. Measure repeated runs after warm-up and report median latency, sample count, database version, dataset size and machine resources. Keep before/after conditions comparable and separate database execution time from application response time.

4. Implement the smallest justified improvement: an index, query rewrite or removal of redundant database calls. Preserve filtering, ordering, duplicates, null handling, decimal precision and authorization boundaries. Do not add pagination or change response semantics merely to make the benchmark faster. If no meaningful bottleneck is demonstrated, report that rather than manufacturing a speedup.

5. Prove before/after result equivalence with automated tests against real MySQL, including empty results, high-volume accounts, duplicate values and boundary cases. Add versioned SQL and rollback instructions for schema changes. Explain index storage and write-cost tradeoffs.

6. Provide a reproducible benchmark command and a Maven test entry point an existing Jenkins pipeline can invoke. Add GitHub Actions correctness validation against real MySQL. Archive test reports and query plans; do not impose brittle wall-clock performance thresholds on shared CI runners. Do not modify or trigger the existing publishing/deployment pipeline.

7. Run the application and record browser verification of login, transaction history and transfers after the change. Use synthetic accounts only. Reuse the Cypress work in PR #361 where appropriate without modifying that PR.

8. Open a PR with the diagnosed bottleneck, before/after execution plans and measured timings, result-equivalence evidence, exact reproduction commands, rollback instructions and limitations. Clearly state that these measurements concern this application and synthetic dataset—not Oracle 19c or a 10-TB deployment.

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/7bb6035bc84d4507b2143c037a290ad4
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/7bb6035bc84d4507b2143c037a290ad4?variant=devin
Requested by: @erikrozi


Devin Review

Status Commit
⚪ Not started —

Run Devin Review

Enable auto-review to review your pull requests automatically.

Devin Review (Staging)


Devin Review

Co-Authored-By: Erik Rozi <erik.rozi@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

  • Disable automatic comment, CI, and merge conflict monitoring

Original prompt from Erik

Repository: COG-GTM/Springboot-BankApp
URL: https://github.com/COG-GTM/Springboot-BankApp
Base branch: DevOps

Investigate and improve SQL query performance in the existing MySQL-backed application. Demonstrate a measured before/after improvement without changing query results or application behavior. Do not migrate databases.

  1. Inspect the entities, repository queries, generated SQL, schema and existing indexes. Select one or two expensive queries used by real application flows. Do not assume an index is missing: inspect the actual database, including indexes created for foreign keys.

  2. Start a disposable real MySQL instance matching the application's supported version. Generate deterministic synthetic data with realistic account/transaction distributions and skew. Make the volume configurable; aim for approximately one million transactions locally and a smaller fixture in CI. Do not use H2 as evidence of MySQL performance.

  3. Capture the baseline SQL, parameters, row counts, indexes and EXPLAIN ANALYZE output where supported. Measure repeated runs after warm-up and report median latency, sample count, database version, dataset size and machine resources. Keep before/after conditions comparable and separate database execution time from application response time.

  4. Implement the smallest justified improvement: an index, query rewrite or removal of redundant database calls. Preserve filtering, ordering, duplicates, null handling, decimal precision and authorization boundaries. Do not add pagination or change response semantics merely to make the benchmark faster. If no meaningful bottleneck is demonstrated, report that rather than manufacturing a speedup.

  5. Prove before/after result equivalence with automated tests against real MySQL, including empty results, high-volume accounts, duplicate values and boundary cases. Add versioned SQL and rollback instructions for schema changes. Explain index storage and write-cost tradeoffs.

  6. Provide a reproducible benc... (1482 chars truncated...)

@socket-security

socket-security Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedorg.testcontainers/​mysql@​1.21.410010090100100

View full report

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Co-Authored-By: Erik Rozi <erik.rozi@cognition.ai>
@devin-ai-integration

devin-ai-integration Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Browser verification passed on disposable MySQL 8.0.40 after applying the non-unique username index. Verified real login, owner-isolated history, sender/recipient transfer balances and entries, rejected transfers, logout and anonymous redirects using synthetic accounts. Reused the scenarios/selectors from PR #361 without modifying it; this was manual browser verification, not a Cypress suite run.

Sender debit Recipient credit
Sender history Recipient history

Recorded browser verification

Full recording

Evidence covers 3bb81f1. The only subsequent change is d34fe9f, correcting the setup-java CI pin; application, migration and test code are identical. The initial CI job could not resolve my incorrect action pin; the replacement is verified against upstream v4.7.1, and the real-MySQL correctness job passed, including archived reports and plans. All six checks are green; Devin Review found no issues on the final commit.

Observed cosmetic issue (Sev 3): the logged-out login logo is broken; it loads after authentication. It does not block any tested flow, and application/security/template files are unchanged by this PR. No browser latency, concurrent load, or null-transaction rendering claim is made.

Written by Devin

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant