Skip to content

fix(gcp): SKU lookup for Cloud SQL, Memorystore and Compute matches tier strings real descriptions may not contain #245

Closed

Description

@cristim

Summary

The GCP pricing extractors for Cloud SQL, Memorystore and Compute Engine find the SKU for a tier or machine type by searching the SKU's free-text Description for the tier or machine-type string. As far as I can tell from Google's published SKU descriptions, real Billing Catalog descriptions do not contain those strings, so real lookups would match nothing and return "no pricing found". The tests only use synthetic descriptions that were written to contain the token, so they cannot detect this.

Found in the independent review of #244 (a slice of #97). Related: #97, #244, #165.

Locations (main at be54dfa)

  • providers/gcp/services/cloudsql/client.go:453-455 (skuMatchesTier, strings.Contains on sku.Description), used by extractSQLPricingFromSKUs :400; error at :362
  • providers/gcp/services/memorystore/client.go:421-423 (skuMatchesTier), extractPricingFromSKUs :368; tiers hardcoded at :295-298 (BASIC, STANDARD_HA); error at :331
  • providers/gcp/services/computeengine/client.go:1142-1144 (skuMatchesMachineType), extractComputePricingFromSKUs :1088; error at :1055
  • SKUs are fetched with Services.Skus.List(serviceID) (cloudsql/client.go:108, memorystore/client.go:102, computeengine/client.go:340).

Failure scenario

Request pricing for Cloud SQL tier db-n1-standard-4 in us-central1. The extractor lowercases the description of each SKU and checks for db-n1-standard-4. Cloud SQL descriptions on Google's CUD-eligible SKU page look like:

  • Cloud SQL for MySQL: Regional - 1 vCPU + 3.75GB RAM in Americas
  • Cloud SQL for My SQL: Zonal - Enterprise Plus Performance Optimized C4A vCPU in Alabama

Neither contains a db-* tier token, so no SKU matches, both prices stay 0, and the caller gets no pricing found for Cloud SQL tier db-n1-standard-4. The same applies to Memorystore (the code's tiers are BASIC and STANDARD_HA, the pricing page lists capacity tiers such as M1 Basic and M1 Standard) and to Compute Engine (machine types such as n1-standard-4 are priced per vCPU and per GB of RAM, e.g. N1 Predefined Instance Core running in Americas, so a machine-type string would not appear).

This is the same before and after #244. #244 tightens the match to a whole token, which does not help if the token is absent from real descriptions.

Verified vs unverified

Verified on main:

  • All three matchers test only sku.Description for the tier/machine-type string. No structured fields (Category.ResourceGroup, Category.UsageType, Category.ResourceFamily, ServiceDisplayName, PricingInfo[].PricingExpression.UsageUnit) are used to select the SKU.
  • Every fixture is synthetic, e.g. Description: "db-n1-standard-1 Cloud SQL" (cloudsql/client_test.go:203), "Memorystore Redis STANDARD_HA" (memorystore/client_coverage_test.go, ~:196).
  • No recorded real Billing Catalog response exists in the repo (no testdata directories; docs/design/gcp-spend-commitments.md has no SKU shapes).

Unverified:

  • The real-data claim comes from description strings on Google's documentation pages (cloud.google.com/skus/sku-groups/cloud-sql-cud-eligible-skus and the pricing pages), not from a live Skus.List call. I used no credentials. The Compute SKU description forms were recalled, not checked. The API may return descriptions that differ from the docs page, and Category fields may also vary.
  • Whether any real description contains the tier string by accident for some tiers.

Impact is therefore marked impact/few and severity medium pending a real catalog capture. If the claim holds, it becomes broader, and priority should be raised.

Acceptance criteria

  1. Capture a recorded real Billing Catalog Skus.List response per service (Cloud SQL, Memorystore, Compute Engine), trimmed to relevant SKUs, committed as a fixture.
  2. Match SKUs on structured fields (Category.ResourceGroup, Category.UsageType, ServiceDisplayName, pricing unit) plus the vCPU/RAM amounts described, instead of the tier/machine-type string in Description. Map tiers and machine types to vCPU/RAM (or Memorystore capacity tier) explicitly.
  3. A test per service runs the extractor against the recorded real data and asserts a price for a known tier and region.
  4. If the capture shows the current matching does work for some case, record that in the issue and close it with the fixture and test added.

Activity

  1. cristim commented on Oct 17, 2026

    @cristim
    MemberAuthor

    Checked against main: the compute part is fixed (pricing.go matches structured SKU fields). Cloud SQL and Memorystore still match on Description via skumatch.HasToken, and those files plus skumatch.go are in open PRs #433, #471 and #493. Acceptance criterion 1 also needs a live Billing Catalog capture, which is an operator action. Blocked on those PRs.

  2. cristim commented on Nov 13, 2026

    @cristim
    MemberAuthor

    Blocker (2026-11-13): the Compute part is confirmed (descriptions carry no machine-type strings). Remaining work is cloudsql/client.go and memorystore/client.go skuMatchesTier, already blocked on PRs 433, 471 and 493.

  3. cristim commented on Nov 14, 2026

    @cristim
    MemberAuthor

    Decision (no code change): a live read-only Billing Catalog capture (Skus.List, GET only, USD, all pages) shows the Cloud SQL and Memorystore match cannot be fixed from the tier string, so I did not change skuMatchesTier / skumatch.MatchesTier.

    Catalog evidence (services/9662-B51E-5089 Cloud SQL: 21436 SKUs; services/5AF5-2C11-D467 Memorystore for Redis: 1110 SKUs):

    • Confirms the issue: 0 Cloud SQL and 0 Memorystore descriptions contain a db- token. The description token match finds nothing for db-n1-standard-4.
    • Every SKU in both services has Category.UsageType = OnDemand. There is no Commit1Yr/Commit3Yr SKU in either catalog (spend-based CUD prices are not in these listings), so the commitment price the extractor is asked for is absent from the data whatever the match.
    • Cloud SQL descriptions encode engine, availability, edition and size, for example Cloud SQL for MySQL: Zonal - 4 vCPU + 15GB RAM in Americas (resourceGroup SQLGen2InstancesN1Standard, unit h, serviceRegions us-central1,...) next to Cloud SQL for MySQL: Regional - Extended support 4 vCPU + 15GB RAM v57 in Iowa. For db-n1-standard-4 in us-central1 six SKUs match vCPU/RAM (Zonal/Regional, plain/Extended support v56/v57). Postgres and SQL Server price vCPU and RAM as separate ...CPU/...RAM SKUs, with no combined per-tier SKU.
    • Memorystore Redis SKUs are Redis Capacity Basic M1 Iowa/South Carolina/Oregon / Redis Standard Node Capacity M2 Iowa (resourceGroup Redis, unit GiBy.h), priced per GiB per capacity tier M1..M5. BASIC/STANDARD_HA select Basic vs Standard only; the M-tier is determined by instance size, which the tier string does not carry.

    So the tier alone does not determine one SKU. Settling it needs these inputs, which the recommendation does not carry today:

    1. Cloud SQL: engine (MySQL/Postgres/SQL Server), availability (Zonal/Regional), edition and extended-support flag, plus an explicit tier to vCPU/RAM map (db-n1-standard-N = N vCPU + 3.75N GB; highmem 6.5N GB).
    2. Memorystore: instance size to capacity tier M1..M5, and Basic vs Standard.
    3. The source of the committed price: the spend-based CUD discount is not in Skus.List for these services, so the owner must say where it comes from (the Commitment/Consumer Procurement API or a documented discount table) before a commitment price can be extracted.

    Proposed follow-up (not started): extend the recommendation/offering input with those fields, then match on resourceGroup + description structure + serviceRegions and fail loud on 0 or >1 matches. Leaving #245 open.

  4. cristim commented on Nov 14, 2026

    @cristim
    MemberAuthor

    Closing with a recorded decision, no code change.

    Blockers PRs are merged: #433 (tiered SKUs from the first paid tier), #471 (fail closed on region-less storage SKUs and ambiguous matches), #493 (no Cloud SQL or Memorystore CUDs from PerformanceRecommend).

    Current main: Cloud SQL and Memorystore no longer emit CUD recommendations, and their pricing path (skumatch.ExtractPricing -> prices.Commitment) returns an error when no commitment SKU matches the tier. A tier string that is missing from SKU descriptions therefore fails loud and never produces a fabricated price, which removes the wrong-price risk this issue describes. Making the match exact would need a catalog key the Billing API does not expose for these services (live catalog capture, 2026-11-13), so the tier-token helper stays as is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions