Skip to content

fix: enforce the table quota, the provisioned-capacity maxima and max_policy_document_bytes - #831

Open
LeeroyHannigan wants to merge 5 commits into
mainfrom
fix/538-enforce-limits-keys
Open

LeeroyHannigan wants to merge 5 commits into
mainfrom
fix/538-enforce-limits-keys

Conversation

@LeeroyHannigan

Copy link
Copy Markdown
Collaborator

What

Six [limits] keys that the server accepted and did not act on now take effect.

  • max_tables_per_account: CreateTable counts the account's tables. When the account already holds the quota, it refuses the new table with LimitExceededException: There is a soft account quota of {n} tables.
  • per_table_max_rcu, per_table_max_wcu, per_account_max_rcu and per_account_max_wcu: CreateTable and UpdateTable refuse a provisioned throughput over a maximum with LimitExceededException. The checks are in crates/core/src/validation/provisioned_limits.rs. crates/engine/src/table_limits.rs reads what the account holds.
    • CreateTable checks the account maximum first. Then it checks the per-table maximum, which bounds the table and its global secondary indexes together.
    • UpdateTable first checks the table's own ProvisionedThroughput against the per-table maximum. Then it checks the account maximum against what the table and its indexes hold after the change.
    • A unit that the request does not raise is not refused. So a table above a lowered maximum can still be lowered.
    • Read units come before write units, as measured.
  • max_policy_document_bytes: the policy size limit was the literal 6,144 in PolicyDocument::from_json. The management API's policy and permissions boundary handlers and the console's policy page now pass the key to from_json_for_write. The authorization cache parses a stored identity policy, permissions boundary or session policy with the same key. The security model manual names the key.

On CreateTable the order is the measured one: the definition checks, then a taken name (ResourceInUseException), then the capacity limits. The table quota comes last.

Other effects:

  • Each CreateTable lists the account's table names. A CreateTable or UpdateTable with a provisioned throughput also describes every table in the account to sum what the account holds.
  • Two concurrent requests can both pass a check. So an account can end one table over the quota, or over a capacity maximum.
  • If an operator lowers max_policy_document_bytes below the size of a stored policy, the policy no longer parses. Requests of its principal then get AccessDeniedException.
  • docs: configuration reference generated from the sample file, pinned to the structs #537 documents max_tables_per_account, the per-table maxima and max_policy_document_bytes as not enforced. Whichever of the two merges second must update those descriptions.

Why

The issue lists the keys and the code that ignored them. The comment on the issue has the answers of Amazon DynamoDB, measured on 2026-10-31. This change uses those sentences where a measurement exists. These answers are not in the measurements:

  • The table quota sentence. It is the wording of the LimitExceededException API reference.
  • CreateTable over the per-table maximum while the account stays under its maximum. It uses the UpdateTable sentence.
  • UpdateTable that raises the table's own throughput over the account maximum but not over the per-table maximum. It uses the account sentence that the measured GSI case returned.
  • The order of the limit checks and the table state check. Over a maximum, UpdateTable on a table that is not ACTIVE gets LimitExceededException. Under the maxima, it gets ResourceInUseException.

Closes #538

Testing done

  • cargo +1.97.0 fmt --all -- --check and cargo +1.97.0 clippy --locked --all-targets -- -D warnings: clean. Pedantic lints flag no line that this change adds.
  • cargo +1.88.0 test --locked -p extenddb-core -p extenddb-auth -p extenddb-engine -p extenddb-server: all pass, with 775 unit tests in core, 372 in auth, 230 in engine and 295 in server.
  • The new tests:
    • Ten tests in provisioned_limits_tests.rs use the maxima and the reserved units of the measured account. They check the measured sentences, and each maximum at its value and one unit above. They also cover a table above a lowered maximum, the table quota and the sum of a request's capacity.
    • the_size_limit_is_the_one_the_caller_passes checks both parse paths at the limit and one byte over it.
    • a_stored_policy_over_the_configured_size_does_not_load checks that the authorization cache refuses a stored policy one byte over the configured limit. It fails on a mutant that ignores the limit on the read path.
  • A SQLite build of this branch, served on 127.0.0.1 with max_tables_per_account = 3, both per-table maxima at 100 and both account maxima at 150. Each answer below is the expected one:
    • DescribeLimits reports 150, 150, 100 and 100.
    • CreateTable with an undefined key and 101 RCU: the ValidationException for the undefined key.
    • CreateTable at 101 RCU, and at 101 WCU: the per-table sentence for each unit. At 100 RCU: created.
    • CreateTable at 51 RCU with 100 reserved: the account sentence, reserved by the account: 100. Limit: 150. Requested: 51. At 50 RCU: created.
    • UpdateTable of that table to 101 RCU: the per-table sentence. To 60 RCU: the account sentence with provisioned for the table and indexes: 50 and Requested: 60. Down to 40 RCU: accepted.
    • UpdateTable with a GSI create of 11 RCU on that table: the account sentence with reserved by the account: 140, provisioned for the table and indexes: 40 and Requested: 51.
    • With 3 tables, CreateTable of a fourth: There is a soft account quota of 3 tables. Of a taken name: ResourceInUseException.
  • The same server with max_policy_document_bytes = 400:
    • The server refuses a 622-byte user policy with Invalid policy document: policy parse error: policy document exceeds maximum size (400 bytes). It stores a 94-byte one.
    • After a restart at 50 bytes, ListTables by that user gets AccessDeniedException. After a restart at 400 bytes, it succeeds.

Checklist

  • I have read CONTRIBUTING.md
  • All tests pass (cargo test --workspace)
  • Code is formatted (cargo fmt --check)
  • Clippy is clean (cargo clippy -- -W clippy::pedantic)
  • I have added or updated tests for new functionality
  • I have updated documentation if behavior changed
  • Breaking changes are noted below (if any)
  • If this changes the wire protocol, Storage trait, auth model, on-disk
    format, or public CLI surface, an RFC has been accepted or is linked
    below. Otherwise, an ADR captures the decision (link below).

ADR / RFC: n/a

Breaking changes

The server now refuses requests that it accepted before. With the defaults, it refuses a table over 40,000 RCU or WCU and an account over 80,000 RCU or WCU. It also refuses a 2,501st table. A deployment that relies on more must raise the keys.

PolicyDocument::from_json_for_write now takes the size limit as a second argument.


By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.

Pure checks for max_tables_per_account, per_table_max_rcu/wcu and
per_account_max_rcu/wcu, with the LimitExceededException sentences the
service returns for a provisioned value over a maximum.
CreateTable counts the account's tables and refuses one over
max_tables_per_account, and CreateTable and UpdateTable refuse a
provisioned throughput over the per-table or account maximum with
LimitExceededException. A name the account already holds is still
refused as taken first.
The size limit of a policy document was the literal 6,144 in
PolicyDocument::from_json, so the max_policy_document_bytes key changed
nothing. The key is now the limit on both paths.

- Write: from_json_for_write now takes the limit as an argument. The
  management API's policy and permissions boundary handlers and the
  console's policy page pass the key to it.
- Read: the authorization cache parses a stored identity policy,
  permissions boundary or session policy with the key. A stored policy
  over it does not parse, and authorization answers AccessDenied.

from_json keeps the default of 6,144 bytes, now the named constant
DEFAULT_MAX_POLICY_DOCUMENT_BYTES. The security model manual names the
key.
from_json_for_write now takes the limit as a second argument, and the
fuzz target called it with one. The target now drives
from_json_with_size_limit and from_json_for_write with
DEFAULT_MAX_POLICY_DOCUMENT_BYTES, the two calls the server makes, and
its header names them.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Four [limits] keys are accepted and not read or enforced

1 participant