Skip to content

fix: bound the size a compressed payload may expand to - #3502

Open
pjfanning wants to merge 4 commits into
apache:mainfrom
pjfanning:bound-decompression
Open

fix: bound the size a compressed payload may expand to#3502
pjfanning wants to merge 4 commits into
apache:mainfrom
pjfanning:bound-decompression

Conversation

@pjfanning

@pjfanning pjfanning commented Sep 1, 2026

Copy link
Copy Markdown
Member

Motivation

Five serializers gzip their payload and decompress it on the way back in, each with the
same unbounded loop: read the whole GZIPInputStream into a ByteArrayOutputStream and
return toByteArray. gzip expands by up to about three orders of magnitude, so neither
the size of the compressed bytes nor the transport's frame limit bounds the buffer the
decompressed bytes are read into — a payload well inside maximum-frame-size can expand
to hundreds of megabytes.

Two of the twelve call sites are a little further from where a failure would normally be
contained. ClusterMessageSerializer defers GossipEnvelope's decompression into a
thunk, so it runs when the gossip is read rather than on a deserialization thread; and
Welcome is deserialized while the node is still uninitialized.

The Jackson serializers already bound this — JacksonSerializer.gunzip, added in #3491
but nothing else does.

Modification

Add org.apache.pekko.serialization.Decompression (@InternalApi), whose gunzip
stops as soon as the decompressed size passes pekko.serialization.max-decompressed-size
and reports it as a NotSerializableException. Route all twelve call sites through it:

Module Helper Sites
cluster ClusterMessageSerializer.decompress 2 — Welcome, GossipEnvelope
distributed-data SerializationSupport.decompress 6 — Gossip, ORSet, ORMap, LWWMap, PNCounterMap, ORMultiMap
cluster-tools DistributedPubSubMessageSerializer.decompress 2 — Status, Delta
cluster-sharding ClusterShardingMessageSerializer.decompress 1 — CoordinatorState
cluster-metrics MessageSerializer.decompress 1 — MetricsGossipEnvelope

The setting defaults to unlimited, meaning no limit, so the bound is opt-in: a patch
release must not start rejecting payloads an existing cluster legitimately exchanges, and
there is no default that is provably above every deployment's largest ddata or gossip
payload. A negative number such as -1 is also accepted, matching the
pekko.serialization.jackson.read.max-document-length convention (dual spelling per
review on #3515, kept consistent here). Operators who want the protection set a size such
as 256 MiB, larger than anything their cluster legitimately sends. An unlimited maximum
skips the size check in gunzip; config's getBytes refuses both spellings, so the
setting is read as a string first and as a memory size only when it is neither
unlimited nor a negative number.

The four class-based serializers read the maximum once into a val.
SerializationSupport is a public trait whose implementors take system as a
constructor parameter, so it reads the maximum per call instead — a field there would
add abstract accessors to the trait and break binary compatibility, and the lookup is
negligible next to the decompression it guards.

The Jackson serializers are unchanged: they are already bounded and keep their own
pekko.serialization.jackson.compression.max-decompressed-size.

decompress signatures are unchanged, so there is no binary-compatibility impact.

Result

By default, behaviour is unchanged. With a maximum configured, a payload that expands
beyond it is rejected as an ordinary serialization failure naming the setting.

Tests

  • sbt "actor-tests/testOnly org.apache.pekko.serialization.DecompressionSpec" — 9 passed (round trip, the limit boundary, one byte over, the setting named in the message, a highly compressible payload rejected without inflating, no limit applied for a negative maximum, the unlimited default, a configured size read correctly, and a configured -1 read as unlimited)
  • sbt "cluster/testOnly org.apache.pekko.cluster.protobuf.*" — 13 passed, including a new ClusterMessageSerializerDecompressionSpec covering both the eager Welcome path and the deferred GossipEnvelope path
  • sbt "distributed-data/testOnly org.apache.pekko.cluster.ddata.protobuf.*" — 30 passed, including a new SerializationSupportDecompressionSpec covering the per-call lookup in the trait
  • sbt "cluster-tools/testOnly org.apache.pekko.cluster.pubsub.protobuf.*" — 1 passed
  • sbt "cluster-sharding/testOnly org.apache.pekko.cluster.sharding.protobuf.*" — 14 passed
  • sbt "cluster-metrics/testOnly org.apache.pekko.cluster.metrics.protobuf.*" — 9 passed
  • sbt "actor/mimaReportBinaryIssues" "cluster/mimaReportBinaryIssues" "cluster-metrics/mimaReportBinaryIssues" "cluster-sharding/mimaReportBinaryIssues" "cluster-tools/mimaReportBinaryIssues" "distributed-data/mimaReportBinaryIssues" — no issues
  • sbt scalafmtAll headerCreateAll — no changes

References

Extends #3491, which bounded decompression in the Jackson serializer only.

Decompression.scala consolidates the decompress bodies of the five Akka-derived
serializers listed above together with JacksonSerializer.gunzip, so it carries the
derived-from-Akka header and the Lightbend copyright. The three new specs are new code
and carry the standard ASF header.

Motivation:
Five serializers gzip their payload and decompress it on the way back in,
each with the same unbounded loop: read the whole GZIPInputStream into a
ByteArrayOutputStream. gzip expands by up to about three orders of
magnitude, so neither the size of the compressed bytes nor the transport's
frame limit bounds the buffer the decompressed bytes are read into.

Modification:
Add Decompression (@internalapi) with a gunzip that stops once the
decompressed size passes pekko.serialization.max-decompressed-size
(default 256 MiB) and reports it as a NotSerializableException, and route
all twelve call sites through it. The Jackson serializers already bound
decompression and keep their own
pekko.serialization.jackson.compression.max-decompressed-size.

Result:
An over-expanding payload is rejected as an ordinary serialization
failure. No behaviour change for payloads within the limit.
Motivation:
A bounded default could reject a payload an existing cluster legitimately
exchanges, so a patch release carrying a 256 MiB default could break
running clusters on upgrade. The bound should be opt-in.

Modification:
Default pekko.serialization.max-decompressed-size to -1, meaning no limit
and matching the behaviour of earlier releases. A negative maximum skips
the size check in gunzip. Config's getBytes refuses negative numbers, so
the setting is read as a plain long first and as a memory size only when
that is not a negative number.

Result:
Decompression is unbounded by default; configuring a size such as 256 MiB
bounds it.

Tests:
- sbt "actor-tests/testOnly org.apache.pekko.serialization.DecompressionSpec" - 8 passed
- sbt "cluster/testOnly org.apache.pekko.cluster.protobuf.ClusterMessageSerializerDecompressionSpec" - 4 passed
- sbt "distributed-data/testOnly org.apache.pekko.cluster.ddata.protobuf.SerializationSupportDecompressionSpec" - 3 passed
- sbt "actor/scalafmtCheckAll" "actor-tests/scalafmtCheckAll" - clean

References:
Refs apache#3502
@pjfanning

Copy link
Copy Markdown
Member Author

I will submit a fix the merge conflict soon. The aim is to get this backported for the 1.7.1 release. The limit is opt-in to make it easier to roll-out.

Motivation:
Review on apache#3515 noted that an explicit keyword is clearer than a magic
number. Keep the two sibling settings consistent: accept both spellings
here as well.

Modification:
pekko.serialization.max-decompressed-size reads "unlimited" or any
negative number as no limit; the reference.conf default is written as
`unlimited`. New tests cover the keyword default and an explicit -1.

Result:
`max-decompressed-size = unlimited` and `= -1` both disable the bound.

Tests:
- sbt "actor-tests/testOnly org.apache.pekko.serialization.DecompressionSpec" - 9 passed
- sbt "actor/scalafmtCheckAll" "actor-tests/scalafmtCheckAll" - clean

References:
Refs apache#3515, Refs apache#3502
pjfanning added a commit that referenced this pull request Sep 3, 2026
* fix: default the Jackson max-decompressed-size to unlimited

Motivation:
A bounded default could reject a payload an existing system legitimately
exchanges, so a patch release carrying the 256 MiB default from #3491
could break running systems on upgrade. The bound should be opt-in,
matching the change made to pekko.serialization.max-decompressed-size
in #3502.

Modification:
Default pekko.serialization.jackson.compression.max-decompressed-size
(and the jackson3 equivalent) to -1, meaning no limit and matching the
behaviour of releases before #3491. A negative maximum skips the gzip
size check and the LZ4 declared-size check; a negative declared LZ4
size is still rejected, since it is malformed regardless of the limit.
Config's getBytes refuses negative numbers, so the setting is read as a
plain long first and as a memory size only when that is not a negative
number.

Result:
Jackson payload decompression is unbounded by default; configuring a
size such as 256 MiB bounds it.

Tests:
- sbt "serialization-jackson/testOnly org.apache.pekko.serialization.jackson.*" - 122 passed
- sbt "serialization-jackson3/testOnly org.apache.pekko.serialization.jackson3.*" - 120 passed
- sbt "serialization-jackson/scalafmtCheckAll" "serialization-jackson3/scalafmtCheckAll" - clean
- sbt "serialization-jackson/mimaReportBinaryIssues" - no issues

References:
Refs #3491, Refs #3502

* also accept "unlimited" for max-decompressed-size

Motivation:
Review on #3515 noted that Pekko is inconsistent about unlimited
spellings and an explicit keyword is clearer than a magic number, while
the neighbouring read.max-document-length and read.max-token-count
settings use -1. Accept both.

Modification:
The setting reads "unlimited" or any negative number as no limit; the
reference.conf default is written as `unlimited`. Applied to both
serialization-jackson and serialization-jackson3, with a test each for
the keyword.

Result:
`max-decompressed-size = unlimited` and `= -1` both disable the bound.

Tests:
- sbt "serialization-jackson/testOnly org.apache.pekko.serialization.jackson.*" - 123 passed
- sbt "serialization-jackson3/testOnly org.apache.pekko.serialization.jackson3.*" - 121 passed
- sbt "serialization-jackson/scalafmtCheckAll" "serialization-jackson3/scalafmtCheckAll" - clean

References:
Refs #3515
@pjfanning pjfanning added this to the 2.0.0-M5 milestone Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant