Skip to content

Handle nullability more accurately#2933

Open
Youssef1313 wants to merge 11 commits into
microsoft:mainfrom
Youssef1313:dev/ygerges/nullable
Open

Handle nullability more accurately#2933
Youssef1313 wants to merge 11 commits into
microsoft:mainfrom
Youssef1313:dev/ygerges/nullable

Conversation

@Youssef1313

@Youssef1313 Youssef1313 commented Jul 9, 2026

Copy link
Copy Markdown
Member

My previous PR was trying to handle nullability for enums specifically. This PR is more generalized to handle nulls more correctly as it was still buggy for non-enums.

In addition, the previous fix for enums didn't also fix all enum scenarios. It only fixed it when we are given "oneOf (null, schemaWithEnum)". But if we are given a schema reference instead, we are not going to add "null" to the reference and we will produce the wrong thing.

I'll leave comments on some specific updated tests to explain.

@Youssef1313 Youssef1313 requested a review from a team as a code owner July 9, 2026 10:30
// NOTE: nullable property has no effect on the schema if type is not specified, so it is omitted in the serialized output.
// Arrange
var expected = @"nullable: true";
var expected = @"{ }";

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both behaviors are technically okay. But it's better to avoid emitting nullable when the spec says that it has no effect. This is less misleading for people reading the document and no familiar with the spec details.

{
"type": "string",
"oneOf": [
{

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The behavior here was previously buggy.

Even though "nullable" was set to true. A null value would have been still rejected because we won't match any of the "oneOf" schemas.

nullable only widens the "type" when present, but if the schema is still constrained to not allow null, a null would be rejected.

https://spec.openapis.org/oas/v3.0.4.html#fixed-fields-20

This keyword only takes effect if type is explicitly defined within the same Schema Object. A true value indicates that both null values and values of the type specified by type are allowed. Other Schema Object constraints retain their defined behavior, and therefore may disallow the use of null as a value. A false value leaves the specified or default type unmodified. The default value is false.

The new behavior is more aligned with the object model given by the user.

The user specified a oneOf child schema with Type = null, and we serialize the semantically equivalent schema that's valid for OpenAPI 3.0 (which doesn't use draft 2020-12 and so can't use the straightforward { "type": "null" } approach.

@Youssef1313 Youssef1313 force-pushed the dev/ygerges/nullable branch from 67ef969 to 937b8eb Compare July 9, 2026 10:38
@Youssef1313 Youssef1313 force-pushed the dev/ygerges/nullable branch from 937b8eb to a70df43 Compare July 9, 2026 10:39
Extensions.TryGetValue(OpenApiConstants.NullableExtension, out var nullExtRawValue) &&
nullExtRawValue is JsonNodeExtension { Node: JsonNode jsonNode } &&
jsonNode.GetValueKind() is JsonValueKind.True;
internal bool IsNullable

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we are parsing a 3.0 document, we might encounter "nullable" property.

We might not know (yet) if this nullable property should have any effect or not (because we don't know if a future type property will be encountered or not).

As a result, we shouldn't be modifying OpenApiSchema.Type right away as soon as we see "nullable": true when deserializing.

To address this, IsNullable is made settable internally.

We will set it without modifying the type to indicate that "nullable": true was encountered while deserializing a document.

Then, at the end of the deserialization when we got all the information, we add "null" to OpenApiSchema.Type if nullable was set and we have a non-null type.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is confusing because of the "upgrade" path. Let's acknowledged that conversions are lossy by nature, and that we do everything we can to enable roundtrips with as good fidelity as possible.

on roundtrips and upgrades

nullable: true
type: string

should roundtrip in 3.0, but should also lead to

type: ["string", "null"]

In 3.1/3.2

on the presence of nullable without type

the OpenAPI 3.0 specification states

This keyword only takes effect if type is explicitly defined within the same Schema Object

It is NOT incorrect to have nullable: true with no type, it only means that keyword/field is meaningless.

While the library ensures the document is structured properly, it does not enforce any specific semantics (other than the logical conversions).

on v2 serialization

v2 is a 12 years old standard, v3: 9 years old. While those still have a large user base due to some tooling never upgrading (some of it becoming abandonware), I don't believe we should over optimize for those. Especially if that comes at the potential expense of the more modern versions.

We do provide the nullable as an extension to provide more context to the consumer when something was downgraded. But honestly, I doubt that a tool that has not taken the time to upgrade in 9 years at this point will take the time to implement that extension. Yes we should roundtrip that extension when possible to avoid losses, but that'd a much lower priority.

on v3.1/3.2

the specification clearly states that having just type: null is perfectly fine.

conclusion

With all that in mind. I don't want to see "multiple states" on the schema object that need to be kept in sync. Those schema objects can be initialized from deserialization, but also from the object model, and can be mutated afterwards. It'd be too easy for the state to become "out of sync" on one of the paths because the caller does not (and shouldn't know) the implementation details.

For deserialization:

  • When we encounter nullable: true in v2, or x-nullable: true in v3, we should set the type to "add null" (preserving any previous type value) as one of the values. (and drop the extension in v2 to avoid any confusion)
  • When deserializing the type, it should preserve the type null value that might have been set before, this way we guard against any order of those properties.
  • (for v3.1, 3.2, we should deserialize the type only)

For serialization:

  • If the type contains null in v2, emit the x-nullable: true extension.
  • If the type contains null in v3, emit the nullable: true extension.
  • For any other version, just emit the type.

For the object model:

  • we should only expose the object model as defined in 3.2, and have fields (as in state) for the things defined in 3.2.

Doing anything else with any/one/all of is a slippery slope, especially when we talk about upgrades etc. We should not be doing any of that.
This also assumes that we're serializing a "null" value for enum correctly, and not silently dropping it.

I hope this clarifies the cleanup at hand. Let me know what you think.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@baywet Does it look better after the last refactoring?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we encounter nullable: true in v2, or x-nullable: true in v3, we should set the type to "add null" (preserving any previous type value) as one of the values. (and drop the extension in v2 to avoid any confusion)

I'm assuming you swapped the version for where nullable and x-nullable belong. But when we encounter it, I do not think we should add "null" to the type right away. We might not know yet if the user has "type" defined in the first place. What I want to make sure is, if I have the following two schemas in 3.0:

{
  "nullable": true,
  "type": "string"
}

and

{
  "nullable": true
}

In the first schema, I expect the OpenApiSchema.Type to be JsonSchemaType.String | JsonSchemaType.Null, but for the second, OpenApiSchema.Type should remain null.

So, we don't always just add JsonSchemaType.Null to the Type, as it's incorrect, because nullable only has a meaning when type is explicitly defined, and so deserialization should take that into account.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to avoid the additional state/field in OpenApiSchema object model, we can instead use the extensions dictionary during deserialization to have an internal extension that we "encountered" nullable. And in FinalizeDeserialization, we always remove the internal extension, and at that point we know whether or not we should add JsonSchemaType.Null.

Comment on lines -521 to -522
IList<IOpenApiSchema>? effectiveOneOf = OneOf;
IList<IOpenApiSchema>? effectiveAnyOf = AnyOf;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For correctness, we are no longer altering/modifying the oneOf/anyOf.
If we find oneOf/anyOf that has a child schema with null type, we will serialize it as enum with a single null element, which is valid for OpenAPI 3.0.

Comment thread src/Microsoft.OpenApi/Reader/V2/OpenApiSchemaDeserializer.cs

@baywet baywet left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution!

Comment thread src/Microsoft.OpenApi/Models/OpenApiSchema.cs Outdated
Comment thread src/Microsoft.OpenApi/Models/OpenApiSchema.cs Outdated
Comment thread src/Microsoft.OpenApi/Models/OpenApiSchema.cs Outdated
Comment thread src/Microsoft.OpenApi/Models/OpenApiSchema.cs Outdated

@baywet baywet left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting closer (I'm only focusing my review on schema object field, and deserialization so far)


schema.Extensions.Remove(OpenApiConstants.NullableExtension);
}
schema.FinalizeDeserialization();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we implementing reading the extension at all in this version? it should only read nullable. The extension should only apply to v2

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@baywet This is pre-existing behavior, I think. I'm not sure why it existed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, and I think it was a mistake to begin with. Added here #1820

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a test asserting the current behavior: LoadSchemaWithNullableExtensionAsV31Works.
I can change the test and behavior though.

o.Type |= JsonSchemaType.Null;
else
o.Type = JsonSchemaType.Null;
o.IsNullableFromDeserialization = true;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we shouldn't maintain a state on the object for that. Revert this change. (original code before the PR)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not following how I should approach fixing the current bugs in the area then.

Basically, we cannot just say "we have nullable set to true, we we are going to expand the Type with JsonSchemaType.Null". That in itself was a bug.

Comment on lines +114 to +121
/// <summary>
/// This property is internal and is only used to mark the "nullable" state when doing deserialization.
/// It's only set during deserialization to flag whether we encountered "nullable" or not.
/// And it's read:
/// 1. During FinalizeDeserialization to determine whether the Type should be changed to include null or not.
/// 2. When serializing nullable, if IsNullable was false and IsNullableFromDeserialization was true, we will still serialize "nullable".
/// </summary>
internal bool IsNullableFromDeserialization { get; set; }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we shouldn't have this field/state

else
o.Type = JsonSchemaType.Null;
}
o.IsNullableFromDeserialization = bool.Parse(value);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert this change (code from before the PR)


schema.Extensions.Remove(OpenApiConstants.NullableExtension);
}
schema.FinalizeDeserialization();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we shouldn't parse the extension in 3.1

else
o.Type = JsonSchemaType.Null;
}
o.IsNullableFromDeserialization = bool.Parse(value);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert this change (before the PR)


schema.Extensions.Remove(OpenApiConstants.NullableExtension);
}
schema.FinalizeDeserialization();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we shouldn't parse the extension in 3.1

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@baywet This is a pre-existing behavior, I think, right?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, and I think it was an error to have it in the first place.

@Youssef1313 Youssef1313 requested a review from baywet July 9, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants