Skip to content

Use image-generation output_format for the result media type - #1132

Open
PratikDhanave (PratikDhanave) wants to merge 2 commits into
microsoft:mainfrom
PratikDhanaveFork:fix/openai-imagegen-output-format
Open

PratikDhanave (PratikDhanave) wants to merge 2 commits into
microsoft:mainfrom
PratikDhanaveFork:fix/openai-imagegen-output-format

Conversation

@PratikDhanave

Copy link
Copy Markdown
Contributor

The Responses image-generation result media type was hardcoded to png on two paths — imageGenerationContents (non-streaming) and the output_item.done non-partial fallback — so an image generated as webp/jpeg was surfaced as image/png. The streaming partial path already does this correctly with cmp.Or(event.OutputFormat, "png").

Python derives the media type from the image (detect_media_type_from_base64(...) or "image/png"); the call also carries output_format.

Change

  • Replace the two "png" literals with cmp.Or(item.OutputFormat, "png") (ResponseOutputItemImageGenerationCall.OutputFormat), matching the partial path.

Test

  • TestResponsesImageGenerationCall_UsesOutputFormat: an image_generation_call with "output_format":"webp" now yields image/webp. Fails before (image/png), passes after.

The non-streaming and output_item.done image-generation paths hardcoded a
png media type on the emitted result, so a webp/jpeg image was mislabeled
image/png. The streaming partial path already derives it from the call's
output_format. Use cmp.Or(item.OutputFormat, "png") at both remaining
sites, matching the partial path and the Python client.
Copilot AI lite review requested due to automatic review settings September 21, 2026 06:14
@github-actions github-actions Bot added area:provider Changes files in the provider area area:provider/openai Changes files in the provider / openai area size:medium At most 100 changed lines across at most 5 files labels Sep 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The streaming fallback path lacks regression coverage for non-PNG output formats.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

Updates image-generation result media types to honor output_format instead of defaulting to PNG.

Changes:

  • Uses OutputFormat for non-streaming and streaming fallback results.
  • Adds WebP regression coverage for non-streaming responses.
File Summary
provider/​openaiprovider/​responses.go Derives image media types from output format.
provider/​openaiprovider/​responses_test.go Tests WebP media type handling.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

// for this item, so emit only the finished result here to avoid a
// duplicate tool call.
u.Contents = []message.Content{imageGenerationResult(item.ID, item.Result, "png", item)}
u.Contents = []message.Content{imageGenerationResult(item.ID, item.Result, cmp.Or(item.OutputFormat, "png"), item)}
@github-actions github-actions Bot added kind:code Changes production behavior or code kind:tests Changes tests, fixtures, or test infrastructure labels Sep 21, 2026
@github-actions

This comment has been minimized.

Add a no-partial streaming test so the output_item.done image-generation
fallback path (not just the non-streaming path) is verified to honor
output_format for the result media type.
@github-actions

Copy link
Copy Markdown
Contributor

Scope: user-visible behavior (internal implementation, but observable via message.DataContent.MediaType)
Changed Go contract: None (no exported functions/types/signatures changed). The unexported imageGenerationResult/imageGenerationContents helpers in provider/openaiprovider/responses.go now use cmp.Or(item.OutputFormat, "png") instead of a hardcoded "png" literal on two paths (non-streaming imageGenerationContents and the streaming output_item.done non-partial fallback), affecting the value of the pre-existing public message.DataContent.MediaType field returned to callers.
Upstream evidence reviewed: python/packages/openai/agent_framework_openai/_chat_client.py_image_generation_item_to_contents (non-streaming/output_item.done fallback path, ~L2337) uses detect_media_type_from_base64(data_str=image_result) or 'image/png'; the response.image_generation_call.partial_image streaming handler (~L3707) uses the same detect_media_type_from_base64(...) or 'image/png' pattern. python/packages/core/agent_framework/_types.py::detect_media_type_from_base64 (L83) is the shared helper. Both Python paths already derive the real media type instead of hardcoding png.
Result: aligned. This PR brings the Go non-streaming and non-partial streaming paths in line with the behavior the streaming partial path already had (cmp.Or(event.OutputFormat, "png"), responses.go L1557), and matches Python's intent of surfacing the actual image format with a png fallback rather than always reporting png. Go derives the format from the API-provided output_format field rather than sniffing base64 magic bytes as Python does, but both achieve the same observable contract: correct media type when known, png otherwise. No exported API changed and no cross-repo divergence found; this is a straightforward parity bugfix with test coverage (TestResponsesImageGenerationCall_UsesOutputFormat, TestResponsesStreamingImageGenerationCall_NoPartial_UsesOutputFormat).

Generated by Go API Consistency Review Agent · copilot · auto · 34.9 AIC · ⌖ 5.3 AIC · ⊞ 9.2K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider/openai Changes files in the provider / openai area area:provider Changes files in the provider area kind:code Changes production behavior or code kind:tests Changes tests, fixtures, or test infrastructure size:medium At most 100 changed lines across at most 5 files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants