diff --git a/docs-site/assets/kagent-docs/snippets/setup/responses-api-files.md b/docs-site/assets/kagent-docs/snippets/setup/responses-api-files.md new file mode 100644 index 00000000..84a216dc --- /dev/null +++ b/docs-site/assets/kagent-docs/snippets/setup/responses-api-files.md @@ -0,0 +1,11 @@ +| What the message includes | What the runtime sends | +| ------------------------- | ---------------------- | +| An `http://` or `https://` link with an `image/` media type | An `input_image` part with the link. | +| An `http://` or `https://` link with any other media type, such as a PDF | An `input_file` part with the link. | +| Inline image bytes | An `input_image` part with the bytes as a `data:` URL. | +| Inline bytes of a PDF, `.docx`, `.xlsx`, or `.pptx` file | An `input_file` part with the bytes as a `data:` URL. The bytes must match the media type. A file whose bytes do not match, such as a text file labeled `application/pdf`, reaches the model as a note that the file could not be read. When the inline PDF and Office files in the Session total more than 50 MB, the runtime sends the newest files and replaces each older file with a note. | +| Inline bytes of any other type | The file's text, or a note when the file cannot be read as text. | + +The runtime refuses a link whose scheme is not `http` or `https`. When the newest user message includes such a link, the turn fails with `cannot send link "" with media type "": this provider integration can only pass http and https links`. A link in an older message does not fail the turn, because the runtime sends links from older messages as bare URLs. A model with a fetch tool can still retrieve the file. + +When a link or an inline PDF or Office file has a display name, the runtime adds a text part that reads `File: ` immediately before the file part. An inline image does not get this text part. For the file types and sizes that the UI accepts, see [Attach files to a message]({{< link path="observability/launch-ui#attach-files-to-a-message" >}}). diff --git a/docs-site/content/kagent/1.x/operations/upgrade-from-0x.md b/docs-site/content/kagent/1.x/operations/upgrade-from-0x.md index 2386cf48..f910ac68 100644 --- a/docs-site/content/kagent/1.x/operations/upgrade-from-0x.md +++ b/docs-site/content/kagent/1.x/operations/upgrade-from-0x.md @@ -163,11 +163,17 @@ The export splits into three groups: resources that carry forward with small cha ### Resources that carry forward -ModelConfig, ModelProviderConfig, and RemoteMCPServer keep every field that `v1alpha2` sets except one. `v1alpha3` renames the ModelConfig field `apiKeyPassthrough` to `tokenPassthrough`. The field's behavior does not change. `v1alpha3` also adds fields that 0.10.x did not serve, so these resources need no other change to their `spec`. +ModelConfig, ModelProviderConfig, and RemoteMCPServer keep most of the fields that `v1alpha2` sets. `v1alpha3` renames the ModelConfig field `apiKeyPassthrough` to `tokenPassthrough`, and the field's behavior does not change. `v1alpha3` removes the following ModelConfig fields. When a resource sets any of them, the API server rejects the resource and reports the field as unknown. + +- `spec.openAI.tokenExchange` +- `spec.tls.caCertSecretRef` and `spec.tls.caCertSecretKey` +- `spec.tls.disableSystemCAs` + +kagent 1.0 cannot pass a mounted credential or certificate file to an agent. In place of `tokenExchange`, use an endpoint that accepts a static API key. For what to use in place of a custom certificate authority bundle, see [Credentials that kagent rejects]({{< link path="setup/model-providers/about-model-providers#credentials-that-kagent-rejects" >}}). 1. Recreate the Secrets that your ModelConfigs name before you apply them, or the ModelConfigs resolve to nothing. -2. Convert these resources to `v1alpha3`. The export also holds Agents and other kinds that `v1alpha3` does not serve, so the command keeps only ModelConfig, ModelProviderConfig, and RemoteMCPServer resources. The command sets each `apiVersion`, renames the `apiKeyPassthrough` field, and removes the metadata that the 0.10.x API server set. +2. Convert these resources to `v1alpha3`. The export also holds Agents and other kinds that `v1alpha3` does not serve, so the command keeps only ModelConfig, ModelProviderConfig, and RemoteMCPServer resources. The command sets each `apiVersion`, renames the `apiKeyPassthrough` field, and removes the metadata that the 0.10.x API server set. The command does not delete the ModelConfig fields that `v1alpha3` removed, so delete them from the converted file before you apply it. Otherwise, the apply fails. ```bash yq '.items |= map(select(.kind == "ModelConfig" or .kind == "ModelProviderConfig" or .kind == "RemoteMCPServer")) | .items[].apiVersion = "api.kagent.dev/v1alpha3" diff --git a/docs-site/content/kagent/1.x/setup/model-providers/about-model-providers.md b/docs-site/content/kagent/1.x/setup/model-providers/about-model-providers.md index b1c15636..809c07e8 100644 --- a/docs-site/content/kagent/1.x/setup/model-providers/about-model-providers.md +++ b/docs-site/content/kagent/1.x/setup/model-providers/about-model-providers.md @@ -83,7 +83,6 @@ Header injection accepts one shape of credential, a static string. A credential | `AnthropicVertexAI` and `GeminiVertexAI` | A Google service account key is signed locally to obtain a token, and the `kagent` and `byo` harnesses also mount it as a file. | `Anthropic` or `Bedrock` for Claude models, and `Gemini` for Gemini models. See [Google Vertex AI]({{< link path="setup/model-providers/google-vertexai" >}}). | | `SAPAICore` | OAuth2 client credentials are exchanged for a token before any request. | A provider that authenticates with an API key. See [SAP AI Core]({{< link path="setup/model-providers/sap-ai-core" >}}). | | A `credentialRef` in a Harness `spec.env` entry | An arbitrary variable does not name a destination or a header to bind it to. | A ModelConfig or a RemoteMCPServer, each of which carries a destination. See [Agent harness]({{< link path="agents/agent-harness#configure-a-harness" >}}). | -| `openAI.tokenExchange` | The block reads a mounted service account file to acquire a token. | An endpoint that accepts a static API key. See [OpenAI]({{< link path="setup/model-providers/openai" >}}). | | A custom certificate authority bundle | The bundle is mounted as a file, and `tls` defines no field for one. The API server rejects `caCertSecretRef` and `caCertSecretKey` as unknown fields. | An endpoint whose certificate chains to a public CA. Setting `tls.disableVerify: true` skips certificate verification entirely and belongs only in a test environment. | A rejected credential reports one of two messages. A credential that cannot be injected reports `cannot use gateway header injection`, and one that needs a mounted file reports `ModelConfig requires volume mounts unsupported by Substrate ActorTemplate`. diff --git a/docs-site/content/kagent/1.x/setup/model-providers/azure-openai.md b/docs-site/content/kagent/1.x/setup/model-providers/azure-openai.md index c7d46395..c36a2bb0 100644 --- a/docs-site/content/kagent/1.x/setup/model-providers/azure-openai.md +++ b/docs-site/content/kagent/1.x/setup/model-providers/azure-openai.md @@ -94,21 +94,11 @@ spec: ## Send a file to the model -The `responses` format sends a file to Azure as a link or as inline bytes. Azure fetches a link only on the turn that sends it. In later turns, the runtime replaces the link with the bare URL as text. +In the `responses` format, the `kagent` runtime sends a file to Azure as a link or as inline bytes. Azure fetches a link only in the turn that sends it. In later turns, the runtime replaces the link with the bare URL as text. A link reaches Azure regardless of its media type. Inline bytes reach Azure as a file only for an image, a PDF, or a `.docx`, `.xlsx`, or `.pptx` file. Azure documents file input only for PDF files. A `.docx`, `.xlsx`, or `.pptx` file might fail on Azure, although the runtime sends it. -| What the message includes | What the runtime sends | -| ------------------------ | ---------------------- | -| An `http://` or `https://` link with an `image/` media type | An `input_image` part with the link. | -| An `http://` or `https://` link with any other media type, such as a PDF | An `input_file` part with the link. | -| Inline image bytes | An `input_image` part with the bytes as a `data:` URL. | -| Inline bytes of a PDF, `.docx`, `.xlsx`, or `.pptx` file | An `input_file` part with the bytes as a `data:` URL. The bytes must match the media type. A file that does not, such as a text file labeled `application/pdf`, reaches the model as a note that it could not be read. When the files in the conversation total more than 50 MB, the runtime sends the newest files, and a note replaces each older file. | -| Inline bytes of any other type | The file's text, or a note when the file cannot be read as text. | - -The runtime refuses a link on any other scheme. When the newest user message includes such a link, the turn fails with `cannot send link "" with media type "": this provider integration can only pass http and https links`. A link in an older message does not fail the turn, because the runtime sends the links from older messages as bare URLs. A model with a fetch tool can still retrieve the file. - -A link attachment with a display name gets an extra text part that reads `File: ` that is placed immediately before the file part. Azure rejects a filename beside a file URL, so the name reaches the model as ordinary text. +{{< reuse "kagent-docs/snippets/setup/responses-api-files.md" >}} ## Authentication diff --git a/docs-site/content/kagent/1.x/setup/model-providers/openai.md b/docs-site/content/kagent/1.x/setup/model-providers/openai.md index 577f5449..669b75bf 100644 --- a/docs-site/content/kagent/1.x/setup/model-providers/openai.md +++ b/docs-site/content/kagent/1.x/setup/model-providers/openai.md @@ -67,8 +67,13 @@ The `openAI` block takes the following optional settings. For every field, inclu | `n` | How many completions to request. | | `timeout` | How long to wait on a request to the provider. | -> [!WARNING] -> The `openAI` block also accepts `tokenExchange`, which acquires a bearer token from a mounted service account file. That configuration mounts a credential file into the agent, which a Harness does not accept. For the full list of configurations that this affects, see [About model providers]({{< link path="setup/model-providers/about-model-providers" >}}). +The `openAI` block in `v1alpha3` does not include the `tokenExchange` field that 0.x accepted. When a ModelConfig sets `tokenExchange`, the API server rejects the ModelConfig and reports the field as unknown. + +## Send a file to the model + +When `apiFormat` is `responses`, the `kagent` runtime sends supported files through the Responses API without first extracting them as text. The provider fetches a link only in the turn that sends it. In later turns, the runtime replaces the link with the bare URL as text. + +{{< reuse "kagent-docs/snippets/setup/responses-api-files.md" >}} ## Use the ModelConfig