Skip to content

feat: add synchronous and asynchronous OpenAI embeddings - #89

Open
ArjunR00T wants to merge 2 commits into
adaumsilva:mainfrom
ArjunR00T:feat/add-openai-embedder
Open

ArjunR00T wants to merge 2 commits into
adaumsilva:mainfrom
ArjunR00T:feat/add-openai-embedder

Conversation

@ArjunR00T

@ArjunR00T ArjunR00T commented Oct 5, 2026 •

Copy link
Copy Markdown

Description

Adds an OpenAI embedding integration with synchronous and asynchronous APIs, token-aware batching, and clear error handling. Closes #6

Changes

  • Added OpenAIEmbedder with configurable model, API key, token limits, batch size, overflow behavior, and concurrency.
  • Added lazy OpenAI client management with sync/async context managers and close methods.
  • Added a default asynchronous aembed() implementation to the Embedder base class.
  • Exported OpenAIEmbedder from ragframework.embeddings.
  • Added tiktoken to the [openai] optional dependency.
  • Added mocked unit tests for configuration, batching, truncation, ordering, sync/async calls, and error handling.
  • Updated the examples README and added an [Unreleased] changelog entry.

Type of change

  • Bug fix
  • New feature / integration
  • Documentation update
  • Refactor / code quality

Checklist

  • pytest tests/ -v passes locally — collection currently errors in tests/test_document/test_html.py with KeyError: 'file' during pytest parameter-ID generation.
  • ruff check ragframework/ passes
  • mypy ragframework/ passes
  • New or updated tests cover the changes
  • Optional dependencies are guarded with a helpful import error
  • Docstrings updated where applicable
  • CHANGELOG.md updated under [Unreleased]

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI Embeddings Integration

1 participant