Repository navigation
Python: Add llmman local model sample using the Ollama connector - #14373
Eric Curtin (ericcurtin) wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The new sample includes a curl ... | sh install instruction, which is a risky pattern to recommend in repository samples and should be replaced with a safer pointer to upstream install docs.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a new Python concept sample demonstrating how to run chat completion against a local llmman server by reusing the existing OllamaChatCompletion connector with a different host, and updates the concepts README to surface the new sample alongside other local-model examples.
Changes:
- Add
llmman_chat_completion.pysample usingOllamaChatCompletion(host="http://localhost:17434"). - Update the Local Models section header to mention the Ollama connector and llmman.
- Add the new llmman sample link under Local Models in the concepts README.
File summaries
| File | Description |
|---|---|
| python/samples/concepts/README.md | Updates Local Models section header and adds a link to the new llmman chat sample. |
| python/samples/concepts/local_models/llmman_chat_completion.py | New concept sample showing how to use the Ollama connector against llmman’s Ollama-compatible API host. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| # Install and start the server, then pull a model: | ||
| # curl -fsSL https://raw.githubusercontent.com/llmmanorg/llmman/main/install.sh | sh | ||
| # llmman serve | ||
| # llmman pull gemma4 |
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (1 commit(s)): 7b6d991d5c78
Model: claude-opus-4.8
Overview
This is a documentation/sample-only change: a new local_models concept sample that points the existing OllamaChatCompletion connector at llmman's http://localhost:17434 host, plus a README index entry. No library or public-API code is touched, so there is no compatibility, lifecycle, serialization, or security-boundary risk; the connector constructor args, settings.options usage, and service_url() behavior are all correct and match the vetted sibling samples. The one substantive gap is that the sample documents the llmman server setup but omits the semantic-kernel[ollama] extra it requires, so it fails at import in a standard install.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.
Affected areas: python/samples/concepts/local_models/llmman_chat_completion.py
| # llmman is a local model runner that serves the Ollama API (alongside | ||
| # OpenAI- and Anthropic-compatible ones) on port 17434, so the existing | ||
| # Ollama connector works unchanged; only the host differs. | ||
| # Install and start the server, then pull a model: |
There was a problem hiding this comment.
The setup instructions cover the llmman server but omit the Python dependency this sample needs. Unlike the existing ollama_chat_completion.py (which uses the OpenAI connector), this sample imports OllamaChatCompletion, which imports from ollama import AsyncClient at module load. The ollama package is an optional extra (pyproject.toml:115, ollama = ["ollama ~= 0.4"]), so a user running this sample after a standard pip install semantic-kernel hits ModuleNotFoundError: No module named 'ollama' at line 6 before anything executes. Please document the extra in the setup block, matching the convention in foundry_local_chatbot.py (which documents pip install foundry-local-sdk), e.g.:
# pip install "semantic-kernel[ollama]"
llmman (https://github.com/llmmanorg/llmman) serves the Ollama API on port 17434, so OllamaChatCompletion works against it unchanged; only the host differs. Add a concept sample mirroring the existing local model samples and list it in the concepts README. No library code or public API is changed.
7b6d991 to
503893e
Compare
|
Thank you for the contribution. We are not adding new features to Semantic Kernel (SK) at the moment, so I am closing this PR. Please consider creating a feature request issue in microsoft/agent-framework. |
Motivation and Context
Adds a sample for llmman, a local model runner that serves the Ollama API on port 17434. No library code or public API changes, so no issue was filed (trivial change per CONTRIBUTING).
Description
python/samples/concepts/local_models/llmman_chat_completion.py: new sample usingOllamaChatCompletionwithhost="http://localhost:17434", mirroring the Ollama / LM Studio samples. Reusing the Ollama connector keeps function calling, images andoptionsworking unchanged.python/samples/concepts/README.md: list the sample under "Local Models" and mention the Ollama connector/llmman in the section header.Testing:
ruff check,ruff format --checkandpy_compilepass on the new file;kernel.get_service("local-gpt")resolves to anOllamaChatCompletionwithservice_url()http://localhost:17434. Not run against a live llmman server.Contribution Checklist