Repository navigation
feat(chatbot): separate answer instructions from prompt matching (#376) - #795
PRAteek-singHWY wants to merge 6 commits into
Conversation
|
Review (with #796): Architecturally sound — separating retrieval prompt from answer instructions (#376) prevents language directives from polluting embeddings. Frontend + API + prompt client changes are coherent. Blockers: Both PRs CONFLICTING on main; #796 is stacked on #795. Large diff (~1k lines on #796). Needs rebase of #795 first, then #796 onto updated #795, then review CI + manual chatbot smoke test. Not merging now — good post-GSoC candidate. |
…swer-language-input # Conflicts: # application/frontend/src/pages/chatbot/chatbot.tsx # application/prompt_client/openai_prompt_client.py # application/prompt_client/prompt_client.py # application/prompt_client/vertex_prompt_client.py # application/web/web_main.py
The OpenAI and Vertex clients this change was written against are gone from main, so instructions now go through chat_prompt.j2. They are user input, so the template gives them their own block limited to language, tone and format, and they are capped at 200 chars. Non-string values fall back to the default instead of crashing. Matching still uses the question alone.
The UI field is one line but the API takes anything, and a newline could close the template's block and open a fake one.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 59 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Summary
fixes #376
Separates answer instructions from the user prompt, so a language or style request ("Answer in Chinese") no longer goes into the embedding used for matching.
Problem
Language and style directives were part of the same text used for embeddings and matching, which made retrieval less accurate.
Changes
Instructionsfield in the chatbot UI, defaultAnswer in English, max 200 chars./rest/v1/completiontakespromptandinstructionsseparately.promptonly.instructionsreach the LLM only, throughchat_prompt.j2, on both the grounded and the fallback path.Ported to the LiteLLM client (2026-11-05)
The OpenAI and Vertex clients this PR was first written against were removed from main, so the change is now in
PromptHandler's LiteLLMcreate_chat_completion/query_llmand inchat_prompt.j2. Main merged in, no conflicts with main.Instructions are user input, so:
ANSWER_INSTRUCTIONSblock, and rule 11 says they only control language, tone and format and never override rules 1 to 10Answer in EnglishTests
application/tests/prompt_client_test.py: embeddings use the prompt only, instructions reach the completion, the fallback gets the default, normalisation, and the rendered prompt carries the block on both paths.application/tests/web_main_test.py::TestMain::test_completion_passes_instructions_separatelyVerification
Merge order
This conflicts with #1161 in
prompt_client.py. #1161 should merge first; I will then merge main in here again and re-addinstructions=on its completion calls.