Repository navigation
0.3.1a3: segment clustering, MLX multimodal (all stages), tool calls, cache limit - #47
Merged
Merged
Conversation
… limit - Router: padding mask, telemetry over all micro-batches, lazy decode uses the grouped kernel (fidelity 1.0 at every budget). - Cluster locality: cluster_locality_mode="segment" learns domain clusters per document from content alone (default stays "window"). - MLX: cache limit min(RAM/8, 16 GiB) applied in the kernel; recipe/env can only lower it. Multimodal fusion port with Torch parity and checkpoint round-trip; pretrain/SFT accept multimodal batches. - Tools: chat template renders <tools>/<tool_call>; chat_records writes per-token .lmask supervision read by SFTRecords; serve passes tools. - Monitor/WebUI: progress-based ETA per phase; stopped runs show their state instead of a clock ETA. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
np.where(mask, uint16_seq, -100) stays uint16 under NEP 50 and wraps -100 to 65436; cast the sequence to int32 first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tool-call inference - chronos.serving.tool_calls: streaming <tool_call> parser that never leaks a partial tag; invalid blocks stay content. - /api/chat returns Ollama message.tool_calls; /v1/chat/completions returns OpenAI tool_calls (string arguments, finish_reason "tool_calls"), streamed or not. Requests without tools are unchanged. Multimodal preference stages on MLX - DPO/ORPO: chosen and rejected condition on the prompt's media. - GRPO: rollouts condition on the prompt's media (encoded once, same logits); the policy update re-encodes so encoders train. - Distill: media reach the student; a text-only teacher is accepted. - Data: MultimodalStreamingDPODataset + preference_collate, load_grpo_prompt_records, shared record_media resolver. - Workbench: media data in DPO/ORPO/GRPO selects MLX under auto; an explicit PyTorch backend raises instead of dropping the media. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FonaTech
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cluster_locality_mode="segment"scores each document, split at token-0 separators, as 1 − I(segment; cluster)/log C. The default stays"window".min(RAM/8, 16 GiB)is applied when an MLX model is constructed. A recipe or env setting can only lower it.multimodal_collate. A text-only teacher is accepted.MultimodalStreamingDPODatasetandpreference_collate.autoroutes these stages to MLX, and an explicit PyTorch backend raises an error.<tools>and<tool_call>, and.lmaskcarries per-token supervision.<tool_call>output is parsed into Ollamamessage.tool_callsand OpenAItool_callswithfinish_reason: "tool_calls", streaming and non-streaming. The stream never leaks partial tags, and requests without tools are unchanged..lmaskcontext labels came out as 65436 instead of -100. Labels are now built in int32.Tests
six_stage_loss_paths) segfaults on the MLX CPU backend. Run separately on the GPU, it passes.ruff==0.17.4: clean.test_tool_call_serving: chunking-invariant parsing, a template round-trip, and both APIs streamed and non-streamed.test_mlx_multimodal_stages: data path, tiling parity, a check that media change the loss, rollout encoder reuse, andrun_mlx_stageend to end for dpo, orpo, grpo and distill with checks that fusion and encoder weights move.🤖 Generated with Claude Code