Skip to content
#

pretraining-llms

Star

Here are 3 public repositories matching this topic...

Language: All
Filter by language
All 3 Python 2 Jupyter Notebook 1

ksm26 / Pretraining-LLMs

Star 27

Master the essential steps of pretraining large language models (LLMs). Learn to create high-quality datasets, configure model architectures, execute training runs, and assess model performance for efficient and effective LLM pretraining.

machine-learning text-generation data-preparation performance-assessment developer-advocacy ai-training hugging-face large-language-models llm-evaluation model-configuration meta-llama pretraining-llms training-runs high-quality-datasets model-initialization depth-upscaling cost-effective-pretraining
  • Updated Aug 7, 2024
  • Jupyter Notebook

kandivault-source / Sprocket-500M

Star 1

A 501M-parameter language model trained from scratch on 20B tokens: custom BPE tokenizer, original Llama-style architecture, training and SFT pipeline, and unattended cloud orchestration. 54 hours on one H100 for about $165.

nlp machine-learning tokenizer cuda pytorch transformer llama language-model from-scratch bpe pretraining llm pretraining-model fine-tuning-llm h100 gguf bpe-tokenizer pretraining-llms gguf-models gguf-quantization
  • Updated Aug 13, 2026
  • Python

Add this topic to your repo

To associate your repository with the pretraining-llms topic, visit your repo's landing page and select "manage topics."

Learn more