How to Train an LLM: The Complete Path from Pretraining to Fine-Tuning

What Are You Actually Training?

Many people imagine training a large language model like this: throw a mountain of documents at a machine, and it becomes smart on its own.

The reality is far more specific. An LLM is essentially a “predict the next token” function—given a piece of text, it outputs a probability distribution over the next token. Training is the process of using data to repeatedly adjust the parameters inside that function so its predictions get better and better.

Once prediction is good enough, the model “incidentally” picks up grammar, facts, reasoning, and even a degree of world knowledge. This explains a counterintuitive fact: nobody explicitly taught the model “what law is”—it simply read enough of it.

So training an LLM requires three things: data, compute, and algorithms. The difference between training methods is fundamentally a different set of trade-offs among these three.

Below, we walk through the mainstream approaches from heaviest to lightest.

The Three Stages of LLM Training

Strictly speaking, a modern LLM is born through three stages. They are not either/or choices—they are an assembly line:

Stage	Data	Goal	Cost
Pretraining	Massive unlabeled text	Learn language and world knowledge	Very high
Supervised fine-tuning (SFT)	Instruction–response pairs	Learn to follow instructions	Medium
Alignment	Human / AI preference data	Helpful, honest, harmless	Medium

Pretraining: the model reads internet-scale corpora by “predicting the next token,” building a general-purpose foundation. This stage consumes the vast majority of the training budget.
Supervised fine-tuning (SFT): the pretrained result is a “base model” that only continues text; it cannot answer questions well. Fine-tuning it on tens of thousands to millions of “instruction → ideal response” examples turns it into a usable assistant.
Alignment: make outputs better match human preferences—more helpful and safer. RLHF and DPO live at this layer.

With this pipeline in mind, the “methods” below are easy to place.

Method 1: Pretraining from Scratch

Start from randomly initialized parameters and train a model end to end.

This is the most thorough and the most expensive approach:

Data: typically more than 1 trillion tokens, mixed from web pages, books, code, and papers
Compute: often thousands of GPUs running for months
Cost: pretraining a mainstream open model generally costs millions to tens of millions of dollars

Examples: the bases of open models like LLaMA, Qwen, DeepSeek, and Mistral were built this way.

For individuals and small teams, pretraining from scratch is essentially infeasible. But if you want to understand the principles, several excellent small-scale reproductions are worth doing:

nanoGPT / nanochat (Andrej Karpathy): a few hundred lines of code that explain GPT training clearly
TinyStories: tiny models trained on minimal English stories that can still tell coherent ones

Method 2: Continued Pretraining

Keep the model; just take an existing base and pretrain it further on your domain corpus.

If you have a large body of medical records, legal documents, or private code and want the model to “absorb” that knowledge, you can continue pretraining on that corpus.

Retains existing abilities while absorbing domain knowledge
Cheaper than from-scratch, but it still updates all parameters, so the compute barrier is real
Suited to teams with massive domain corpora, not individuals with a few hundred Q&A pairs

A key rule of thumb: use continued pretraining or RAG to inject new knowledge; use fine-tuning to change behavior or format.

Method 3: Full Fine-Tuning

Update every parameter of the model using instruction data.

High ceiling: the model can deeply adapt to your task distribution
The price is memory and compute: full fine-tuning of a 7B model typically needs tens of GB of VRAM plus a stack of engineering optimizations
Data quality matters far more than quantity—a few thousand carefully crafted instructions often beat hundreds of thousands of noisy ones

If your dataset is large, your task differs notably from general capability, and you have compute to spare, full fine-tuning is worth it.

Method 4: Parameter-Efficient Fine-Tuning (PEFT)

This is the most realistic entry point for individuals and small teams training “their own small model.” The core idea: freeze most of the original model and train only a small set of new parameters.

LoRA

Inject a pair of low-rank matrices beside the original weight matrices and train only those. Trainable parameters are often just 0.1%–1% of the original model, slashing memory and time while approaching full fine-tuning quality.

QLoRA

On top of LoRA, first quantize the base model to 4-bit, then attach LoRA. This lets a single 24GB GPU fine-tune a 7B–13B model—practically the default setup for hobbyists.

Other PEFT Methods

Method	Idea	Notes
LoRA	Inject low-rank matrices	Most mainstream; best cost/quality balance
QLoRA	4-bit quantize + LoRA	Lowest memory; single-GPU friendly
Adapter	Insert small network layers	Clean structure, slight inference latency
Prefix / Prompt Tuning	Train virtual tokens	Very few parameters, limited ceiling

The core of a LoRA fine-tune looks roughly like this (with Hugging Face PEFT):

from peft import LoraConfig, get_peft_model

config = LoraConfig(
    r=8,                    # low-rank dimension
    lora_alpha=16,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    task_type="CAUSAL_LM",
)

model = get_peft_model(base_model, config)
model.print_trainable_parameters()
# trainable params: 4,194,304 || all params: 6,742,609,920 || 0.06%

Fewer than 0.1% of parameters are trained, yet the model’s behavior changes dramatically—that is the appeal of PEFT.

Method 5: Alignment Training

Fine-tuning makes the model “answer”; alignment makes it “answer in a way people find satisfying.” Main approaches:

RLHF (Reinforcement Learning from Human Feedback): first train a reward model to score outputs, then optimize the main model with PPO. Strong results, but the pipeline is complex and unstable, requiring several models at once.
DPO (Direct Preference Optimization): train directly on “good answer vs. bad answer” preference pairs, skipping the reward model and PPO. Simple, stable, and currently the most common alignment method.
GRPO (Group Relative Policy Optimization): have the model generate a group of answers to the same question and compare them within the group to compute advantages, removing the value network. DeepSeek’s reasoning models use it for reinforcement learning.
RLAIF: replace human annotation with AI-generated feedback to lower the cost of preference data.

General advice: nail SFT first, then consider alignment. If the base is not tuned well, jumping straight to RLHF often just wastes compute.

Method 6: Knowledge Distillation

Let a large model be the teacher and a small one the student.

The student learns not only the standard answers (hard labels) but also the teacher’s full probability distribution (soft labels), and even the teacher’s reasoning process (chain-of-thought distillation).

Goal: approach a 70B model’s performance on a specific task at 7B cost
Commonly used to “compress” large-model ability into a locally deployable small model
Requires access to the teacher’s output distribution, or at least a large batch of high-quality reasoning data

Two Things Often Mistaken for “Training”

RAG (Retrieval-Augmented Generation): store knowledge in a vector database and retrieve it at answer time to feed the model. It changes no parameters at all. To give a model frequently changing private knowledge quickly, RAG is usually cheaper and easier to maintain than fine-tuning.
Prompt engineering: steer the model with prompts. Zero training cost, but capability is bounded by the base model.

One line to tell them apart: RAG adds knowledge, fine-tuning changes behavior, prompting shapes expression.

Engineering Problems You Cannot Avoid

Data: cleaning, deduplication, quality filtering, domain mixing, tokenizer choice—data sets the ceiling
Parallelism: data parallelism (DP), tensor parallelism (TP), pipeline parallelism (PP), ZeRO / FSDP memory optimization
Training frameworks: PyTorch, Hugging Face Transformers / TRL / PEFT, DeepSpeed, Megatron-LM
Low-barrier tools: Unsloth (faster, less memory), Axolotl, LLaMA-Factory (config-driven)
Evaluation and inference: benchmarks, plus deployment options like vLLM / Ollama

If You Just Want to Train “Your Own Small Model”

A recommended path for individuals and small teams:

Do not touch pretraining from scratch—that is a big-lab game
Pick an open small model as the base: Qwen2.5-0.5B/1.5B/7B and Llama 3.2 1B/3B are good starting points
Start with QLoRA SFT—it runs on a single GPU
Begin with a few hundred to a few thousand high-quality instructions; get it working before scaling up
Apply DPO when you need preference alignment
Always keep a fixed evaluation set and compare after every change

Four Common Misconceptions

More data is always better: low-quality data drags a model down; quality beats quantity
Fine-tuning injects new knowledge: fine-tuning is better at changing behavior and output format; use continued pretraining or RAG to add knowledge
Full fine-tuning always beats LoRA: when data is limited, LoRA is often comparable at an order of magnitude lower cost
Training is the finish line: evaluation, deployment, and iteration are where the long-term work actually lies

Summary

Training an LLM is not one move but a spectrum:

Want a general-purpose foundation → pretraining from scratch (big labs only)
Want to add domain knowledge → continued pretraining
Want to change model behavior → SFT (full or LoRA/QLoRA)
Want outputs to better match preferences → DPO / GRPO and other alignment methods
Want to shrink a large model’s ability → distillation
Want private knowledge without changing parameters → RAG

For most people, the genuinely feasible starting point is “an open small model + QLoRA fine-tuning.” Get that loop working and you will find training a model is much closer than you thought.