Open-Weight Text LLMs, From First Principles: Architecture, Training, and Openness
A first-principles guide to decoder-only data flow, GQA, MLA, sparse attention, MoE, MTP, recurrent hybrids, SFT, DPO, GRPO, and distillation—followed by architecture, training, license, and reproducibility comparisons across the major open-weight text LLM families.
LLM Quantization from the Ground Up: Why 4-bit Runs at All, How GPTQ/AWQ/NF4 Squeeze 16 Bits into 4, and What's Inside a GGUF File
A 7B model needs 14 GB in FP16 — but overflowing VRAM is only the first wall. The second: decode must stream every weight for every token, so speed is locked to memory bandwidth. Quantization tears down both walls at once by cutting each weight from 16 bits to 4. This article dissects the mechanism to the bottom: what exponent/mantissa each control, the affine map behind scale and zero-point, why the scale is held hostage by outliers and how group-wise quantization fights back; then the four engineering routes — LLM.int8()'s mixed-precision isolation, GPTQ's second-order Hessian compensation, AWQ/SmoothQuant's scale migration, NF4's normal-quantile grid; closing with what Q4_K_M actually decodes to, why dequant-on-the-fly is faster in a memory-bound kernel, and the iron rule: big model + 4-bit beats small model + FP16.
Rerankers from the Ground Up: Where the Cross-Encoder Score Comes From, the Seven Tuning Knobs, and How to Design Rerank in RAG
To stay precomputable, the bi-encoder squeezes documents into single vectors — so it can't tell "actually answers" from "similar topic." Rerank is the second stage that closes that gap. This article dissects the mechanism to the bottom: how the cross-encoder concatenates query+document for early interaction, how the relevance score (a logit) comes out of [CLS]+linear layer and is carved by MS MARCO + hard negatives; then ColBERT's MaxSim, monoT5's P("true"), RankGPT's listwise permutation generation; closing with the seven tuning knobs (top_k/top_n/max_length/threshold...), the full RAG+rerank flow, and a design checklist.
Word Segmenters from the Ground Up: The Paradigms Behind jieba, the Mainstream Tools, and How They Compare (vs. Subword Tokenizers)
jieba is a segmenter, but on a different level from a subword tokenizer. Chinese has no spaces and splits are ambiguous, so segmentation is fundamentally disambiguation. This article converges segmentation into four underlying paradigms — maximum matching, dictionary+statistics (jieba's DAG+DP+HMM), sequence labeling (HMM→CRF→BiLSTM-CRF→BERT), and Japanese lattice (MeCab) — dissects each mechanism, surveys and compares mainstream tools (jieba/THULAC/pkuseg/HanLP/LTP/CkipTagger/MeCab), and ties back to the previous article on how segmenters and subword tokenizers coexist complementarily in RAG.
Tokenizers from the Ground Up: The Internals and Comparison of BPE, WordPiece, Unigram, and SentencePiece
The model only eats integers, so the tokenizer is the throat between text and model. Starting from why the word/char extremes both fail, we dissect the internal algorithms of BPE (incl. byte-level), WordPiece, and Unigram — merge vs prune, frequency vs probability, character vs byte — clarify that SentencePiece is a framework not an algorithm, and close with a comparison table plus downstream engineering consequences (vocab size, token fertility, glitch tokens) grounded in manufacturing RAG.
PEFT & LLM Fine-Tuning from the Ground Up: From the Memory Bill and Intrinsic Dimension to LoRA, QLoRA, and SFT/RLHF/DPO
Why does a full fine-tune of 7B need hundreds of GB? Starting from Adam's memory bill and intrinsic-dimension theory, we dissect the mechanisms and trade-offs of Adapter, Prefix/Prompt Tuning, LoRA, QLoRA, DoRA, (IA)³, and BitFit, then switch to the orthogonal axis — what SFT, RLHF, and DPO actually shape the model into — and close with a two-axis decision map grounded in manufacturing RAG.