Four Advanced Retrieval Topics, In Depth: SPLADE Learned Sparse, Chunking Strategies, Weighted RRF, and IK Hot-Reload Ops
The series finale — four advanced topics covered in more detail than before. SPLADE: how BERT's MLM head + ReLU/log saturation/max pooling/FLOPS regularization turns text into an interpretable, semantically-expanded, inverted-index-friendly 'learned sparse vector,' compared with BM25/dense/ColBERT. Chunking: fixed/recursive/semantic/sentence-window/parent-document/propositions/RAPTOR tree summaries/late chunking, each dissected with trade-offs and a comparison table. Weighted RRF: normalization and weight-tuning compared against plain RRF, weighted score, relative score, and LTR fusion, plus how to tune weights on a dev set. IK hot-reload ops: the remote_ext_dict Last-Modified/ETag polling-and-reload mechanism, and the most common pitfall — already-indexed old data must be reindexed.
Tokenizers and the Retrieval Ranking Flow, Hands-On: Custom Dictionaries, jieba & Friends, and a BM25→RRF Walkthrough
Post #4 was the algorithm map; this one is hands-on. It answers three concrete questions: why professional vocabulary (part numbers, process jargon) demands a custom dictionary; how the Chinese segmenters (jieba/CKIP/IK/pkuseg/THULAC) and the subword camp (BPE/WordPiece/Unigram) work under the hood with each one's pros and cons; and how a single query flows from tokenization → BM25 retrieval-and-ranking → dense-vector ranking → RRF fusion — which docs each lane retrieves, how each ranks them, and how to set the parameters — traced end-to-end on a 5-document part-number example. BM25 is built up from how term frequency is counted, why mid-to-low-frequency terms carry the most resolving power (Luhn), and IDF plus k1/b.
The RAG Retrieval Algorithm Playbook: Tokenizers, BM25, MaxSim, RRF, Query Rewriting, CRAG, and More
Embeddings and a vector DB give you dense nearest neighbors, but real RAG retrieval is a relay of algorithms. This post takes them apart stage by stage: subword tokenizers (BPE/WordPiece/SentencePiece), sparse BM25 (with its TF-saturation + length-normalization formula), dense and multi-vector DPR/MaxSim, rank-only normalization-free RRF fusion, pre-retrieval query rewriting (rewrite/multi-query/HyDE/step-back/decomposition), post-retrieval correction (CRAG's evaluator routing and Self-RAG's reflection tokens), and reranking + de-duplication (cross-encoder/MMR). Each with mechanism, formula, and terminal-style ASCII — chained into a full retrieval flow for your pgvector part-number RAG.
How Vector Databases Work: ANN, HNSW, and a Tour of the Engines
Embeddings turn text into vectors, but retrieval means finding nearest neighbors among millions or billions of them in milliseconds — that's the real value of a vector database. This post starts from why brute force fails and why high dimensions break tree indexes, then unpacks HNSW (Hierarchical Navigable Small World) word by word: small-world short+long links, greedy navigation, the skip-list hierarchy, and what M / efConstruction / efSearch each do on the recall↔latency axis. It contrasts IVF/PQ, explains how embeddings plug in via 'same model, same metric' plus metadata pre/post-filtering, then tours pgvector, FAISS, Milvus, Qdrant, Weaviate, Pinecone, and Chroma one by one — landing it in your pgvector RAG.
A Field Guide to Embedding Models: BGE-M3, Decoder-LLMs, and What Sets Them Apart
BGE-M3, multilingual-e5, NV-Embed, OpenAI text-embedding-3, Qwen3-Embedding… models are countless, but the real differences are just three bottom-level axes: output form (dense/sparse/multi-vector, deciding index and precision cost), backbone (encoder vs decoder-LLM, deciding knowledge and retrofit cost), and training recipe (contrastive + hard negatives + instructions + self-distillation). This post spreads the models across those axes with BGE-M3's 'three output forms from one model' as the centerpiece, ties back to last post's embed-then-rerank pipeline, and ends with a derivable selection chain for manufacturing part-number retrieval.
It's All Matrices — So Why Do We Need an Embedding Model?
Computers only know numbers, so text must become vectors first. But one-hot is a vector too, and an embedding lookup is mechanically just one matrix multiply — so what's actually special about an embedding model? This post takes it all the way down with a mechanism / training-objective / product frame: from the distributional hypothesis and word2vec negative sampling, through self-attention, to SBERT prying open the anisotropic cone with contrastive learning — showing that what's special isn't the matrix, but a loss that places every row where geometric distance equals semantic distance, then landing it back on the bi-encoder's role in RAG retrieval.