Revision Q/A

A query is an imperfect expression of an information need. The same place name may express a tourism, history or navigation task. Decide which need is intended before judging results; keyword overlap alone does not determine relevance.

The crawl–index–retrieve pipeline remains useful, but a modern system can add vertical search, semantic retrieval, multi-stage neural ranking, interaction logging, personalisation and conversational interfaces. Sketch these as additional components rather than claiming the 1998 architecture is unchanged.

A stemmer maps word forms to a common stem to reduce morphological mismatch. Porter is rule-based suffix stripping. Krovetz combines linguistic rules with dictionary checks; it is not a statistical frequency-based stemmer. Conflation can increase recall but over-stemming can hurt precision.

When terms are ordered by decreasing frequency, frequency is approximately inversely proportional to rank: f(r) ≈ C/r. A few terms dominate occurrences; many terms form a long tail. This motivates careful treatment of frequent terms, compressed postings and rarity-sensitive weighting such as IDF.

Inspect corpus document frequencies, existing stoplists and domain queries; propose candidates, then validate on held-out retrieval tasks. Apply compatible analysis at indexing and query time. Alternatives include retaining terms with low IDF weights or using efficient query processing. Do not remove domain-critical words solely because they are frequent.

Normalisation trades distinctions for matching opportunities. Case folding and stemming can improve recall but conflate different meanings. Adding n-grams can increase the index size. Stemming costs analysis time but can reduce the vocabulary; measure total indexing and retrieval time rather than assuming either becomes faster.

Users increasingly ask for synthesised answers. Risks include hallucinations, stale knowledge and unverifiable evidence. Retrieval-Augmented Generation (RAG) retrieves sources before generation: query → retrieval → reranking → evidence context → generated answer with citations. Retrieval helps grounding but does not guarantee factual correctness.

Worked example: a ranking has relevant documents at positions 1, 3 and 5, with four relevant documents in the collection. P@5=3/5=0.6; recall@5=3/4=0.75; AP=(1+2/3+3/5)/4≈0.5667. Unretrieved relevant documents remain in the AP denominator. MAP averages AP over queries, not competing systems for one query.

The first relevant result is at rank 1 for q1 and rank 5 for q2. RR values are 1 and 1/5; MRR=(1+0.2)/2=0.6. Later relevant results do not affect Reciprocal Rank.

Worked example with binary relevance [0,1,1]: DCG@3=1/log2(3)+1/log2(4)≈1.13093. If the collection contains exactly two relevant documents, IDCG@3=1+1/log2(3)≈1.63093, so nDCG@3≈0.69343. Define the relevance-gain convention and use the same query judgments for both DCG and IDCG.

Rank-Biased Precision is RBP=(1−p)Σ r_i p^(i−1). The user begins at rank 1 and continues with probability p, giving expected depth 1/(1−p). For p=0.8 and binary relevance [1,0,1], the observed contribution is 0.328 and the unobserved-tail bound is 0.8³=0.512.

Join run entries to qrels by query ID and document ID, then evaluate in run order. Example: relevant IDs are A and C; the returned list is B,A,C. Precision@3=2/3, recall@3=1, RR=1/2 and AP=(1/2+2/3)/2=7/12. Retrieval scores determine order but are not relevance labels.

Implement precision, recall, AP, RR, nDCG and RBP with explicit cutoffs and relevance conventions. Validate perfect rankings, no relevant results, missing judgments and ties against a trusted evaluator. A visual demo is useful for intuition, not a substitute for tests.

Term-based models exploit overlap, frequency and corpus statistics. Boolean retrieval supplies sets without graded ranking; raw TF weights all terms equally and can favour longer documents. IDF discounts common terms, while BM25 adds term-frequency saturation and length normalisation.

For a document containing the terms retrieval, retrieval, model, normalised TF(retrieval)=2/3 and TF(model)=1/3. The denominator is three token occurrences, not two unique terms. Raw TF would instead be 2 and 1.

Using the illustrative convention IDF(t)=ln(N/(1+df(t))), let N=12, df(retrieval)=3 and df(model)=5. IDF values are ln(3)≈1.0986 and ln(2)≈0.6931. Repeated occurrences within one document do not increase document frequency.

Continue the preceding examples: TF-IDF(retrieval)=(2/3)ln(3)≈0.7324 and TF-IDF(model)=(1/3)ln(2)≈0.2310. The local term count and global rarity play different roles; always state raw versus normalised TF and the log convention.

For the query retrieval, only its document weight contributes, giving approximately 0.7324 in the preceding example. For retrieval model, sum both weights to obtain approximately 0.9635. This is a matching-term sum, not cosine normalisation.

Raw term counts can reward repetition without bound. BM25 limits marginal gains with TF saturation and accounts for document length. k1 controls saturation; b controls length normalisation. Some full BM25 formulations also include query frequency and relevance-feedback statistics; do not silently replace an assignment’s specified simplified formula.

BM25 needs lexical overlap after analysis; “laptop” and “notebook” can mismatch. Self-attention forms contextual token representations with softmax(QKᵀ/√d_k)V, letting each token combine evidence from other permitted positions. Contextual representations enable semantic matching but do not guarantee it.

A standard Transformer encoder uses bidirectional self-attention. A causal decoder masks future positions. During training, known sequence positions can still be processed in parallel under the causal mask; autoregressive generation produces successive tokens sequentially.

BERT means Bidirectional Encoder Representations from Transformers and uses an encoder backbone. GPT means Generative Pre-trained Transformer and uses a decoder backbone. Vocabulary size, sequence length, embedding dimension and training-token count are different quantities; inspect the exact checkpoint before assigning numerical values.

Add token, position and segment embeddings elementwise. For example, [1,0,2]+[0,1,0]+[1,0,1]=[2,1,3]. Three vectors of width 3 produce one vector of width 3, not 9. A sequence with n tokens and hidden size h has input shape n×h before the batch dimension.

An encoder can separately embed query/document (bi-encoder) or jointly classify a pair (cross-encoder). A decoder can produce an embedding from a context-aware final token, or judge relevance through generated labels/answer-token likelihoods. Decide whether the output is a reusable representation or a pair-specific score.

Use CLS pooling, masked mean pooling, or all-token representations. DPR uses pooled query/document embeddings with dot-product scoring; ColBERT keeps token vectors and sums each query token’s maximum document-token similarity. Pooling chooses representation; dot product/cosine/MaxSim chooses matching.

A causal decoder’s final/EOS representation can summarise preceding text. Encode query and document separately and compare their embeddings with the model’s trained similarity. Some methods modify attention or add contrastive training. Alternatively joint prompting yields a relevance score, which is reranking rather than reusable dense retrieval.

Assuming temperature τ=1, L=−log(exp(0.9)/(exp(0.9)+exp(0.3)+exp(0.1)+exp(0.4)))≈0.9573065. A different temperature changes the answer. InfoNCE is a contrastive Noise-Contrastive Estimation objective: it rewards the positive relative to the negatives.

For margin m=1, positive score 1.5 and negative scores 0.3, 0.8 and 1.2, the hinge losses max(0,m−s+ +s−) are 0, 0.3 and 0.7. Their sum is 1.0 and their mean is 1/3. Specify the aggregation convention; pairs already exceeding the margin contribute zero.

Mine high-scoring non-relevant neighbours with the current dense retriever using Approximate Nearest Neighbour (ANN) search; periodically refresh document embeddings/indexes as the model changes. ANCE uses this idea. Filter known positives and inspect likely false negatives; harder is useful only when the labels are trustworthy.

For TILDE query-likelihood ranking, document-side vocabulary scores are precomputed; queries require tokenisation and lookup rather than a neural query encoder. This shifts work offline. Original TILDE has a large vocabulary-sized document representation; TILDEv2 reduces storage by weighting document/expanded tokens. Do not claim all TILDE variants have no online encoding.

SPLADE (Sparse Lexical and Expansion Model) projects contextual token states through a Masked Language Model (MLM) head into vocabulary dimensions, applies a non-negative log-saturation transform and pools across input tokens. Sparsity regularisation controls active terms. Vocabulary terms absent from the original input may receive weight.

PromptReps uses the last hidden state for a dense representation and vocabulary logits for a sparse representation. Dense ANN retrieval and sparse inverted-index matching supply complementary signals, combined into a hybrid ranking. A neural backbone alone does not make a representation dense.

For already transformed term-activation rows [0.6,0,0.2] and [0.2,0.3,0.4], sum pooling gives [0.8,0.3,0.6]. If starting from raw MLM logits, first apply the model’s non-negative log-saturation transform. The pooled weights are not a probability distribution.

Using the same activation rows, max pooling gives [0.6,0.3,0.4]. It keeps the strongest evidence for each vocabulary term rather than accumulating all occurrences. This is elementwise pooling over vocabulary dimensions, not selecting one whole token row.

Pointwise learning predicts a document label or score; pairwise learning compares two documents; listwise learning uses a ranked list as the training unit. These are training-target categories, not architectural requirements. Metric-aware pairwise objectives also exist, so avoid claiming that only listwise methods can account for a ranking metric.

Expensive feature extraction and rich interactions are affordable only on a small candidate set. Early stages may use BM25, dense ANN or learned sparse retrieval; later LTR/rerankers refine the order. Candidate recall is a ceiling: a reranker cannot recover a relevant document absent from its candidates.

Take the deepest displayed clicked document, dmax. Let imin be its smallest rank in the two original lists. Count clicked IDs in each original prefix through imin; the larger count wins and equal counts tie. Example: A=[a,b,c], B=[b,c,a], clicked in displayed order=[b,c]. Here dmax=c and imin=2: A receives one click and B two.

Inverse Propensity Scoring corrects exposure or examination bias through inverse-probability weighting. Unbiasedness requires correct propensities, positive support and the assumed feedback model. Small propensities can create high variance; position correction alone does not remove every click bias.

Dueling Bandit Gradient Descent: sample a unit direction u; form candidate θ′=θ+δu; rank with current and candidate models; interleave lists; collect clicks and infer preference; if the candidate wins update θ←θ+αu, otherwise retain θ. δ is exploration distance and α is learning rate. Repeat over interactions.

Counterfactual Online Learning to Rank reuses propensity-logged interactions to compare candidate rankers without a new interleaved display for each comparison. It is off-policy because the evaluated ranker differs from the data-collection policy. It still requires feedback, reliable probabilities and sufficient exploration.

monoBERT jointly encodes query and passage, enabling token-level cross-attention before scoring. A single-vector bi-encoder compresses each side independently. Rich joint interaction can improve reranking, but costs a forward pass per pair and cannot precompute a query-independent document score.

The standard BERT input limit is 512 tokens including special tokens and query tokens, not 512 words per document. Long passages must be truncated/chunked. Joint encoding is expensive; performance depends on training-domain match and candidate recall. Softmax output is not automatically a calibrated relevance probability.

Split a document into passages. FirstP uses the first passage; MaxP takes the highest passage score; SumP adds scores but can favour many passages. PARADE (Passage Representation Aggregation for Document Reranking) aggregates query-conditioned passage representations using average, max, attention or Transformers, then predicts a document score.

Pointwise scores one query–document pair. Pairwise compares two documents and needs a sorting/aggregation strategy. Listwise requests an order for a list, subject to context limits and order bias. Setwise chooses the best among a small set, useful in tournament/heap-style ranking. Setwise can reduce comparisons versus pairwise and avoid generating an entire ordered list; exact costs depend on the algorithm.