Revision Q/A · 复习题解

A query is an imperfect expression of an information need. The same place name may express a tourism, history or navigation task. Decide which need is intended before judging results; keyword overlap alone does not determine relevance.

同一个地名可能对应旅游、历史或导航需求。先明确任务，再判断文档是否有帮助，不能只看关键词重合。

The crawl–index–retrieve pipeline remains useful, but a modern system can add vertical search, semantic retrieval, multi-stage neural ranking, interaction logging, personalisation and conversational interfaces. Sketch these as additional components rather than claiming the 1998 architecture is unchanged.

保留抓取、索引和检索主干，再增加多模态/垂直搜索、神经重排序、反馈学习等模块。第33页给出了老师建议思考的变化方向。

A stemmer maps word forms to a common stem to reduce morphological mismatch. Porter is rule-based suffix stripping. Krovetz combines linguistic rules with dictionary checks; it is not a statistical frequency-based stemmer. Conflation can increase recall but over-stemming can hurt precision.

输入是词形，输出是归一化词干。Porter偏规则截尾，Krovetz结合词典检查；不要把stemming等同于保证输出真实词元的lemmatization。

When terms are ordered by decreasing frequency, frequency is approximately inversely proportional to rank: f(r) ≈ C/r. A few terms dominate occurrences; many terms form a long tail. This motivates careful treatment of frequent terms, compressed postings and rarity-sensitive weighting such as IDF.

词频排名乘词频近似常数。高频词占很多存储和处理成本；低频词往往更有区分力，但低频也可能是拼写错误，并非必然重要。

Inspect corpus document frequencies, existing stoplists and domain queries; propose candidates, then validate on held-out retrieval tasks. Apply compatible analysis at indexing and query time. Alternatives include retaining terms with low IDF weights or using efficient query processing. Do not remove domain-critical words solely because they are frequent.

金融领域的高频词也可能重要，否定词尤其不能随便删除。停用词表需要实验验证；另一条路线是保留词、通过IDF降低权重，而不是直接抹去信息。

Normalisation trades distinctions for matching opportunities. Case folding and stemming can improve recall but conflate different meanings. Adding n-grams can increase the index size. Stemming costs analysis time but can reduce the vocabulary; measure total indexing and retrieval time rather than assuming either becomes faster.

文本归一化可能提高召回，也可能混淆含义。增加 n-gram 往往增加索引体积；stemming 对整体速度的影响需要实测，不能仅凭词表变小下结论。

Users increasingly ask for synthesised answers. Risks include hallucinations, stale knowledge and unverifiable evidence. Retrieval-Augmented Generation (RAG) retrieves sources before generation: query → retrieval → reranking → evidence context → generated answer with citations. Retrieval helps grounding but does not guarantee factual correctness.

画出检索增强生成流程，并分别指出检索漏召回和生成误用证据的风险。能生成流畅答案，不代表信息可靠；搜索仍承担找证据、更新知识的作用。

Worked example: a ranking has relevant documents at positions 1, 3 and 5, with four relevant documents in the collection. P@5=3/5=0.6; recall@5=3/4=0.75; AP=(1+2/3+3/5)/4≈0.5667. Unretrieved relevant documents remain in the AP denominator. MAP averages AP over queries, not competing systems for one query.

示例：相关文档位于第1、3、5位，集合共有4篇相关文档。P@5=0.6，recall@5=0.75，AP约0.5667。MAP是在多个查询之间平均AP，不是在同一查询的两个系统之间平均。

The first relevant result is at rank 1 for q1 and rank 5 for q2. RR values are 1 and 1/5; MRR=(1+0.2)/2=0.6. Later relevant results do not affect Reciprocal Rank.

MRR全称Mean Reciprocal Rank，平均倒数排名。q2共有3篇相关文档并不改变RR；它只问第一个答案多久出现。

Worked example with binary relevance [0,1,1]: DCG@3=1/log2(3)+1/log2(4)≈1.13093. If the collection contains exactly two relevant documents, IDCG@3=1+1/log2(3)≈1.63093, so nDCG@3≈0.69343. Define the relevance-gain convention and use the same query judgments for both DCG and IDCG.

二元相关示例[0,1,1]：DCG约1.13093，理想DCG约1.63093，nDCG约0.69343。理想排序由该查询的相关性判断决定。

Rank-Biased Precision is RBP=(1−p)Σ r_i p^(i−1). The user begins at rank 1 and continues with probability p, giving expected depth 1/(1−p). For p=0.8 and binary relevance [1,0,1], the observed contribution is 0.328 and the unobserved-tail bound is 0.8³=0.512.

p表示继续浏览概率。p=0.8时预期浏览5条；前三位相关性为[1,0,1]时，已观察贡献为0.328，未判断尾部上界为0.512。

Join run entries to qrels by query ID and document ID, then evaluate in run order. Example: relevant IDs are A and C; the returned list is B,A,C. Precision@3=2/3, recall@3=1, RR=1/2 and AP=(1/2+2/3)/2=7/12. Retrieval scores determine order but are not relevance labels.

用query ID和docID连接run与qrels。若相关集合是A、C，返回顺序为B、A、C，则P@3=2/3，Recall@3=1，RR=1/2，AP=7/12。

Implement precision, recall, AP, RR, nDCG and RBP with explicit cutoffs and relevance conventions. Validate perfect rankings, no relevant results, missing judgments and ties against a trusted evaluator. A visual demo is useful for intuition, not a substitute for tests.

先定义二元/分级相关、未判断文档和截断规则，再写函数。页面上的练习用于理解；自己的评价代码需要手算例子和工具对照。

Term-based models exploit overlap, frequency and corpus statistics. Boolean retrieval supplies sets without graded ranking; raw TF weights all terms equally and can favour longer documents. IDF discounts common terms, while BM25 adds term-frequency saturation and length normalisation.

把模型演进串起来：是否匹配 → 匹配次数 → 词的稀有性 → 饱和与长度修正。这些改进仍不自动解决同义词零重合的问题。

For a document containing the terms retrieval, retrieval, model, normalised TF(retrieval)=2/3 and TF(model)=1/3. The denominator is three token occurrences, not two unique terms. Raw TF would instead be 2 and 1.

文档retrieval、retrieval、model有3个token、2个独特词。归一化TF分别是2/3和1/3，raw TF则是2和1。

Using the illustrative convention IDF(t)=ln(N/(1+df(t))), let N=12, df(retrieval)=3 and df(model)=5. IDF values are ln(3)≈1.0986 and ln(2)≈0.6931. Repeated occurrences within one document do not increase document frequency.

示例采用ln(N/(1+df))。N=12，两个词的df为3和5，则IDF为ln3和ln2。同一文档内的重复出现不增加df。

Continue the preceding examples: TF-IDF(retrieval)=(2/3)ln(3)≈0.7324 and TF-IDF(model)=(1/3)ln(2)≈0.2310. The local term count and global rarity play different roles; always state raw versus normalised TF and the log convention.

结合前面示例，两个词TF-IDF约为0.7324和0.2310。局部词频和全局稀有程度承担不同作用，需说明TF归一化与对数约定。

For the query retrieval, only its document weight contributes, giving approximately 0.7324 in the preceding example. For retrieval model, sum both weights to obtain approximately 0.9635. This is a matching-term sum, not cosine normalisation.

查询只有retrieval时得分约0.7324；含retrieval和model时约0.9635。这里使用匹配词权重求和，不是cosine。

Raw term counts can reward repetition without bound. BM25 limits marginal gains with TF saturation and accounts for document length. k1 controls saturation; b controls length normalisation. Some full BM25 formulations also include query frequency and relevance-feedback statistics; do not silently replace an assignment’s specified simplified formula.

k1不是返回结果数量k；b=0不做长度归一化，b=1做完整相对长度修正。给同一个词重复1000次不应无限提高相关性。

BM25 needs lexical overlap after analysis; “laptop” and “notebook” can mismatch. Self-attention forms contextual token representations with softmax(QKᵀ/√d_k)V, letting each token combine evidence from other permitted positions. Contextual representations enable semantic matching but do not guarantee it.

Attention里的Q/K/V是投影后的query/key/value向量，不要把其中Q直接等同于用户搜索句。语义模型需要合适训练才会把表达不同但相关的内容拉近。

A standard Transformer encoder uses bidirectional self-attention. A causal decoder masks future positions. During training, known sequence positions can still be processed in parallel under the causal mask; autoregressive generation produces successive tokens sequentially.

Encoder通常使用双向attention；decoder用因果mask屏蔽未来。因果mask不等于训练时不能并行，但逐token生成需要顺序推进。

BERT means Bidirectional Encoder Representations from Transformers and uses an encoder backbone. GPT means Generative Pre-trained Transformer and uses a decoder backbone. Vocabulary size, sequence length, embedding dimension and training-token count are different quantities; inspect the exact checkpoint before assigning numerical values.

BERT是encoder架构，GPT是decoder架构。词表大小、序列长度、embedding维度与训练token数量是四个不同量。

Add token, position and segment embeddings elementwise. For example, [1,0,2]+[0,1,0]+[1,0,1]=[2,1,3]. Three vectors of width 3 produce one vector of width 3, not 9. A sequence with n tokens and hidden size h has input shape n×h before the batch dimension.

token、position、segment向量逐元素相加，不是拼接。3维向量相加仍然3维；n个token、hidden size为h时形状是n×h。

An encoder can separately embed query/document (bi-encoder) or jointly classify a pair (cross-encoder). A decoder can produce an embedding from a context-aware final token, or judge relevance through generated labels/answer-token likelihoods. Decide whether the output is a reusable representation or a pair-specific score.

骨干架构和检索方式是两条维度：encoder不只做dense retrieval，decoder也不只聊天。能否预计算文档，决定在线成本。

Use CLS pooling, masked mean pooling, or all-token representations. DPR uses pooled query/document embeddings with dot-product scoring; ColBERT keeps token vectors and sums each query token’s maximum document-token similarity. Pooling chooses representation; dot product/cosine/MaxSim chooses matching.

DPR是Dense Passage Retrieval；ColBERT是Contextualized Late Interaction over BERT。Mean pooling不是similarity函数；一条向量和多token向量的索引成本也不同。

A causal decoder’s final/EOS representation can summarise preceding text. Encode query and document separately and compare their embeddings with the model’s trained similarity. Some methods modify attention or add contrastive training. Alternatively joint prompting yields a relevance score, which is reranking rather than reusable dense retrieval.

EOS是End Of Sequence。最后位置已看到前文，因此可用于聚合；不能假设任意未训练的decoder最后向量都天然适合检索。

Assuming temperature τ=1, L=−log(exp(0.9)/(exp(0.9)+exp(0.3)+exp(0.1)+exp(0.4)))≈0.9573065. A different temperature changes the answer. InfoNCE is a contrastive Noise-Contrastive Estimation objective: it rewards the positive relative to the negatives.

正样本也要放进分母。先把所有相似度转成exp，再算正样本占比，最后负log。τ未给出时要声明假设，不能随意采用0.05。

For margin m=1, positive score 1.5 and negative scores 0.3, 0.8 and 1.2, the hinge losses max(0,m−s+ +s−) are 0, 0.3 and 0.7. Their sum is 1.0 and their mean is 1/3. Specify the aggregation convention; pairs already exceeding the margin contribute zero.

正分数1.5，负分数0.3、0.8、1.2，margin=1，则损失为0、0.3、0.7，总和1，均值1/3。超过margin的配对损失为0。

Mine high-scoring non-relevant neighbours with the current dense retriever using Approximate Nearest Neighbour (ANN) search; periodically refresh document embeddings/indexes as the model changes. ANCE uses this idea. Filter known positives and inspect likely false negatives; harder is useful only when the labels are trustworthy.

ANCE全称Approximate Nearest Neighbor Negative Contrastive Learning。BM25找词面相似的困难样本，自挖掘找当前模型最容易混淆的语义近邻。未标注不等于真正不相关。

For TILDE query-likelihood ranking, document-side vocabulary scores are precomputed; queries require tokenisation and lookup rather than a neural query encoder. This shifts work offline. Original TILDE has a large vocabulary-sized document representation; TILDEv2 reduces storage by weighting document/expanded tokens. Do not claim all TILDE variants have no online encoding.

TILDE全称Term Independent Likelihood moDEl。限定QL（Query Likelihood）非常重要；QDL/DL变体的在线计算不同。快的原因是预计算与轻量查询，不是完全没有BERT。

SPLADE (Sparse Lexical and Expansion Model) projects contextual token states through a Masked Language Model (MLM) head into vocabulary dimensions, applies a non-negative log-saturation transform and pools across input tokens. Sparsity regularisation controls active terms. Vocabulary terms absent from the original input may receive weight.

每个输入token预测整个词表上的权重，再按词表维度聚合。ReLU和稀疏正则帮助产生很多零；输出维度对应词，而不是匿名语义轴。

PromptReps uses the last hidden state for a dense representation and vocabulary logits for a sparse representation. Dense ANN retrieval and sparse inverted-index matching supply complementary signals, combined into a hybrid ranking. A neural backbone alone does not make a representation dense.

同一prompt产生两种表示：hidden state走dense路线，词表logits走sparse路线；混合的是检索信号，不是把两个模型名字放在一起。

For already transformed term-activation rows [0.6,0,0.2] and [0.2,0.3,0.4], sum pooling gives [0.8,0.3,0.6]. If starting from raw MLM logits, first apply the model’s non-negative log-saturation transform. The pooled weights are not a probability distribution.

已转换的激活向量[0.6,0,0.2]与[0.2,0.3,0.4]相加得到[0.8,0.3,0.6]。若输入是raw logits，需要先转换；结果不要求和为1。

Using the same activation rows, max pooling gives [0.6,0.3,0.4]. It keeps the strongest evidence for each vocabulary term rather than accumulating all occurrences. This is elementwise pooling over vocabulary dimensions, not selecting one whole token row.

相同输入逐维取max得到[0.6,0.3,0.4]，保留每个词项最强证据，而不是选择某一整行。

Pointwise learning predicts a document label or score; pairwise learning compares two documents; listwise learning uses a ranked list as the training unit. These are training-target categories, not architectural requirements. Metric-aware pairwise objectives also exist, so avoid claiming that only listwise methods can account for a ranking metric.

Pointwise、pairwise、listwise区分训练目标单位，不限定模型架构。部分pairwise方法也能考虑排序指标，避免绝对化表述。

Expensive feature extraction and rich interactions are affordable only on a small candidate set. Early stages may use BM25, dense ANN or learned sparse retrieval; later LTR/rerankers refine the order. Candidate recall is a ceiling: a reranker cannot recover a relevant document absent from its candidates.

前段快且尽量不漏，后段贵但排得更准。LTR是Learning to Rank；既要评价最终nDCG，也要检查第一阶段候选的recall。

Take the deepest displayed clicked document, dmax. Let imin be its smallest rank in the two original lists. Count clicked IDs in each original prefix through imin; the larger count wins and equal counts tie. Example: A=[a,b,c], B=[b,c,a], clicked in displayed order=[b,c]. Here dmax=c and imin=2: A receives one click and B two.

找到显示列表中最深的被点击文档，再取其在两个原排序中的较优位置作为统一截断。例A=[a,b,c]、B=[b,c,a]、点击顺序[b,c]，截断为2，A得1票、B得2票。

Inverse Propensity Scoring corrects exposure or examination bias through inverse-probability weighting. Unbiasedness requires correct propensities, positive support and the assumed feedback model. Small propensities can create high variance; position correction alone does not remove every click bias.

IPS用逆概率权重校正曝光或观察偏差，依赖正确倾向概率、正支持及反馈模型假设。小概率会带来高方差，也不能解决所有点击偏差。

Dueling Bandit Gradient Descent: sample a unit direction u; form candidate θ′=θ+δu; rank with current and candidate models; interleave lists; collect clicks and infer preference; if the candidate wins update θ←θ+αu, otherwise retain θ. δ is exploration distance and α is learning rate. Repeat over interactions.

比较对象是两个ranker，不是单个文档。点击提供偏好信号，用它决定是否沿随机探索方向更新；一次交互不保证nDCG上升。

Counterfactual Online Learning to Rank reuses propensity-logged interactions to compare candidate rankers without a new interleaved display for each comparison. It is off-policy because the evaluated ranker differs from the data-collection policy. It still requires feedback, reliable probabilities and sufficient exploration.

Counterfactual OLTR复用记录了倾向概率的交互日志。评估模型与采集策略不同，因此称off-policy；仍需要交互反馈、可靠概率与充分探索。

monoBERT jointly encodes query and passage, enabling token-level cross-attention before scoring. A single-vector bi-encoder compresses each side independently. Rich joint interaction can improve reranking, but costs a forward pass per pair and cannot precompute a query-independent document score.

更有效不等于每个query都更好。代价是在线计算昂贵；先BM25召回再monoBERT重排，是精度与规模的折中。

The standard BERT input limit is 512 tokens including special tokens and query tokens, not 512 words per document. Long passages must be truncated/chunked. Joint encoding is expensive; performance depends on training-domain match and candidate recall. Softmax output is not automatically a calibrated relevance probability.

长文档截断可能丢掉真正答案；只看top-k候选也可能漏掉相关文档。不要把reranker误当作无需第一阶段的全库搜索器。

Split a document into passages. FirstP uses the first passage; MaxP takes the highest passage score; SumP adds scores but can favour many passages. PARADE (Passage Representation Aggregation for Document Reranking) aggregates query-conditioned passage representations using average, max, attention or Transformers, then predicts a document score.

一条路线先得到每段分数再合并；另一条先合并每段表示再打分。MaxP适合一个关键段就能回答的情况，但可能丢失跨段证据。

Pointwise scores one query–document pair. Pairwise compares two documents and needs a sorting/aggregation strategy. Listwise requests an order for a list, subject to context limits and order bias. Setwise chooses the best among a small set, useful in tournament/heap-style ranking. Setwise can reduce comparisons versus pairwise and avoid generating an entire ordered list; exact costs depend on the algorithm.

记住模型一次看到几个文档、输出什么、如何组成最终排名。Pointwise=单篇分数；pairwise=二选一偏好；listwise=列表顺序；setwise=集合中选优。交换输入顺序可检查位置偏好，生成结果还需验证格式与ID。