Revision Q/A · 复习题解

Weeks 1–8 + Weeks 11–13 + Project 2 clarifications · 67 study topics

根据课程知识点独立编写的复习题、讲解与计算示例,不是官方原题或官方答案册。先尝试作答,再展开默认折叠的讲解。本站不发布课件原题文字、截图或原始文件。

Week 1相关知识

Information need, query and relevance

Independent practice question / 独立编写的练习题

Two people enter the same landmark name: one is planning a visit, the other is researching its history. Design a relevance-judging rule for each person. Why would keyword overlap alone give misleading labels?

两个人输入同一个地标名称:一人准备参观,一人研究历史。分别设计相关性判断标准,并解释为什么只看关键词重合会误判。

Show answer / 显示答案 · 展开讲解与示例

A query is an imperfect expression of an information need. The same place name may express a tourism, history or navigation task. Decide which need is intended before judging results; keyword overlap alone does not determine relevance.

同一个地名可能对应旅游、历史或导航需求。先明确任务,再判断文档是否有帮助,不能只看关键词重合。

Week 1相关知识

Search architecture

Independent practice question / 独立编写的练习题

Design a search service that supports web pages, product results and conversational answers. Start with crawling, indexing and retrieval; identify the extra modules needed for these three experiences and explain their roles.

设计同时支持网页、商品结果和对话答案的搜索服务。从抓取、索引、检索出发,补充这三种体验所需的模块,并解释各模块的作用。

Show answer / 显示答案 · 展开讲解与示例

The crawl–index–retrieve pipeline remains useful, but a modern system can add vertical search, semantic retrieval, multi-stage neural ranking, interaction logging, personalisation and conversational interfaces. Sketch these as additional components rather than claiming the 1998 architecture is unchanged.

保留抓取、索引和检索主干,再增加多模态/垂直搜索、神经重排序、反馈学习等模块。第33页给出了老师建议思考的变化方向。

Week 1相关知识

Stemmers

Independent practice question / 独立编写的练习题

An indexing team wants to merge related word forms without merging unrelated meanings. Explain the transformation a stemmer performs, contrast Porter with Krovetz, and identify the precision–recall trade-off the team should test.

索引团队希望合并相关词形,但不希望把不同词义混在一起。说明 stemmer 的输入输出,比较 Porter 与 Krovetz,并指出应测试的 precision–recall 权衡。

Show answer / 显示答案 · 展开讲解与示例

A stemmer maps word forms to a common stem to reduce morphological mismatch. Porter is rule-based suffix stripping. Krovetz combines linguistic rules with dictionary checks; it is not a statistical frequency-based stemmer. Conflation can increase recall but over-stemming can hurt precision.

输入是词形,输出是归一化词干。Porter偏规则截尾,Krovetz结合词典检查;不要把stemming等同于保证输出真实词元的lemmatization。

Week 1相关知识

Zipf’s law

Independent practice question / 独立编写的练习题

A corpus has a few extremely frequent terms and a very long tail of rare ones. Which frequency–rank relationship explains this pattern? Use it to justify one indexing decision and one term-weighting decision.

一个语料库中,少数词极其常见,大量词却很少出现。哪种频率与排名关系解释了这一分布?据此各说明一个索引决策和词项加权决策。

Show answer / 显示答案 · 展开讲解与示例

When terms are ordered by decreasing frequency, frequency is approximately inversely proportional to rank: f(r) ≈ C/r. A few terms dominate occurrences; many terms form a long tail. This motivates careful treatment of frequent terms, compressed postings and rarity-sensitive weighting such as IDF.

词频排名乘词频近似常数。高频词占很多存储和处理成本;低频词往往更有区分力,但低频也可能是拼写错误,并非必然重要。

Week 1相关知识

Domain-specific stoplists

Independent practice question / 独立编写的练习题

You are indexing a technical-support knowledge base. A proposed stoplist removes every frequent word, including words that distinguish failure types. Design a safer selection and evaluation process, and explain when retaining low-weight terms is preferable.

你正在为技术支持知识库建索引。有人建议删除所有高频词,其中包括区分故障类型的词。设计更安全的筛选与评估流程,并说明何时应保留这些词但降低权重。

Show answer / 显示答案 · 展开讲解与示例

Inspect corpus document frequencies, existing stoplists and domain queries; propose candidates, then validate on held-out retrieval tasks. Apply compatible analysis at indexing and query time. Alternatives include retaining terms with low IDF weights or using efficient query processing. Do not remove domain-critical words solely because they are frequent.

金融领域的高频词也可能重要,否定词尤其不能随便删除。停用词表需要实验验证;另一条路线是保留词、通过IDF降低权重,而不是直接抹去信息。

Week 1相关知识

Text processing true/false

Independent practice question / 独立编写的练习题

A search engineer proposes case folding, stemming and adding word bigrams as one package. Predict which changes can merge meanings or enlarge the index, and design measurements rather than assuming indexing and retrieval will become faster.

搜索工程师准备同时启用大小写归一、stemming 和词 bigram。分析哪些变化可能合并不同含义或扩大索引,并设计性能测量,而不是直接假设索引和检索会更快。

Show answer / 显示答案 · 展开讲解与示例

Normalisation trades distinctions for matching opportunities. Case folding and stemming can improve recall but conflate different meanings. Adding n-grams can increase the index size. Stemming costs analysis time but can reduce the vocabulary; measure total indexing and retrieval time rather than assuming either becomes faster.

文本归一化可能提高召回,也可能混淆含义。增加 n-gram 往往增加索引体积;stemming 对整体速度的影响需要实测,不能仅凭词表变小下结论。

Week 1相关知识

Search in the LLM era

Independent practice question / 独立编写的练习题

A helpdesk chatbot answers from model memory but sometimes invents outdated instructions. Draw an evidence-first answering pipeline using IR components. Explain what retrieval improves and which correctness risks remain.

客服聊天机器人只依靠模型记忆,有时编出过期操作说明。用 IR 组件设计先获取证据再回答的流程,说明检索改善了什么,以及仍有哪些正确性风险。

Show answer / 显示答案 · 展开讲解与示例

Users increasingly ask for synthesised answers. Risks include hallucinations, stale knowledge and unverifiable evidence. Retrieval-Augmented Generation (RAG) retrieves sources before generation: query → retrieval → reranking → evidence context → generated answer with citations. Retrieval helps grounding but does not guarantee factual correctness.

画出检索增强生成流程,并分别指出检索漏召回和生成误用证据的风险。能生成流畅答案,不代表信息可靠;搜索仍承担找证据、更新知识的作用。

Week 2相关知识

Precision, AP and MAP

Independent practice question / 独立编写的练习题

A ranking has relevant documents at positions 1, 3 and 5, and the collection contains four relevant documents. Compute P@5, recall@5 and AP. How is MAP different from AP?

某排序在第 1、3、5 位是相关文档,整个集合共有 4 篇相关文档。计算 P@5、recall@5 和 AP。MAP 和 AP 有什么区别?

Show answer / 显示答案 · 展开讲解与示例

Worked example: a ranking has relevant documents at positions 1, 3 and 5, with four relevant documents in the collection. P@5=3/5=0.6; recall@5=3/4=0.75; AP=(1+2/3+3/5)/4≈0.5667. Unretrieved relevant documents remain in the AP denominator. MAP averages AP over queries, not competing systems for one query.

示例:相关文档位于第1、3、5位,集合共有4篇相关文档。P@5=0.6,recall@5=0.75,AP约0.5667。MAP是在多个查询之间平均AP,不是在同一查询的两个系统之间平均。

Week 2相关知识

Reciprocal rank

Independent practice question / 独立编写的练习题

For system S1, the first relevant result is at rank 1 for q1 and at rank 5 for q2. Compute RR for each query and the MRR. Do later relevant results change RR?

系统 S1 在 q1 的第一个相关结果位于第 1 位,在 q2 位于第 5 位。分别计算 RR 和 MRR。后面出现的相关结果会影响 RR 吗?

Show answer / 显示答案 · 展开讲解与示例

The first relevant result is at rank 1 for q1 and rank 5 for q2. RR values are 1 and 1/5; MRR=(1+0.2)/2=0.6. Later relevant results do not affect Reciprocal Rank.

MRR全称Mean Reciprocal Rank,平均倒数排名。q2共有3篇相关文档并不改变RR;它只问第一个答案多久出现。

Week 2相关知识

Normalised DCG

Independent practice question / 独立编写的练习题

With binary relevance [0,1,1] for the top three results and exactly two relevant documents in the collection, compute DCG@3, IDCG@3 and nDCG@3.

前三个结果的二元相关性为 [0,1,1],集合中恰好有 2 篇相关文档。计算 DCG@3、IDCG@3 和 nDCG@3。

Show answer / 显示答案 · 展开讲解与示例

Worked example with binary relevance [0,1,1]: DCG@3=1/log2(3)+1/log2(4)≈1.13093. If the collection contains exactly two relevant documents, IDCG@3=1+1/log2(3)≈1.63093, so nDCG@3≈0.69343. Define the relevance-gain convention and use the same query judgments for both DCG and IDCG.

二元相关示例[0,1,1]:DCG约1.13093,理想DCG约1.63093,nDCG约0.69343。理想排序由该查询的相关性判断决定。

Week 2相关知识

Rank-Biased Precision

Independent practice question / 独立编写的练习题

Write the RBP formula and explain what the persistence parameter p means. For p = 0.8 and binary relevance [1,0,1], compute the observed RBP contribution and the bound on the unobserved tail.

写出 RBP 公式,并解释 persistence 参数 p 的含义。当 p = 0.8、二元相关性为 [1,0,1] 时,计算已观察部分的 RBP 贡献和未观察尾部的上界。

Show answer / 显示答案 · 展开讲解与示例

Rank-Biased Precision is RBP=(1−p)Σ r_i p^(i−1). The user begins at rank 1 and continues with probability p, giving expected depth 1/(1−p). For p=0.8 and binary relevance [1,0,1], the observed contribution is 0.328 and the unobserved-tail bound is 0.8³=0.512.

p表示继续浏览概率。p=0.8时预期浏览5条;前三位相关性为[1,0,1]时,已观察贡献为0.328,未判断尾部上界为0.512。

Week 2相关知识

Working with qrels

Independent practice question / 独立编写的练习题

The qrels mark A and C as relevant and the run returns B, A, C. Compute precision@3, recall@3, RR and AP. Why are retrieval scores not relevance labels?

qrels 中 A 和 C 相关,run 返回的顺序是 B, A, C。计算 precision@3、recall@3、RR 和 AP。为什么 retrieval score 不是 relevance label?

Show answer / 显示答案 · 展开讲解与示例

Join run entries to qrels by query ID and document ID, then evaluate in run order. Example: relevant IDs are A and C; the returned list is B,A,C. Precision@3=2/3, recall@3=1, RR=1/2 and AP=(1/2+2/3)/2=7/12. Retrieval scores determine order but are not relevance labels.

用query ID和docID连接run与qrels。若相关集合是A、C,返回顺序为B、A、C,则P@3=2/3,Recall@3=1,RR=1/2,AP=7/12。

Week 2相关知识

Implement evaluation functions

Independent practice question / 独立编写的练习题

You are reviewing a small ranking-evaluation library before using its results in a report. Specify test cases for perfect rankings, no relevant hits, missing judgments and ties, then explain how to cross-check its metric outputs independently.

你准备在报告中使用一个小型排序评估库。为完美排序、没有相关命中、缺失判断及并列分数设计测试,并说明如何独立交叉核对指标输出。

Show answer / 显示答案 · 展开讲解与示例

Implement precision, recall, AP, RR, nDCG and RBP with explicit cutoffs and relevance conventions. Validate perfect rankings, no relevant results, missing judgments and ties against a trusted evaluator. A visual demo is useful for intuition, not a substitute for tests.

先定义二元/分级相关、未判断文档和截断规则,再写函数。页面上的练习用于理解;自己的评价代码需要手算例子和工具对照。

Week 3相关知识

Lexical models: features and limitations

Independent practice question / 独立编写的练习题

A keyword search service returns an unranked set and later switches to sorting by raw term counts. Diagnose a weakness of each design. Explain how corpus rarity, frequency saturation and length normalisation improve the next version.

关键词搜索服务先返回无序集合,之后改为按原始词频排序。分别诊断两种设计的弱点,并说明全局稀有度、词频饱和及长度归一如何改进下一版。

Show answer / 显示答案 · 展开讲解与示例

Term-based models exploit overlap, frequency and corpus statistics. Boolean retrieval supplies sets without graded ranking; raw TF weights all terms equally and can favour longer documents. IDF discounts common terms, while BM25 adds term-frequency saturation and length normalisation.

把模型演进串起来:是否匹配 → 匹配次数 → 词的稀有性 → 饱和与长度修正。这些改进仍不自动解决同义词零重合的问题。

Week 3相关知识

Normalised term frequency

Independent practice question / 独立编写的练习题

A document contains the tokens retrieval, retrieval, model. Compute the normalised TF of each term. What is the denominator, and how does raw TF differ?

一篇文档的词为 retrieval, retrieval, model。计算每个词的 normalised TF。分母是什么?raw TF 有什么不同?

Show answer / 显示答案 · 展开讲解与示例

For a document containing the terms retrieval, retrieval, model, normalised TF(retrieval)=2/3 and TF(model)=1/3. The denominator is three token occurrences, not two unique terms. Raw TF would instead be 2 and 1.

文档retrieval、retrieval、model有3个token、2个独特词。归一化TF分别是2/3和1/3,raw TF则是2和1。

Week 3相关知识

Inverse document frequency

Independent practice question / 独立编写的练习题

Using IDF(t) = ln(N / (1 + df(t))) with N = 12, df(retrieval) = 3 and df(model) = 5, compute both IDF values. Does repeating a term within one document change df?

使用 IDF(t) = ln(N / (1 + df(t))),N = 12,df(retrieval) = 3,df(model) = 5,计算两个 IDF。一个词在同一文档中重复出现会改变 df 吗?

Show answer / 显示答案 · 展开讲解与示例

Using the illustrative convention IDF(t)=ln(N/(1+df(t))), let N=12, df(retrieval)=3 and df(model)=5. IDF values are ln(3)≈1.0986 and ln(2)≈0.6931. Repeated occurrences within one document do not increase document frequency.

示例采用ln(N/(1+df))。N=12,两个词的df为3和5,则IDF为ln3和ln2。同一文档内的重复出现不增加df。

Week 3相关知识

TF-IDF term weights

Independent practice question / 独立编写的练习题

Continuing the previous two examples, compute the TF-IDF weights of retrieval and model in the document. Which part is local and which is global?

接着前两题,计算这篇文档中 retrieval 和 model 的 TF-IDF 权重。哪一部分是局部信息,哪一部分是全局信息?

Show answer / 显示答案 · 展开讲解与示例

Continue the preceding examples: TF-IDF(retrieval)=(2/3)ln(3)≈0.7324 and TF-IDF(model)=(1/3)ln(2)≈0.2310. The local term count and global rarity play different roles; always state raw versus normalised TF and the log convention.

结合前面示例,两个词TF-IDF约为0.7324和0.2310。局部词频和全局稀有程度承担不同作用,需说明TF归一化与对数约定。

Week 3相关知识

TF-IDF query score

Independent practice question / 独立编写的练习题

Using the TF-IDF weights from the previous question, compute the score of the document for the query “retrieval” and for the query “retrieval model”. Is this cosine similarity?

用上一题的 TF-IDF 权重,分别计算该文档对查询 “retrieval” 和 “retrieval model” 的得分。这是 cosine similarity 吗?

Show answer / 显示答案 · 展开讲解与示例

For the query retrieval, only its document weight contributes, giving approximately 0.7324 in the preceding example. For retrieval model, sum both weights to obtain approximately 0.9635. This is a matching-term sum, not cosine normalisation.

查询只有retrieval时得分约0.7324;含retrieval和model时约0.9635。这里使用匹配词权重求和,不是cosine。

Week 3相关知识

Repeated terms and BM25 components

Independent practice question / 独立编写的练习题

Two pages match the same query: one repeats its keywords, while the other is a much longer page containing many topics. Explain why raw TF can reward both unfairly and assign the appropriate BM25 mechanism to each problem.

两个网页都匹配查询:一个大量重复关键词,另一个篇幅很长且包含众多主题。解释 raw TF 为什么可能不公平地奖励它们,并分别指出 BM25 的对应处理机制。

Show answer / 显示答案 · 展开讲解与示例

Raw term counts can reward repetition without bound. BM25 limits marginal gains with TF saturation and accounts for document length. k1 controls saturation; b controls length normalisation. Some full BM25 formulations also include query frequency and relevance-feedback statistics; do not silently replace an assignment’s specified simplified formula.

k1不是返回结果数量k;b=0不做长度归一化,b=1做完整相对长度修正。给同一个词重复1000次不应无限提高相关性。

Week 4相关知识

Vocabulary mismatch and self-attention

Independent practice question / 独立编写的练习题

For a search about repairing a laptop, a useful document uses “notebook” instead of “laptop”. Contrast the evidence available to lexical matching and contextual token representations, then explain the Q, K and V computation behind self-attention.

搜索笔记本电脑维修时,一篇有用文档使用 notebook 而非 laptop。比较词面匹配与上下文 token 表示可利用的证据,并说明 self-attention 中 Q、K、V 的计算。

Show answer / 显示答案 · 展开讲解与示例

BM25 needs lexical overlap after analysis; “laptop” and “notebook” can mismatch. Self-attention forms contextual token representations with softmax(QKᵀ/√d_k)V, letting each token combine evidence from other permitted positions. Contextual representations enable semantic matching but do not guarantee it.

Attention里的Q/K/V是投影后的query/key/value向量,不要把其中Q直接等同于用户搜索句。语义模型需要合适训练才会把表达不同但相关的内容拉近。

Week 4相关知识

Transformer components

Independent practice question / 独立编写的练习题

Inspect two attention masks: one permits every token to use both left and right context, while the other blocks future positions. Identify their usual roles and distinguish parallel training from token-by-token generation.

检查两种 attention mask:一种允许每个 token 使用左右上下文,另一种屏蔽未来位置。指出它们的典型用途,并区分并行训练与逐 token 生成。

Show answer / 显示答案 · 展开讲解与示例

A standard Transformer encoder uses bidirectional self-attention. A causal decoder masks future positions. During training, known sequence positions can still be processed in parallel under the causal mask; autoregressive generation produces successive tokens sequentially.

Encoder通常使用双向attention;decoder用因果mask屏蔽未来。因果mask不等于训练时不能并行,但逐token生成需要顺序推进。

Week 4相关知识

BERT versus GPT

Independent practice question / 独立编写的练习题

You need one backbone for bidirectional text encoding and another for next-token generation. Assign BERT and GPT to these roles using their attention structure, not model size or tokenisation as the deciding criterion.

你需要一种骨干模型进行双向文本编码,另一种进行 next-token 生成。依据 attention 结构为 BERT 和 GPT 分配角色,不要把模型大小或分词方式当作决定依据。

Show answer / 显示答案 · 展开讲解与示例

BERT means Bidirectional Encoder Representations from Transformers and uses an encoder backbone. GPT means Generative Pre-trained Transformer and uses a decoder backbone. Vocabulary size, sequence length, embedding dimension and training-token count are different quantities; inspect the exact checkpoint before assigning numerical values.

BERT是encoder架构,GPT是decoder架构。词表大小、序列长度、embedding维度与训练token数量是四个不同量。

Week 4相关知识

Input embedding matrix

Independent practice question / 独立编写的练习题

A token embedding is [1,0,2], its position embedding is [0,1,0] and its segment embedding is [1,0,1]. What is the input vector? For n tokens with hidden size h, what is the input shape?

某 token 的 token embedding 为 [1,0,2],position embedding 为 [0,1,0],segment embedding 为 [1,0,1]。输入向量是什么?n 个 token、hidden size 为 h 时输入形状是什么?

Show answer / 显示答案 · 展开讲解与示例

Add token, position and segment embeddings elementwise. For example, [1,0,2]+[0,1,0]+[1,0,1]=[2,1,3]. Three vectors of width 3 produce one vector of width 3, not 9. A sequence with n tokens and hidden size h has input shape n×h before the batch dimension.

token、position、segment向量逐元素相加,不是拼接。3维向量相加仍然3维;n个token、hidden size为h时形状是n×h。

Week 4相关知识

Use encoders/decoders for ranking

Independent practice question / 独立编写的练习题

Your retrieval system must precompute document representations, but its second stage can afford a joint query–document pass. Propose encoder-based and decoder-based designs and label each output as a reusable embedding or a pair-specific relevance score.

检索系统需要预计算文档表示,但第二阶段可以承受 query–document 联合计算。分别提出 encoder 和 decoder 方案,并标出输出是可复用 embedding 还是特定配对的相关性分数。

Show answer / 显示答案 · 展开讲解与示例

An encoder can separately embed query/document (bi-encoder) or jointly classify a pair (cross-encoder). A decoder can produce an embedding from a context-aware final token, or judge relevance through generated labels/answer-token likelihoods. Decide whether the output is a reusable representation or a pair-specific score.

骨干架构和检索方式是两条维度:encoder不只做dense retrieval,decoder也不只聊天。能否预计算文档,决定在线成本。

Week 5相关知识

Encoder text representations

Independent practice question / 独立编写的练习题

A team is choosing between one vector per text and one vector per token. Trace how DPR and ColBERT turn encoder outputs into retrieval scores, separating representation pooling from the final matching operation.

团队正在选择每段文本一个向量,还是每个 token 一个向量。追踪 DPR 和 ColBERT 从 encoder 输出到检索分数的数据流,并区分表示 pooling 与最终匹配操作。

Show answer / 显示答案 · 展开讲解与示例

Use CLS pooling, masked mean pooling, or all-token representations. DPR uses pooled query/document embeddings with dot-product scoring; ColBERT keeps token vectors and sums each query token’s maximum document-token similarity. Pooling chooses representation; dot product/cosine/MaxSim chooses matching.

DPR是Dense Passage Retrieval;ColBERT是Contextualized Late Interaction over BERT。Mean pooling不是similarity函数;一条向量和多token向量的索引成本也不同。

Week 5相关知识

Decoder similarities

Independent practice question / 独立编写的练习题

A decoder model is offered in two search designs: separately encode texts for an ANN index, or jointly prompt it with a query and each candidate. Explain what each design produces, what can be cached, and which belongs to reranking.

同一个 decoder 被用于两种搜索设计:分别编码文本建立 ANN 索引,或把查询与每个候选一起输入 prompt。解释两种设计的输出、可缓存内容,以及哪一种属于 reranking。

Show answer / 显示答案 · 展开讲解与示例

A causal decoder’s final/EOS representation can summarise preceding text. Encode query and document separately and compare their embeddings with the model’s trained similarity. Some methods modify attention or add contrastive training. Alternatively joint prompting yields a relevance score, which is reranking rather than reusable dense retrieval.

EOS是End Of Sequence。最后位置已看到前文,因此可用于聚合;不能假设任意未训练的decoder最后向量都天然适合检索。

Week 5相关知识

InfoNCE calculation

Independent practice question / 独立编写的练习题

Given sim(q, p+) = 0.9 and negatives 0.3, 0.1 and 0.4, compute the InfoNCE loss with temperature τ = 1. What happens if τ changes?

已知 sim(q, p+) = 0.9,负例相似度为 0.3、0.1、0.4,在温度 τ = 1 时计算 InfoNCE loss。τ 改变时会怎样?

Show answer / 显示答案 · 展开讲解与示例

Assuming temperature τ=1, L=−log(exp(0.9)/(exp(0.9)+exp(0.3)+exp(0.1)+exp(0.4)))≈0.9573065. A different temperature changes the answer. InfoNCE is a contrastive Noise-Contrastive Estimation objective: it rewards the positive relative to the negatives.

正样本也要放进分母。先把所有相似度转成exp,再算正样本占比,最后负log。τ未给出时要声明假设,不能随意采用0.05。

Week 5相关知识

Margin loss calculation

Independent practice question / 独立编写的练习题

With margin m = 1, positive score 1.5 and negative scores 0.3, 0.8 and 1.2, compute each hinge loss, their sum and their mean.

margin m = 1,正例分数 1.5,负例分数 0.3、0.8、1.2。计算每个 hinge loss,以及它们的和与平均值。

Show answer / 显示答案 · 展开讲解与示例

For margin m=1, positive score 1.5 and negative scores 0.3, 0.8 and 1.2, the hinge losses max(0,m−s+ +s−) are 0, 0.3 and 0.7. Their sum is 1.0 and their mean is 1/3. Specify the aggregation convention; pairs already exceeding the margin contribute zero.

正分数1.5,负分数0.3、0.8、1.2,margin=1,则损失为0、0.3、0.7,总和1,均值1/3。超过margin的配对损失为0。

Week 5相关知识

Harder negatives

Independent practice question / 独立编写的练习题

A dense retriever already separates its BM25-mined negatives but still confuses semantically close passages. Design a refreshed ANN-based negative-mining loop and explain how false negatives can undermine the training signal.

一个 dense retriever 已能区分 BM25 挖出的负例,却仍混淆语义相近的段落。设计持续更新的 ANN 负例挖掘流程,并解释 false negatives 如何破坏训练信号。

Show answer / 显示答案 · 展开讲解与示例

Mine high-scoring non-relevant neighbours with the current dense retriever using Approximate Nearest Neighbour (ANN) search; periodically refresh document embeddings/indexes as the model changes. ANCE uses this idea. Filter known positives and inspect likely false negatives; harder is useful only when the labels are trustworthy.

ANCE全称Approximate Nearest Neighbor Negative Contrastive Learning。BM25找词面相似的困难样本,自挖掘找当前模型最容易混淆的语义近邻。未标注不等于真正不相关。

Week 6相关知识

Why TILDE is efficient

Independent practice question / 独立编写的练习题

A search endpoint has a strict online latency budget but can do substantial work when documents are indexed. Explain how TILDE moves neural computation offline and how TILDEv2 changes the storage trade-off.

搜索接口的在线延迟预算很紧,但建索引时可以进行大量计算。解释 TILDE 如何把神经计算移到离线,以及 TILDEv2 如何改变存储权衡。

Show answer / 显示答案 · 展开讲解与示例

For TILDE query-likelihood ranking, document-side vocabulary scores are precomputed; queries require tokenisation and lookup rather than a neural query encoder. This shifts work offline. Original TILDE has a large vocabulary-sized document representation; TILDEv2 reduces storage by weighting document/expanded tokens. Do not claim all TILDE variants have no online encoding.

TILDE全称Term Independent Likelihood moDEl。限定QL(Query Likelihood)非常重要;QDL/DL变体的在线计算不同。快的原因是预计算与轻量查询,不是完全没有BERT。

Week 6相关知识

SPLADE term weights

Independent practice question / 独立编写的练习题

A learned sparse index assigns weight to a useful query term that never appeared literally in a document. Trace a SPLADE-style path from contextual token states to vocabulary weights, including the transformation, pooling and sparsity control.

一个 learned sparse 索引给文档中未直接出现的有用查询词分配了权重。按 SPLADE 风格追踪上下文 token 状态到词表权重的流程,包括变换、pooling 和稀疏控制。

Show answer / 显示答案 · 展开讲解与示例

SPLADE (Sparse Lexical and Expansion Model) projects contextual token states through a Masked Language Model (MLM) head into vocabulary dimensions, applies a non-negative log-saturation transform and pools across input tokens. Sparsity regularisation controls active terms. Vocabulary terms absent from the original input may receive weight.

每个输入token预测整个词表上的权重,再按词表维度聚合。ReLU和稀疏正则帮助产生很多零;输出维度对应词,而不是匿名语义轴。

Week 6相关知识

PromptReps hybrid retrieval

Independent practice question / 独立编写的练习题

You can extract both hidden states and vocabulary logits from a prompted model. Map them to a PromptReps-style dense branch and sparse branch, and identify the index and matching operation needed by each.

你可以从 prompted model 中提取 hidden states 和词表 logits。把它们分别接入 PromptReps 风格的 dense 与 sparse 分支,并指出每个分支需要的索引和匹配操作。

Show answer / 显示答案 · 展开讲解与示例

PromptReps uses the last hidden state for a dense representation and vocabulary logits for a sparse representation. Dense ANN retrieval and sparse inverted-index matching supply complementary signals, combined into a hybrid ranking. A neural backbone alone does not make a representation dense.

同一prompt产生两种表示:hidden state走dense路线,词表logits走sparse路线;混合的是检索信号,不是把两个模型名字放在一起。

Week 6相关知识

SPLADEv1 sum pooling

Independent practice question / 独立编写的练习题

Two tokens have already-transformed activation rows [0.6, 0, 0.2] and [0.2, 0.3, 0.4]. Compute the SPLADEv1 sum-pooled weights. Are they a probability distribution?

两个 token 已经变换后的激活行为 [0.6, 0, 0.2] 和 [0.2, 0.3, 0.4]。计算 SPLADEv1 的 sum pooling 结果。它是概率分布吗?

Show answer / 显示答案 · 展开讲解与示例

For already transformed term-activation rows [0.6,0,0.2] and [0.2,0.3,0.4], sum pooling gives [0.8,0.3,0.6]. If starting from raw MLM logits, first apply the model’s non-negative log-saturation transform. The pooled weights are not a probability distribution.

已转换的激活向量[0.6,0,0.2]与[0.2,0.3,0.4]相加得到[0.8,0.3,0.6]。若输入是raw logits,需要先转换;结果不要求和为1。

Week 6相关知识

SPLADEv2 max pooling

Independent practice question / 独立编写的练习题

Using the same rows [0.6, 0, 0.2] and [0.2, 0.3, 0.4], compute the SPLADEv2 max-pooled weights and explain how max pooling differs from sum pooling.

使用同样的两行 [0.6, 0, 0.2] 和 [0.2, 0.3, 0.4],计算 SPLADEv2 的 max pooling 结果,并说明 max pooling 和 sum pooling 的区别。

Show answer / 显示答案 · 展开讲解与示例

Using the same activation rows, max pooling gives [0.6,0.3,0.4]. It keeps the strongest evidence for each vocabulary term rather than accumulating all occurrences. This is elementwise pooling over vocabulary dimensions, not selecting one whole token row.

相同输入逐维取max得到[0.6,0.3,0.4],保留每个词项最强证据,而不是选择某一整行。

Week 7相关知识

Offline Learning to Rank

Independent practice question / 独立编写的练习题

Three training pipelines consume a single labelled document, a preferred document pair, and a ranked list respectively. Classify their LTR learning units and assess the claim that a pairwise objective cannot account for nDCG.

三个训练流程分别接收单篇带标签文档、具有偏好关系的文档对,以及排序列表。按 LTR 学习单位分类,并评价“pairwise 目标不能考虑 nDCG”这一说法。

Show answer / 显示答案 · 展开讲解与示例

Pointwise learning predicts a document label or score; pairwise learning compares two documents; listwise learning uses a ranked list as the training unit. These are training-target categories, not architectural requirements. Metric-aware pairwise objectives also exist, so avoid claiming that only listwise methods can account for a ranking metric.

Pointwise、pairwise、listwise区分训练目标单位,不限定模型架构。部分pairwise方法也能考虑排序指标,避免绝对化表述。

Week 7相关知识

Cascade ranking

Independent practice question / 独立编写的练习题

A search service proposes running its most expensive ranker over the full collection. Redesign it as a candidate-generation and reranking cascade; explain the latency benefit and the failure that even a perfect final ranker cannot repair.

搜索服务计划让最昂贵的 ranker 处理整个语料库。把它改为候选生成加 reranking 的级联结构,并解释延迟收益,以及即使最终 ranker 完美也无法补救的失误。

Show answer / 显示答案 · 展开讲解与示例

Expensive feature extraction and rich interactions are affordable only on a small candidate set. Early stages may use BM25, dense ANN or learned sparse retrieval; later LTR/rerankers refine the order. Candidate recall is a ceiling: a reranker cannot recover a relevant document absent from its candidates.

前段快且尽量不漏,后段贵但排得更准。LTR是Learning to Rank;既要评价最终nDCG,也要检查第一阶段候选的recall。

Week 7相关知识

Worked interleaving preference

Independent practice question / 独立编写的练习题

Ranker A = [a, b, c], ranker B = [b, c, a]. In the interleaved display the user clicks b and c. Using balanced interleaving, which ranker is preferred?

Ranker A = [a, b, c],ranker B = [b, c, a]。在 interleaved 列表中用户点击了 b 和 c。按照 balanced interleaving,哪个 ranker 更受偏好?

Show answer / 显示答案 · 展开讲解与示例

Take the deepest displayed clicked document, dmax. Let imin be its smallest rank in the two original lists. Count clicked IDs in each original prefix through imin; the larger count wins and equal counts tie. Example: A=[a,b,c], B=[b,c,a], clicked in displayed order=[b,c]. Here dmax=c and imin=2: A receives one click and B two.

找到显示列表中最深的被点击文档,再取其在两个原排序中的较优位置作为统一截断。例A=[a,b,c]、B=[b,c,a]、点击顺序[b,c],截断为2,A得1票、B得2票。

Week 7相关知识

IPS estimator

Independent practice question / 独立编写的练习题

Two equally relevant results receive different click rates because one is displayed near the top. Explain how IPS uses examination propensities to compensate, and state why tiny or incorrect propensities can make the estimate unreliable.

两个同样相关的结果因为展示位置不同而获得不同点击率。解释 IPS 如何用 examination propensity 补偿,并说明很小或错误的 propensity 为什么会使估计不可靠。

Show answer / 显示答案 · 展开讲解与示例

Inverse Propensity Scoring corrects exposure or examination bias through inverse-probability weighting. Unbiasedness requires correct propensities, positive support and the assumed feedback model. Small propensities can create high variance; position correction alone does not remove every click bias.

IPS用逆概率权重校正曝光或观察偏差,依赖正确倾向概率、正支持及反馈模型假设。小概率会带来高方差,也不能解决所有点击偏差。

Week 7相关知识

DBGD with balanced interleaving

Independent practice question / 独立编写的练习题

A live ranker has parameter vector θ and proposes a nearby challenger θ′ = θ + δu. Design one DBGD comparison-and-update cycle with interleaving; explain when θ changes and why exploration distance and learning rate are separate settings.

在线 ranker 的参数为 θ,它提出附近的挑战者 θ′ = θ + δu。设计一次带 interleaving 的 DBGD 比较与更新循环,说明何时更新 θ,以及为何探索距离与学习率需要分开设置。

Show answer / 显示答案 · 展开讲解与示例

Dueling Bandit Gradient Descent: sample a unit direction u; form candidate θ′=θ+δu; rank with current and candidate models; interleave lists; collect clicks and infer preference; if the candidate wins update θ←θ+αu, otherwise retain θ. δ is exploration distance and α is learning rate. Repeat over interactions.

比较对象是两个ranker,不是单个文档。点击提供偏好信号,用它决定是否沿随机探索方向更新;一次交互不保证nDCG上升。

Week 7相关知识

Counterfactual OLTR

Independent practice question / 独立编写的练习题

You have interaction logs from a known logging policy and want to compare new rankers without a new interleaved display for every comparison. Explain the counterfactual approach and why logged clicks alone are not sufficient evidence.

你拥有已知 logging policy 产生的交互日志,希望比较新 ranker,而不是每次都重新做 interleaved 展示。解释 counterfactual 方法,并说明为何仅有点击日志还不足以可靠比较。

Show answer / 显示答案 · 展开讲解与示例

Counterfactual Online Learning to Rank reuses propensity-logged interactions to compare candidate rankers without a new interleaved display for each comparison. It is off-policy because the evaluated ranker differs from the data-collection policy. It still requires feedback, reliable probabilities and sufficient exploration.

Counterfactual OLTR复用记录了倾向概率的交互日志。评估模型与采集策略不同,因此称off-policy;仍需要交互反馈、可靠概率与充分探索。

Week 8相关知识

monoBERT versus dense retrieval

Independent practice question / 独立编写的练习题

A first-stage bi-encoder retrieves candidates quickly, but a joint query–passage model resolves subtle term interactions better. Explain the representational advantage of monoBERT and the per-candidate computation that prevents using it like an ANN retriever.

第一阶段 bi-encoder 快速取得候选,但联合 query–passage 模型更能处理细微词项交互。解释 monoBERT 的表示优势,以及使它不能像 ANN retriever 一样使用的逐候选计算成本。

Show answer / 显示答案 · 展开讲解与示例

monoBERT jointly encodes query and passage, enabling token-level cross-attention before scoring. A single-vector bi-encoder compresses each side independently. Rich joint interaction can improve reranking, but costs a forward pass per pair and cannot precompute a query-independent document score.

更有效不等于每个query都更好。代价是在线计算昂贵;先BM25召回再monoBERT重排,是精度与规模的折中。

Week 8相关知识

monoBERT limitations

Independent practice question / 独立编写的练习题

A team feeds long documents into monoBERT and interprets its softmax outputs as calibrated relevance probabilities. Identify the token-budget, runtime, candidate-recall and calibration issues they must check before trusting the rankings.

团队把长文档输入 monoBERT,并把 softmax 输出当作已校准的相关性概率。指出信任排序前应检查的 token 预算、运行成本、候选 recall 及校准问题。

Show answer / 显示答案 · 展开讲解与示例

The standard BERT input limit is 512 tokens including special tokens and query tokens, not 512 words per document. Long passages must be truncated/chunked. Joint encoding is expensive; performance depends on training-domain match and candidate recall. Softmax output is not automatically a calibrated relevance probability.

长文档截断可能丢掉真正答案;只看top-k候选也可能漏掉相关文档。不要把reranker误当作无需第一阶段的全库搜索器。

Week 8相关知识

Handle long documents: MaxP and PARADE

Independent practice question / 独立编写的练习题

In a long report, the useful evidence might occur in the first passage, one later passage, or several passages together. Compare FirstP, MaxP and SumP, then explain how PARADE changes the point at which passage evidence is combined.

一篇长报告的有用证据可能位于第一段、后面的某一段,或分散在多个段落。比较 FirstP、MaxP、SumP,并说明 PARADE 在哪个阶段合并段落证据。

Show answer / 显示答案 · 展开讲解与示例

Split a document into passages. FirstP uses the first passage; MaxP takes the highest passage score; SumP adds scores but can favour many passages. PARADE (Passage Representation Aggregation for Document Reranking) aggregates query-conditioned passage representations using average, max, attention or Transformers, then predicts a document score.

一条路线先得到每段分数再合并;另一条先合并每段表示再打分。MaxP适合一个关键段就能回答的情况,但可能丢失跨段证据。

Week 8相关知识

Four decoder reranking methods

Independent practice question / 独立编写的练习题

You must rerank a candidate set with an LLM under a limited inference budget. Compare scoring one document, a pair, a complete list and a small set at a time; explain the comparison-count and output-generation trade-offs.

你需要在有限推理预算内用 LLM 重排候选。比较每次处理单篇、文档对、完整列表及小集合的方法,并解释比较次数和输出生成方式的权衡。

Show answer / 显示答案 · 展开讲解与示例

Pointwise scores one query–document pair. Pairwise compares two documents and needs a sorting/aggregation strategy. Listwise requests an order for a list, subject to context limits and order bias. Setwise chooses the best among a small set, useful in tournament/heap-style ranking. Setwise can reduce comparisons versus pairwise and avoid generating an entire ordered list; exact costs depend on the algorithm.

记住模型一次看到几个文档、输出什么、如何组成最终排名。Pointwise=单篇分数;pairwise=二选一偏好;listwise=列表顺序;setwise=集合中选优。交换输入顺序可检查位置偏好,生成结果还需验证格式与ID。

Week 11 module supplement相关知识

Link authority and anchor text

Independent practice question / 独立编写的练习题

How can hyperlinks and anchor text each help ranking? What risk do they introduce?

hyperlink 和 anchor text 分别怎样帮助排序?它们带来什么风险?

Show answer / 显示答案 · 展开讲解与示例

A hyperlink can act as an authority signal, while its anchor text can describe the target with words absent from that page. Incoming anchor text may be indexed for the target and weighted by source authority.

链接既可能把 authority 赋给目标页,anchor text 又可能用目标页没有出现的词描述它。可把 incoming anchor text 加入目标页索引并按来源权威度加权,但 link farm 和操纵锚文本会产生 spam。

Week 11 module supplement相关知识

PageRank calculation logic

Independent practice question / 独立编写的练习题

Explain how a page receives PageRank from its in-links in one update. Does the page with the most in-links always win?

说明一次更新中,一个页面怎样从入链获得 PageRank。入链最多的页面一定得分最高吗?

Show answer / 显示答案 · 展开讲解与示例

A page receives PageRank mass from each in-linking page in proportion to that source's score divided by its out-degree, plus teleport mass. Repeat the update until convergence.

每个 in-link 来源把自己的 PageRank 按 out-degree 分配,再加 teleport mass,反复迭代直到收敛。In-link 更多不一定分数更高;来源质量、来源出链数、damping 与 dangling-page 处理都会影响结果。

Week 11 module supplement相关知识

PageRank versus relevance

Independent practice question / 独立编写的练习题

How is PageRank different from a query-dependent relevance score such as BM25? Can a high-PageRank page be irrelevant?

PageRank 和 BM25 这类依赖查询的相关性分数有什么不同?PageRank 高的页面可能不相关吗?

Show answer / 显示答案 · 展开讲解与示例

PageRank is query-independent authority from link structure. BM25, dense retrieval or another content model scores the current query.

高 PageRank 表示在链接图中更有权威,不代表对当前 query 相关。实际 ranker 需要组合 link、content、freshness 等信号,不能让 PageRank 直接替代 query matching。

Week 11 module supplement相关知识

PRP assumptions and MMR diversification

Independent practice question / 独立编写的练习题

What does the Probability Ranking Principle assume? When do those assumptions fail, and how does MMR respond?

Probability Ranking Principle 有哪些假设?这些假设什么时候不成立?MMR 怎样应对?

Show answer / 显示答案 · 展开讲解与示例

The Probability Ranking Principle orders documents by estimated relevance probability but assumes accurate estimates and relevance independent of already ranked results. MMR balances query relevance against similarity to selected documents.

PRP 按估计相关概率排序,但假设估计准确,并假设当前文档 relevance 独立于已排结果;ambiguity 与 redundancy 会破坏这些条件。MMR 则在 relevance 和对已选结果的 similarity 之间折中。

Week 11 · Worked practice回看讲解与来源

Why must PageRank preserve probability mass? / PageRank 为什么不能丢概率?

Independent practice question / 独立编写的练习题

Links: A → B, C; B → C; C has no out-links. Start from a uniform vector with d = 0.85 and compute one PageRank update. Why must C’s mass be redistributed?

链接为 A → B, C;B → C;C 没有出链。从均匀向量出发、d = 0.85,计算一次 PageRank 更新。为什么必须重新分配 C 的概率质量?

Show answer / 显示答案 · Explanation / 解释与例子

For A → B,C; B → C; C dangling, start uniformly and set d=0.85. The first update is (0.144444…, 0.286111…, 0.569444…), which sums to 1. Without C’s redistribution, the sum is only 0.716667…. One iteration is not convergence; all entries must use the previous vector.

无出链节点 C 的权重仍要分配出去。否则概率总量减少不是“模型收敛”,而是更新规则漏项。

Week 11 · Worked practice回看讲解与来源

Does λ always mean the same thing? / 不同公式中的 λ 是同一个参数吗?

Independent practice question / 独立编写的练习题

Does λ mean the same thing in the lecture’s PageRank formula and in MMR? What does the common value 0.85 usually describe?

λ 在讲义的 PageRank 公式和 MMR 中含义相同吗?常见的 0.85 通常描述什么?

Show answer / 显示答案 · Explanation / 解释与例子

No. Lecture p.17 uses λ for teleportation, with 1−λ as the link-following factor. MMR uses λ for relevance weight. The familiar PageRank value 0.85 commonly describes following links, not teleportation. Translate each symbol into its verbal definition before calculating.

先看符号定义,再代数值。相同字母不代表两个模型共享同一个超参数。

Week 11 · Worked practice回看讲解与来源

Calculate the second MMR result / 手算 MMR 的第二名

Independent practice question / 独立编写的练习题

A is already selected. B has relevance 0.6 and similarity 0.9 to A; C has relevance 0.5 and similarity 0.1. Which document does MMR pick second at λ = 0.5 and at λ = 0.9? At what λ do they tie?

A 已被选中。B 的相关性为 0.6、与 A 的相似度为 0.9;C 的相关性为 0.5、相似度为 0.1。λ = 0.5 和 λ = 0.9 时 MMR 第二个选谁?λ 为多少时打平?

Show answer / 显示答案 · Explanation / 解释与例子

A is already selected. B has relevance 0.6 and similarity to A of 0.9; C has relevance 0.5 and similarity 0.1. At λ=0.5, B scores −0.15 and C scores 0.20, so C is second. At λ=0.9, B scores 0.45 and C scores 0.44, so B is second. The exact tie occurs at λ=8/9.

C 相关性稍低,但在 λ=0.5 时补充的信息更多。权重变大后,相关性优势重新占上风。

Week 11 · Worked practice回看讲解与来源

Can MMR be computed once for every document? / MMR 能只打分一次然后排序吗?

Independent practice question / 独立编写的练习题

Can MMR scores be computed once for every document before selection starts? If a candidate has similarities 0.9 and 0.1 to two selected documents, which value enters the penalty?

MMR 分数能在开始选择之前对所有文档一次算好吗?如果候选文档与两个已选文档的相似度为 0.9 和 0.1,惩罚项用哪个值?

Show answer / 显示答案 · Explanation / 解释与例子

Not in general. After each selection, update the maximum similarity against the entire selected set. For similarities 0.9 and 0.1 to two selected documents, the penalty uses 0.9, not their mean 0.5. A near-duplicate can be redundant even when it differs from the other selected item.

已选集合改变,候选的边际价值也改变。MMR 是逐个选取,不是固定 pointwise 分数。

Week 11 · Worked practice回看讲解与来源

Does ordinary nDCG prove good intent coverage? / 普通 nDCG 高能证明覆盖更多意图吗?

Independent practice question / 独立编写的练习题

Does a high ordinary nDCG prove that a ranking covers different user intents well? Which metrics measure intent coverage?

普通 nDCG 高能证明排序很好地覆盖了不同的用户意图吗?哪些指标能衡量意图覆盖?

Show answer / 显示答案 · Explanation / 解释与例子

No. Duplicate results with equally high relevance labels can achieve high nDCG while covering one intent. Use subtopic judgments for α-nDCG or an intent-aware measure. Plain ERR models satisfaction-based stopping, whereas ERR-IA explicitly aggregates intent-conditioned utility. Distinct metric assumptions answer distinct questions.

多样性需要额外的子主题或意图信息。相关性、意图覆盖与答案 groundedness 不能用一个分数互相替代。

Project 2回看讲解与来源

Which data belongs to the main task and which to the bonus? / 主任务和 bonus 分别使用哪组数据?

Independent practice question / 独立编写的练习题

For Project 2, which data belongs to the main task and which to the bonus task? What should you verify and document?

Project 2 中,哪些数据属于主任务,哪些属于 bonus 任务?你需要核实和记录什么?

Show answer / 显示答案 · Explanation / 解释与例子

Staff clarified that the main task’s old “development set” wording means the sampled collection, queries and qrels. Train queries/qrels are for the bonus; a later reply says official dev files were added under the bonus folder. These concern different scopes. Verify the latest files and document actual model-selection data; the website has not audited the new files.

不要把主任务的术语修正和 bonus 新增 dev 文件当成同一件事;请按当前教师说明分别处理。

Week 12 · Self-authored practice / 自编练习回看讲义笔记

Which feedback signal should be trusted most? / 哪类反馈最可信?

Independent practice question / 独立编写的练习题

Rank these feedback signals by how much you would trust them: explicit relevance judgements, clicks and pseudo-relevance feedback. Why?

按可信程度给这三种反馈信号排序:显式相关性判断、点击、pseudo-relevance feedback。为什么?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

Direct user relevance judgements are normally more certain than inferred behaviour, while pseudo-relevance feedback merely assumes the initial top results are relevant.

用户明确判断通常比行为推断更确定;PRF 更只是把首轮 top results 当作相关。Click 可能来自排名、可见性或展示方式,不能自动等同 relevance。

Week 12 · Self-authored practice / 自编练习回看讲义笔记

Calculate a Rocchio update / 手算一次 Rocchio 更新

Independent practice question / 独立编写的练习题

With q₀ = (1, 0), relevant centroid (0.8, 0.6), non-relevant centroid (0.2, 0.9), α = 1, β = 0.75 and γ = 0.25, compute the Rocchio update q′. Is q′ an evaluation score?

q₀ = (1, 0),相关文档中心 (0.8, 0.6),不相关文档中心 (0.2, 0.9),α = 1,β = 0.75,γ = 0.25。计算 Rocchio 更新后的 q′。q′ 是评价分数吗?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

For q₀=(1,0), relevant centroid (0.8,0.6), non-relevant centroid (0.2,0.9), and α=1, β=0.75, γ=0.25, q′=(1.55,0.225).

计算是 (1,0)+0.75(0.8,0.6)−0.25(0.2,0.9)。Updated vector 是下一轮 query direction,不是 evaluation score。

Week 12 · Self-authored practice / 自编练习回看讲义笔记

Why can pseudo-relevance feedback drift? / PRF 为什么会漂移?

Independent practice question / 独立编写的练习题

The query “jaguar speed” initially retrieves mostly car pages, but the user meant the animal. What happens under pseudo-relevance feedback, and what bounds PRF quality?

查询 “jaguar speed” 最初主要检索到汽车页面,但用户想找的是动物。使用 pseudo-relevance feedback 会发生什么?PRF 的质量受什么限制?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

If “jaguar speed” initially retrieves car pages while the user meant the animal, adding engine and horsepower reinforces the wrong interpretation.

PRF 的上限由首轮排名决定,也受 expansion-term selection 和 weighting 影响。错误词义进入反馈循环,就会把 query 越拉越远。

Week 12 · Self-authored practice / 自编练习回看讲义笔记

Compare two RRF candidates / 比较两个 RRF 候选

Independent practice question / 独立编写的练习题

With k = 60, document A is ranked 1 and 4 in two runs, and B is ranked 2 and 1. Compute their RRF scores. Which ranks higher?

k = 60 时,文档 A 在两个 run 中排第 1 和第 4,B 排第 2 和第 1。计算它们的 RRF 分数。哪个排得更高?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

With k=60, A at ranks 1 and 4 scores 1/61+1/64≈0.03202. B at ranks 2 and 1 scores 1/62+1/61≈0.03252, so B ranks above A.

RRF 只用 positions,不用 raw retrieval scores。计算前要说明 rank 从 1 开始。

Week 12 · Self-authored practice / 自编练习回看讲义笔记

Why normalise scores before CombSUM? / CombSUM 前为什么归一化?

Independent practice question / 独立编写的练习题

Why normalise scores before CombSUM? Apply per-list min–max normalisation to [−8, −5, −2]. What is one weakness of min–max?

为什么在 CombSUM 之前要做分数归一化?对 [−8, −5, −2] 做每个列表的 min–max 归一化。min–max 有什么弱点?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

BM25, query-likelihood and neural scores may use unrelated ranges. For scores [−8,−5,−2], min–max gives [0,0.5,1].

这保证 shift invariance,但新的极端 maximum 会压缩其他差距。Normalisation 是 fusion design 的一部分,不是 cosmetic step。

Week 12 · Self-authored practice / 自编练习回看 practical 讲解

What must stay consistent in IDF-r reduction? / IDF-r 精简中哪些设置必须一致?

Independent practice question / 独立编写的练习题

When you shorten a query by keeping its highest-IDF terms (IDF-r reduction), what must stay consistent, and which edge cases must you document?

用保留最高 IDF 词的方式缩短查询(IDF-r reduction)时,哪些东西必须保持一致?需要说明哪些边界情况?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

Analyze the query with the same analyzer used for retrieval, then compute document frequency against the same index. Rank by the stated IDF convention and retain the requested count.

要明确重复词、并列 IDF、n 超过 term 数时的处理。留下的词更稀有,不代表一定保住完整 intent;仍须用相同 qids 与 qrels 评价。

Week 12 · Self-authored practice / 自编练习回看 practical 讲解

Avoid Borda's indexing trap / 避免 Borda 的下标错误

Independent practice question / 独立编写的练习题

For a three-document list, what Borda score does each rank receive under (n − r + 1)/n? Write the equivalent expression for Python’s zero-based enumerate, and explain the common mistake.

对一个三篇文档的列表,用 (n − r + 1)/n 时每个排名得多少 Borda 分?写出 Python 从 0 开始的 enumerate 对应的表达式,并解释常见的错误。

Show answer / 显示答案 · Self-authored explanation / 自编讲解

For a three-document list, one-based ranks 1, 2 and 3 receive 1, 2/3 and 1/3 from (n−r+1)/n. Python enumerate starts at zero, so use (n−i)/n.

若把 zero-based i 错代进 (n−i+1)/n,第一名会得到 4/3。先统一 rank 定义,再实现公式。

Week 12 · Self-authored practice / 自编练习回看 practical 讲解

Compare CombSUM and CombMNZ / 比较 CombSUM 与 CombMNZ

Independent practice question / 独立编写的练习题

After normalisation, d1 has scores 0.9 and 0.2, d2 has 0.4 once and d3 has 0.8 once. Compute CombSUM and CombMNZ. When does CombMNZ’s extra factor help, and when does it hurt?

归一化后,d1 的分数为 0.9 和 0.2,d2 只有一次 0.4,d3 只有一次 0.8。计算 CombSUM 和 CombMNZ。CombMNZ 的额外因子什么时候有帮助,什么时候有害?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

After per-query, per-run normalisation, let d1 score .9 and .2, d2 score .4 once, and d3 score .8 once. CombSUM gives 1.1, .4 and .8; CombMNZ gives 2.2, .4 and .8 because d1 has two positive supports.

CombMNZ 奖励多个系统共同支持,可能增强独立证据,也可能放大多个相似系统重复的错误。

Week 13 · Self-authored practice / 自编练习回看讲义笔记

Use block starts to find a document identifier / 用块首 identifier 查找文档

Independent practice question / 独立编写的练习题

A postings list stores gaps [3, 4, 1, 7, 15, 1, 14, 7] in blocks of four, and each block stores its first identifier directly. What is the 7th document identifier, and how many additions are needed?

一个 postings list 以每块 4 个存储差值 [3, 4, 1, 7, 15, 1, 14, 7],每块直接存首个 identifier。第 7 个文档 identifier 是多少?需要几次加法?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

Decoding the gaps gives identifiers 3, 7, 8, 15, 30, 31, 45 and 52. The second block starts at 30, so the 7th identifier is 30 + 1 + 14 = 45 with two additions, instead of six additions from the start of the list.

第二块从 30 开始,所以第 7 个是 30 + 1 + 14 = 45,只需两次加法;不分块则要从头累加六次。

Week 13 · Self-authored practice / 自编练习回看讲义笔记

Encode integers with Variable Byte / 用 Variable Byte 编码整数

Independent practice question / 独立编写的练习题

Using Variable Byte coding where the continuation bit is 1 on every byte except the last, encode 5, 130 and 300.

用 Variable Byte 编码(除最后一个字节外 continuation bit 都为 1),编码 5、130 和 300。

Show answer / 显示答案 · Self-authored explanation / 自编讲解

5 fits in seven bits: 00000101. 130 = 1·128 + 2: 10000001 00000010. 300 = 2·128 + 44: 10000010 00101100. Small integers, such as gaps, therefore need fewer bytes than a fixed 32-bit representation.

5 只需一个字节;130 与 300 需要两个字节。差值这类小整数比固定 32 位表示省空间。

Week 13 · Self-authored practice / 自编练习回看讲义笔记

Quantize a BM25 contribution / 量化一个 BM25 贡献

Independent practice question / 独立编写的练习题

Index-wide BM25 contributions range from 0 to 20. With b = 8 bits and the mapping 2^b·(s − s_min)/(s_max − s_min), what is the impact of a contribution of 7.5? What goes wrong at the maximum?

全索引的 BM25 贡献范围是 0 到 20。取 b = 8 位、映射 2^b·(s − s_min)/(s_max − s_min),贡献 7.5 的 impact 是多少?最大值会出什么问题?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

256 × 7.5 / 20 = 96. The maximum maps to 256, which needs nine bits, so implementations scale by 2^b − 1 = 255 or clamp the top value. Quantization also merges nearby scores, a small loss of precision accepted for compression and speed.

最大值映射到 256,需要 9 位,所以实际会乘 255 或截断;量化也会合并相近分数,用少量精度换取压缩与速度。

Week 13 · Self-authored practice / 自编练习回看讲义笔记

Decide which documents need scoring / 判断哪些文档需要打分

Independent practice question / 独立编写的练习题

Lists a, b and c have upper bounds 4, 7 and 3, and the current top-k threshold is 9. Which of the documents {a, c}, {a, b} and {b, c} must be scored? Which lists are non-essential for MaxScore?

列表 a、b、c 的上界分别为 4、7、3,当前 top-k 阈值为 9。文档 {a, c}、{a, b}、{b, c} 中哪些必须打分?对 MaxScore 来说哪些列表是 non-essential?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

Bounds: {a, c} = 7 < 9, skip; {a, b} = 11 and {b, c} = 10, both must be scored exactly. Sorting bounds gives c (3), a (4), b (7); c and a together reach only 7 < 9, so they are non-essential and only list b generates candidates.

{a, c} 上界 7 < 9 可跳过;另两个必须精确打分。c 和 a 合计只有 7,是 non-essential,只有 b 产生候选。

Week 13 · Self-authored practice / 自编练习回看讲义笔记

Explain why learned sparse retrieval is slower / 解释 learned sparse 为什么更慢

Independent practice question / 独立编写的练习题

A learned sparse model gives the token for a very common word a high weight in almost every document. Why does this slow down dynamic pruning compared with BM25?

某 learned sparse 模型给一个常见词的 token 在几乎所有文档中都赋了高权重。和 BM25 相比,为什么这会拖慢 dynamic pruning?

Show answer / 显示答案 · Self-authored explanation / 自编讲解

BM25 gives long lists low impacts, so their upper bounds are small and many documents can be skipped. A long list with high learned impacts has a high upper bound that rarely falls below the threshold, so more documents are scored and more blocks are decompressed. Expansion also makes queries longer, adding more lists to traverse.

长列表 + 高上界意味着很少能跳过,打分和解压都变多;expansion 让查询变长,也增加了要遍历的列表。