Revision Q/A

Weeks 1–8, Weeks 11–13 and Project 2 clarifications · 67 study topics

Independently authored practice questions, explanations and worked examples based on course concepts, not official questions or an official answer key. Try each question before opening its answer. Original slide wording, slide images and lecture files are not published here.

Week 1Review topic

Information need, query and relevance

Independent practice question

Two people enter the same landmark name: one is planning a visit, the other is researching its history. Design a relevance-judging rule for each person. Why would keyword overlap alone give misleading labels?

Show answer · Explanation and worked example

A query is an imperfect expression of an information need. The same place name may express a tourism, history or navigation task. Decide which need is intended before judging results; keyword overlap alone does not determine relevance.

Week 1Review topic

Search architecture

Independent practice question

Design a search service that supports web pages, product results and conversational answers. Start with crawling, indexing and retrieval; identify the extra modules needed for these three experiences and explain their roles.

Show answer · Explanation and worked example

The crawl–index–retrieve pipeline remains useful, but a modern system can add vertical search, semantic retrieval, multi-stage neural ranking, interaction logging, personalisation and conversational interfaces. Sketch these as additional components rather than claiming the 1998 architecture is unchanged.

Week 1Review topic

Stemmers

Independent practice question

An indexing team wants to merge related word forms without merging unrelated meanings. Explain the transformation a stemmer performs, contrast Porter with Krovetz, and identify the precision–recall trade-off the team should test.

Show answer · Explanation and worked example

A stemmer maps word forms to a common stem to reduce morphological mismatch. Porter is rule-based suffix stripping. Krovetz combines linguistic rules with dictionary checks; it is not a statistical frequency-based stemmer. Conflation can increase recall but over-stemming can hurt precision.

Week 1Review topic

Zipf’s law

Independent practice question

A corpus has a few extremely frequent terms and a very long tail of rare ones. Which frequency–rank relationship explains this pattern? Use it to justify one indexing decision and one term-weighting decision.

Show answer · Explanation and worked example

When terms are ordered by decreasing frequency, frequency is approximately inversely proportional to rank: f(r) ≈ C/r. A few terms dominate occurrences; many terms form a long tail. This motivates careful treatment of frequent terms, compressed postings and rarity-sensitive weighting such as IDF.

Week 1Review topic

Domain-specific stoplists

Independent practice question

You are indexing a technical-support knowledge base. A proposed stoplist removes every frequent word, including words that distinguish failure types. Design a safer selection and evaluation process, and explain when retaining low-weight terms is preferable.

Show answer · Explanation and worked example

Inspect corpus document frequencies, existing stoplists and domain queries; propose candidates, then validate on held-out retrieval tasks. Apply compatible analysis at indexing and query time. Alternatives include retaining terms with low IDF weights or using efficient query processing. Do not remove domain-critical words solely because they are frequent.

Week 1Review topic

Text processing true/false

Independent practice question

A search engineer proposes case folding, stemming and adding word bigrams as one package. Predict which changes can merge meanings or enlarge the index, and design measurements rather than assuming indexing and retrieval will become faster.

Show answer · Explanation and worked example

Normalisation trades distinctions for matching opportunities. Case folding and stemming can improve recall but conflate different meanings. Adding n-grams can increase the index size. Stemming costs analysis time but can reduce the vocabulary; measure total indexing and retrieval time rather than assuming either becomes faster.

Week 1Review topic

Search in the LLM era

Independent practice question

A helpdesk chatbot answers from model memory but sometimes invents outdated instructions. Draw an evidence-first answering pipeline using IR components. Explain what retrieval improves and which correctness risks remain.

Show answer · Explanation and worked example

Users increasingly ask for synthesised answers. Risks include hallucinations, stale knowledge and unverifiable evidence. Retrieval-Augmented Generation (RAG) retrieves sources before generation: query → retrieval → reranking → evidence context → generated answer with citations. Retrieval helps grounding but does not guarantee factual correctness.

Week 2Review topic

Precision, AP and MAP

Independent practice question

A ranking has relevant documents at positions 1, 3 and 5, and the collection contains four relevant documents. Compute P@5, recall@5 and AP. How is MAP different from AP?

Show answer · Explanation and worked example

Worked example: a ranking has relevant documents at positions 1, 3 and 5, with four relevant documents in the collection. P@5=3/5=0.6; recall@5=3/4=0.75; AP=(1+2/3+3/5)/4≈0.5667. Unretrieved relevant documents remain in the AP denominator. MAP averages AP over queries, not competing systems for one query.

Week 2Review topic

Reciprocal rank

Independent practice question

For system S1, the first relevant result is at rank 1 for q1 and at rank 5 for q2. Compute RR for each query and the MRR. Do later relevant results change RR?

Show answer · Explanation and worked example

The first relevant result is at rank 1 for q1 and rank 5 for q2. RR values are 1 and 1/5; MRR=(1+0.2)/2=0.6. Later relevant results do not affect Reciprocal Rank.

Week 2Review topic

Normalised DCG

Independent practice question

With binary relevance [0,1,1] for the top three results and exactly two relevant documents in the collection, compute DCG@3, IDCG@3 and nDCG@3.

Show answer · Explanation and worked example

Worked example with binary relevance [0,1,1]: DCG@3=1/log2(3)+1/log2(4)≈1.13093. If the collection contains exactly two relevant documents, IDCG@3=1+1/log2(3)≈1.63093, so nDCG@3≈0.69343. Define the relevance-gain convention and use the same query judgments for both DCG and IDCG.

Week 2Review topic

Rank-Biased Precision

Independent practice question

Write the RBP formula and explain what the persistence parameter p means. For p = 0.8 and binary relevance [1,0,1], compute the observed RBP contribution and the bound on the unobserved tail.

Show answer · Explanation and worked example

Rank-Biased Precision is RBP=(1−p)Σ r_i p^(i−1). The user begins at rank 1 and continues with probability p, giving expected depth 1/(1−p). For p=0.8 and binary relevance [1,0,1], the observed contribution is 0.328 and the unobserved-tail bound is 0.8³=0.512.

Week 2Review topic

Working with qrels

Independent practice question

The qrels mark A and C as relevant and the run returns B, A, C. Compute precision@3, recall@3, RR and AP. Why are retrieval scores not relevance labels?

Show answer · Explanation and worked example

Join run entries to qrels by query ID and document ID, then evaluate in run order. Example: relevant IDs are A and C; the returned list is B,A,C. Precision@3=2/3, recall@3=1, RR=1/2 and AP=(1/2+2/3)/2=7/12. Retrieval scores determine order but are not relevance labels.

Week 2Review topic

Implement evaluation functions

Independent practice question

You are reviewing a small ranking-evaluation library before using its results in a report. Specify test cases for perfect rankings, no relevant hits, missing judgments and ties, then explain how to cross-check its metric outputs independently.

Show answer · Explanation and worked example

Implement precision, recall, AP, RR, nDCG and RBP with explicit cutoffs and relevance conventions. Validate perfect rankings, no relevant results, missing judgments and ties against a trusted evaluator. A visual demo is useful for intuition, not a substitute for tests.

Week 3Review topic

Lexical models: features and limitations

Independent practice question

A keyword search service returns an unranked set and later switches to sorting by raw term counts. Diagnose a weakness of each design. Explain how corpus rarity, frequency saturation and length normalisation improve the next version.

Show answer · Explanation and worked example

Term-based models exploit overlap, frequency and corpus statistics. Boolean retrieval supplies sets without graded ranking; raw TF weights all terms equally and can favour longer documents. IDF discounts common terms, while BM25 adds term-frequency saturation and length normalisation.

Week 3Review topic

Normalised term frequency

Independent practice question

A document contains the tokens retrieval, retrieval, model. Compute the normalised TF of each term. What is the denominator, and how does raw TF differ?

Show answer · Explanation and worked example

For a document containing the terms retrieval, retrieval, model, normalised TF(retrieval)=2/3 and TF(model)=1/3. The denominator is three token occurrences, not two unique terms. Raw TF would instead be 2 and 1.

Week 3Review topic

Inverse document frequency

Independent practice question

Using IDF(t) = ln(N / (1 + df(t))) with N = 12, df(retrieval) = 3 and df(model) = 5, compute both IDF values. Does repeating a term within one document change df?

Show answer · Explanation and worked example

Using the illustrative convention IDF(t)=ln(N/(1+df(t))), let N=12, df(retrieval)=3 and df(model)=5. IDF values are ln(3)≈1.0986 and ln(2)≈0.6931. Repeated occurrences within one document do not increase document frequency.

Week 3Review topic

TF-IDF term weights

Independent practice question

Continuing the previous two examples, compute the TF-IDF weights of retrieval and model in the document. Which part is local and which is global?

Show answer · Explanation and worked example

Continue the preceding examples: TF-IDF(retrieval)=(2/3)ln(3)≈0.7324 and TF-IDF(model)=(1/3)ln(2)≈0.2310. The local term count and global rarity play different roles; always state raw versus normalised TF and the log convention.

Week 3Review topic

TF-IDF query score

Independent practice question

Using the TF-IDF weights from the previous question, compute the score of the document for the query “retrieval” and for the query “retrieval model”. Is this cosine similarity?

Show answer · Explanation and worked example

For the query retrieval, only its document weight contributes, giving approximately 0.7324 in the preceding example. For retrieval model, sum both weights to obtain approximately 0.9635. This is a matching-term sum, not cosine normalisation.

Week 3Review topic

Repeated terms and BM25 components

Independent practice question

Two pages match the same query: one repeats its keywords, while the other is a much longer page containing many topics. Explain why raw TF can reward both unfairly and assign the appropriate BM25 mechanism to each problem.

Show answer · Explanation and worked example

Raw term counts can reward repetition without bound. BM25 limits marginal gains with TF saturation and accounts for document length. k1 controls saturation; b controls length normalisation. Some full BM25 formulations also include query frequency and relevance-feedback statistics; do not silently replace an assignment’s specified simplified formula.

Week 4Review topic

Vocabulary mismatch and self-attention

Independent practice question

For a search about repairing a laptop, a useful document uses “notebook” instead of “laptop”. Contrast the evidence available to lexical matching and contextual token representations, then explain the Q, K and V computation behind self-attention.

Show answer · Explanation and worked example

BM25 needs lexical overlap after analysis; “laptop” and “notebook” can mismatch. Self-attention forms contextual token representations with softmax(QKᵀ/√d_k)V, letting each token combine evidence from other permitted positions. Contextual representations enable semantic matching but do not guarantee it.

Week 4Review topic

Transformer components

Independent practice question

Inspect two attention masks: one permits every token to use both left and right context, while the other blocks future positions. Identify their usual roles and distinguish parallel training from token-by-token generation.

Show answer · Explanation and worked example

A standard Transformer encoder uses bidirectional self-attention. A causal decoder masks future positions. During training, known sequence positions can still be processed in parallel under the causal mask; autoregressive generation produces successive tokens sequentially.

Week 4Review topic

BERT versus GPT

Independent practice question

You need one backbone for bidirectional text encoding and another for next-token generation. Assign BERT and GPT to these roles using their attention structure, not model size or tokenisation as the deciding criterion.

Show answer · Explanation and worked example

BERT means Bidirectional Encoder Representations from Transformers and uses an encoder backbone. GPT means Generative Pre-trained Transformer and uses a decoder backbone. Vocabulary size, sequence length, embedding dimension and training-token count are different quantities; inspect the exact checkpoint before assigning numerical values.

Week 4Review topic

Input embedding matrix

Independent practice question

A token embedding is [1,0,2], its position embedding is [0,1,0] and its segment embedding is [1,0,1]. What is the input vector? For n tokens with hidden size h, what is the input shape?

Show answer · Explanation and worked example

Add token, position and segment embeddings elementwise. For example, [1,0,2]+[0,1,0]+[1,0,1]=[2,1,3]. Three vectors of width 3 produce one vector of width 3, not 9. A sequence with n tokens and hidden size h has input shape n×h before the batch dimension.

Week 4Review topic

Use encoders/decoders for ranking

Independent practice question

Your retrieval system must precompute document representations, but its second stage can afford a joint query–document pass. Propose encoder-based and decoder-based designs and label each output as a reusable embedding or a pair-specific relevance score.

Show answer · Explanation and worked example

An encoder can separately embed query/document (bi-encoder) or jointly classify a pair (cross-encoder). A decoder can produce an embedding from a context-aware final token, or judge relevance through generated labels/answer-token likelihoods. Decide whether the output is a reusable representation or a pair-specific score.

Week 5Review topic

Encoder text representations

Independent practice question

A team is choosing between one vector per text and one vector per token. Trace how DPR and ColBERT turn encoder outputs into retrieval scores, separating representation pooling from the final matching operation.

Show answer · Explanation and worked example

Use CLS pooling, masked mean pooling, or all-token representations. DPR uses pooled query/document embeddings with dot-product scoring; ColBERT keeps token vectors and sums each query token’s maximum document-token similarity. Pooling chooses representation; dot product/cosine/MaxSim chooses matching.

Week 5Review topic

Decoder similarities

Independent practice question

A decoder model is offered in two search designs: separately encode texts for an ANN index, or jointly prompt it with a query and each candidate. Explain what each design produces, what can be cached, and which belongs to reranking.

Show answer · Explanation and worked example

A causal decoder’s final/EOS representation can summarise preceding text. Encode query and document separately and compare their embeddings with the model’s trained similarity. Some methods modify attention or add contrastive training. Alternatively joint prompting yields a relevance score, which is reranking rather than reusable dense retrieval.

Week 5Review topic

InfoNCE calculation

Independent practice question

Given sim(q, p+) = 0.9 and negatives 0.3, 0.1 and 0.4, compute the InfoNCE loss with temperature τ = 1. What happens if τ changes?

Show answer · Explanation and worked example

Assuming temperature τ=1, L=−log(exp(0.9)/(exp(0.9)+exp(0.3)+exp(0.1)+exp(0.4)))≈0.9573065. A different temperature changes the answer. InfoNCE is a contrastive Noise-Contrastive Estimation objective: it rewards the positive relative to the negatives.

Week 5Review topic

Margin loss calculation

Independent practice question

With margin m = 1, positive score 1.5 and negative scores 0.3, 0.8 and 1.2, compute each hinge loss, their sum and their mean.

Show answer · Explanation and worked example

For margin m=1, positive score 1.5 and negative scores 0.3, 0.8 and 1.2, the hinge losses max(0,m−s+ +s−) are 0, 0.3 and 0.7. Their sum is 1.0 and their mean is 1/3. Specify the aggregation convention; pairs already exceeding the margin contribute zero.

Week 5Review topic

Harder negatives

Independent practice question

A dense retriever already separates its BM25-mined negatives but still confuses semantically close passages. Design a refreshed ANN-based negative-mining loop and explain how false negatives can undermine the training signal.

Show answer · Explanation and worked example

Mine high-scoring non-relevant neighbours with the current dense retriever using Approximate Nearest Neighbour (ANN) search; periodically refresh document embeddings/indexes as the model changes. ANCE uses this idea. Filter known positives and inspect likely false negatives; harder is useful only when the labels are trustworthy.

Week 6Review topic

Why TILDE is efficient

Independent practice question

A search endpoint has a strict online latency budget but can do substantial work when documents are indexed. Explain how TILDE moves neural computation offline and how TILDEv2 changes the storage trade-off.

Show answer · Explanation and worked example

For TILDE query-likelihood ranking, document-side vocabulary scores are precomputed; queries require tokenisation and lookup rather than a neural query encoder. This shifts work offline. Original TILDE has a large vocabulary-sized document representation; TILDEv2 reduces storage by weighting document/expanded tokens. Do not claim all TILDE variants have no online encoding.

Week 6Review topic

SPLADE term weights

Independent practice question

A learned sparse index assigns weight to a useful query term that never appeared literally in a document. Trace a SPLADE-style path from contextual token states to vocabulary weights, including the transformation, pooling and sparsity control.

Show answer · Explanation and worked example

SPLADE (Sparse Lexical and Expansion Model) projects contextual token states through a Masked Language Model (MLM) head into vocabulary dimensions, applies a non-negative log-saturation transform and pools across input tokens. Sparsity regularisation controls active terms. Vocabulary terms absent from the original input may receive weight.

Week 6Review topic

PromptReps hybrid retrieval

Independent practice question

You can extract both hidden states and vocabulary logits from a prompted model. Map them to a PromptReps-style dense branch and sparse branch, and identify the index and matching operation needed by each.

Show answer · Explanation and worked example

PromptReps uses the last hidden state for a dense representation and vocabulary logits for a sparse representation. Dense ANN retrieval and sparse inverted-index matching supply complementary signals, combined into a hybrid ranking. A neural backbone alone does not make a representation dense.

Week 6Review topic

SPLADEv1 sum pooling

Independent practice question

Two tokens have already-transformed activation rows [0.6, 0, 0.2] and [0.2, 0.3, 0.4]. Compute the SPLADEv1 sum-pooled weights. Are they a probability distribution?

Show answer · Explanation and worked example

For already transformed term-activation rows [0.6,0,0.2] and [0.2,0.3,0.4], sum pooling gives [0.8,0.3,0.6]. If starting from raw MLM logits, first apply the model’s non-negative log-saturation transform. The pooled weights are not a probability distribution.

Week 6Review topic

SPLADEv2 max pooling

Independent practice question

Using the same rows [0.6, 0, 0.2] and [0.2, 0.3, 0.4], compute the SPLADEv2 max-pooled weights and explain how max pooling differs from sum pooling.

Show answer · Explanation and worked example

Using the same activation rows, max pooling gives [0.6,0.3,0.4]. It keeps the strongest evidence for each vocabulary term rather than accumulating all occurrences. This is elementwise pooling over vocabulary dimensions, not selecting one whole token row.

Week 7Review topic

Offline Learning to Rank

Independent practice question

Three training pipelines consume a single labelled document, a preferred document pair, and a ranked list respectively. Classify their LTR learning units and assess the claim that a pairwise objective cannot account for nDCG.

Show answer · Explanation and worked example

Pointwise learning predicts a document label or score; pairwise learning compares two documents; listwise learning uses a ranked list as the training unit. These are training-target categories, not architectural requirements. Metric-aware pairwise objectives also exist, so avoid claiming that only listwise methods can account for a ranking metric.

Week 7Review topic

Cascade ranking

Independent practice question

A search service proposes running its most expensive ranker over the full collection. Redesign it as a candidate-generation and reranking cascade; explain the latency benefit and the failure that even a perfect final ranker cannot repair.

Show answer · Explanation and worked example

Expensive feature extraction and rich interactions are affordable only on a small candidate set. Early stages may use BM25, dense ANN or learned sparse retrieval; later LTR/rerankers refine the order. Candidate recall is a ceiling: a reranker cannot recover a relevant document absent from its candidates.

Week 7Review topic

Worked interleaving preference

Independent practice question

Ranker A = [a, b, c], ranker B = [b, c, a]. In the interleaved display the user clicks b and c. Using balanced interleaving, which ranker is preferred?

Show answer · Explanation and worked example

Take the deepest displayed clicked document, dmax. Let imin be its smallest rank in the two original lists. Count clicked IDs in each original prefix through imin; the larger count wins and equal counts tie. Example: A=[a,b,c], B=[b,c,a], clicked in displayed order=[b,c]. Here dmax=c and imin=2: A receives one click and B two.

Week 7Review topic

IPS estimator

Independent practice question

Two equally relevant results receive different click rates because one is displayed near the top. Explain how IPS uses examination propensities to compensate, and state why tiny or incorrect propensities can make the estimate unreliable.

Show answer · Explanation and worked example

Inverse Propensity Scoring corrects exposure or examination bias through inverse-probability weighting. Unbiasedness requires correct propensities, positive support and the assumed feedback model. Small propensities can create high variance; position correction alone does not remove every click bias.

Week 7Review topic

DBGD with balanced interleaving

Independent practice question

A live ranker has parameter vector θ and proposes a nearby challenger θ′ = θ + δu. Design one DBGD comparison-and-update cycle with interleaving; explain when θ changes and why exploration distance and learning rate are separate settings.

Show answer · Explanation and worked example

Dueling Bandit Gradient Descent: sample a unit direction u; form candidate θ′=θ+δu; rank with current and candidate models; interleave lists; collect clicks and infer preference; if the candidate wins update θ←θ+αu, otherwise retain θ. δ is exploration distance and α is learning rate. Repeat over interactions.

Week 7Review topic

Counterfactual OLTR

Independent practice question

You have interaction logs from a known logging policy and want to compare new rankers without a new interleaved display for every comparison. Explain the counterfactual approach and why logged clicks alone are not sufficient evidence.

Show answer · Explanation and worked example

Counterfactual Online Learning to Rank reuses propensity-logged interactions to compare candidate rankers without a new interleaved display for each comparison. It is off-policy because the evaluated ranker differs from the data-collection policy. It still requires feedback, reliable probabilities and sufficient exploration.

Week 8Review topic

monoBERT versus dense retrieval

Independent practice question

A first-stage bi-encoder retrieves candidates quickly, but a joint query–passage model resolves subtle term interactions better. Explain the representational advantage of monoBERT and the per-candidate computation that prevents using it like an ANN retriever.

Show answer · Explanation and worked example

monoBERT jointly encodes query and passage, enabling token-level cross-attention before scoring. A single-vector bi-encoder compresses each side independently. Rich joint interaction can improve reranking, but costs a forward pass per pair and cannot precompute a query-independent document score.

Week 8Review topic

monoBERT limitations

Independent practice question

A team feeds long documents into monoBERT and interprets its softmax outputs as calibrated relevance probabilities. Identify the token-budget, runtime, candidate-recall and calibration issues they must check before trusting the rankings.

Show answer · Explanation and worked example

The standard BERT input limit is 512 tokens including special tokens and query tokens, not 512 words per document. Long passages must be truncated/chunked. Joint encoding is expensive; performance depends on training-domain match and candidate recall. Softmax output is not automatically a calibrated relevance probability.

Week 8Review topic

Handle long documents: MaxP and PARADE

Independent practice question

In a long report, the useful evidence might occur in the first passage, one later passage, or several passages together. Compare FirstP, MaxP and SumP, then explain how PARADE changes the point at which passage evidence is combined.

Show answer · Explanation and worked example

Split a document into passages. FirstP uses the first passage; MaxP takes the highest passage score; SumP adds scores but can favour many passages. PARADE (Passage Representation Aggregation for Document Reranking) aggregates query-conditioned passage representations using average, max, attention or Transformers, then predicts a document score.

Week 8Review topic

Four decoder reranking methods

Independent practice question

You must rerank a candidate set with an LLM under a limited inference budget. Compare scoring one document, a pair, a complete list and a small set at a time; explain the comparison-count and output-generation trade-offs.

Show answer · Explanation and worked example

Pointwise scores one query–document pair. Pairwise compares two documents and needs a sorting/aggregation strategy. Listwise requests an order for a list, subject to context limits and order bias. Setwise chooses the best among a small set, useful in tournament/heap-style ranking. Setwise can reduce comparisons versus pairwise and avoid generating an entire ordered list; exact costs depend on the algorithm.

Week 11 module supplementReview topic

Link authority and anchor text

Independent practice question

How can hyperlinks and anchor text each help ranking? What risk do they introduce?

Show answer · Explanation and worked example

A hyperlink can act as an authority signal, while its anchor text can describe the target with words absent from that page. Incoming anchor text may be indexed for the target and weighted by source authority. Both signals can help ranking, clustering, classification and crawling, but link farms and manipulated anchors can create spam.

Week 11 module supplementReview topic

PageRank calculation logic

Independent practice question

Explain how a page receives PageRank from its in-links in one update. Does the page with the most in-links always win?

Show answer · Explanation and worked example

A page receives PageRank mass from each in-linking page in proportion to that source's score divided by its out-degree, plus teleport mass. Repeat the update until convergence. A page with more in-links does not necessarily win: endorsement quality, source out-degree, damping and dangling-page handling all matter.

Week 11 module supplementReview topic

PageRank versus relevance

Independent practice question

How is PageRank different from a query-dependent relevance score such as BM25? Can a high-PageRank page be irrelevant?

Show answer · Explanation and worked example

PageRank is query-independent authority from link structure. BM25, dense retrieval or another content model scores the current query. A popular page can have high PageRank yet be irrelevant to a particular need, so practical ranking combines signals rather than replacing query matching with PageRank.

Week 11 module supplementReview topic

PRP assumptions and MMR diversification

Independent practice question

What does the Probability Ranking Principle assume? When do those assumptions fail, and how does MMR respond?

Show answer · Explanation and worked example

The Probability Ranking Principle orders documents by estimated relevance probability but assumes accurate estimates and relevance independent of already ranked results. Ambiguity and redundancy weaken those assumptions. MMR instead balances query relevance against similarity to selected documents, trading relevance for useful novelty and intent coverage.

Week 11 · Worked practiceReview topic and source

Why must PageRank preserve probability mass?

Independent practice question

Links: A → B, C; B → C; C has no out-links. Start from a uniform vector with d = 0.85 and compute one PageRank update. Why must C’s mass be redistributed?

Show answer · Explanation and worked example

For A → B,C; B → C; C dangling, start uniformly and set d=0.85. The first update is (0.144444…, 0.286111…, 0.569444…), which sums to 1. Without C’s redistribution, the sum is only 0.716667…. One iteration is not convergence; all entries must use the previous vector.

Week 11 · Worked practiceReview topic and source

Does λ always mean the same thing?

Independent practice question

Does λ mean the same thing in the lecture’s PageRank formula and in MMR? What does the common value 0.85 usually describe?

Show answer · Explanation and worked example

No. Lecture p.17 uses λ for teleportation, with 1−λ as the link-following factor. MMR uses λ for relevance weight. The familiar PageRank value 0.85 commonly describes following links, not teleportation. Translate each symbol into its verbal definition before calculating.

Week 11 · Worked practiceReview topic and source

Calculate the second MMR result

Independent practice question

A is already selected. B has relevance 0.6 and similarity 0.9 to A; C has relevance 0.5 and similarity 0.1. Which document does MMR pick second at λ = 0.5 and at λ = 0.9? At what λ do they tie?

Show answer · Explanation and worked example

A is already selected. B has relevance 0.6 and similarity to A of 0.9; C has relevance 0.5 and similarity 0.1. At λ=0.5, B scores −0.15 and C scores 0.20, so C is second. At λ=0.9, B scores 0.45 and C scores 0.44, so B is second. The exact tie occurs at λ=8/9.

Week 11 · Worked practiceReview topic and source

Can MMR be computed once for every document?

Independent practice question

Can MMR scores be computed once for every document before selection starts? If a candidate has similarities 0.9 and 0.1 to two selected documents, which value enters the penalty?

Show answer · Explanation and worked example

Not in general. After each selection, update the maximum similarity against the entire selected set. For similarities 0.9 and 0.1 to two selected documents, the penalty uses 0.9, not their mean 0.5. A near-duplicate can be redundant even when it differs from the other selected item.

Week 11 · Worked practiceReview topic and source

Does ordinary nDCG prove good intent coverage?

Independent practice question

Does a high ordinary nDCG prove that a ranking covers different user intents well? Which metrics measure intent coverage?

Show answer · Explanation and worked example

No. Duplicate results with equally high relevance labels can achieve high nDCG while covering one intent. Use subtopic judgments for α-nDCG or an intent-aware measure. Plain ERR models satisfaction-based stopping, whereas ERR-IA explicitly aggregates intent-conditioned utility. Distinct metric assumptions answer distinct questions.

Project 2Review topic and source

Which data belongs to the main task and which to the bonus?

Independent practice question

For Project 2, which data belongs to the main task and which to the bonus task? What should you verify and document?

Show answer · Explanation and worked example

Staff clarified that the main task’s old “development set” wording means the sampled collection, queries and qrels. Train queries/qrels are for the bonus; a later reply says official dev files were added under the bonus folder. These concern different scopes. Verify the latest files and document actual model-selection data; the website has not audited the new files.

Week 12 · Self-authored practiceReview lecture notes

Which feedback signal should be trusted most?

Independent practice question

Rank these feedback signals by how much you would trust them: explicit relevance judgements, clicks and pseudo-relevance feedback. Why?

Show answer · Self-authored explanation and worked example

Direct user relevance judgements are normally more certain than inferred behaviour, while pseudo-relevance feedback merely assumes the initial top results are relevant. A click can reflect rank, visibility or presentation rather than relevance. Match the confidence of the evidence to how aggressively the query is changed.

Week 12 · Self-authored practiceReview lecture notes

Calculate a Rocchio update

Independent practice question

With q₀ = (1, 0), relevant centroid (0.8, 0.6), non-relevant centroid (0.2, 0.9), α = 1, β = 0.75 and γ = 0.25, compute the Rocchio update q′. Is q′ an evaluation score?

Show answer · Self-authored explanation and worked example

For q₀=(1,0), relevant centroid (0.8,0.6), non-relevant centroid (0.2,0.9), and α=1, β=0.75, γ=0.25, q′=(1,0)+0.75(0.8,0.6)−0.25(0.2,0.9)=(1.55,0.225). The updated vector defines a new query direction; it is not itself an evaluation score.

Week 12 · Self-authored practiceReview lecture notes

Why can pseudo-relevance feedback drift?

Independent practice question

The query “jaguar speed” initially retrieves mostly car pages, but the user meant the animal. What happens under pseudo-relevance feedback, and what bounds PRF quality?

Show answer · Self-authored explanation and worked example

If the query “jaguar speed” initially retrieves mostly car pages while the user meant the animal, expansion terms such as engine and horsepower reinforce the wrong interpretation. PRF quality is bounded by the initial ranking and by how expansion terms are selected and weighted.

Week 12 · Self-authored practiceReview lecture notes

Compare two RRF candidates

Independent practice question

With k = 60, document A is ranked 1 and 4 in two runs, and B is ranked 2 and 1. Compute their RRF scores. Which ranks higher?

Show answer · Self-authored explanation and worked example

With k=60, A at ranks 1 and 4 scores 1/61+1/64≈0.03202. B at ranks 2 and 1 scores 1/62+1/61≈0.03252, so B ranks above A. RRF uses positions, not raw retrieval scores; specify that ranks start at 1.

Week 12 · Self-authored practiceReview lecture notes

Why normalise scores before CombSUM?

Independent practice question

Why normalise scores before CombSUM? Apply per-list min–max normalisation to [−8, −5, −2]. What is one weakness of min–max?

Show answer · Self-authored explanation and worked example

BM25, query-likelihood and neural scores may use unrelated ranges. For scores [−8,−5,−2], per-list min–max gives [0,0.5,1]. This is shift invariant, but an extreme new maximum would compress the other gaps. Normalisation is part of the fusion design, not a cosmetic step.

Week 12 · Self-authored practiceReview practical walkthrough

What must stay consistent in IDF-r reduction?

Independent practice question

When you shorten a query by keeping its highest-IDF terms (IDF-r reduction), what must stay consistent, and which edge cases must you document?

Show answer · Self-authored explanation and worked example

Analyze the query with the same analyzer used for retrieval, then compute every term's document frequency against the same index. Rank terms by the stated IDF convention and retain the requested count. Document how duplicate terms, ties, and n larger than the analyzed term count behave; a shorter query can lose essential intent even when its remaining terms are rarer.

Week 12 · Self-authored practiceReview practical walkthrough

Avoid Borda's indexing trap

Independent practice question

For a three-document list, what Borda score does each rank receive under (n − r + 1)/n? Write the equivalent expression for Python’s zero-based enumerate, and explain the common mistake.

Show answer · Self-authored explanation and worked example

For a three-document list, one-based ranks 1, 2 and 3 receive 1, 2/3 and 1/3 from (n−r+1)/n. Python enumerate starts at zero, so the equivalent expression is (n−i)/n. Using (n−i+1)/n with zero-based i incorrectly gives the first document 4/3.

Week 12 · Self-authored practiceReview practical walkthrough

Compare CombSUM and CombMNZ

Independent practice question

After normalisation, d1 has scores 0.9 and 0.2, d2 has 0.4 once and d3 has 0.8 once. Compute CombSUM and CombMNZ. When does CombMNZ’s extra factor help, and when does it hurt?

Show answer · Self-authored explanation and worked example

After per-query, per-run normalisation, suppose d1 has scores .9 and .2, d2 has .4 once, and d3 has .8 once. CombSUM gives 1.1, .4 and .8. CombMNZ multiplies by the number of positive-score runs, so the scores become 2.2, .4 and .8. The extra factor rewards agreement, which can help when systems provide independent evidence but hurt when several systems repeat the same error.

Week 13 · Self-authored practiceReview lecture notes

Use block starts to find a document identifier

Independent practice question

A postings list stores gaps [3, 4, 1, 7, 15, 1, 14, 7] in blocks of four, and each block stores its first identifier directly. What is the 7th document identifier, and how many additions are needed?

Show answer · Self-authored explanation and worked example

Decoding the gaps gives identifiers 3, 7, 8, 15, 30, 31, 45 and 52. The second block starts at 30, so the 7th identifier is 30 + 1 + 14 = 45 with two additions, instead of six additions from the start of the list.

Week 13 · Self-authored practiceReview lecture notes

Encode integers with Variable Byte

Independent practice question

Using Variable Byte coding where the continuation bit is 1 on every byte except the last, encode 5, 130 and 300.

Show answer · Self-authored explanation and worked example

5 fits in seven bits: 00000101. 130 = 1·128 + 2: 10000001 00000010. 300 = 2·128 + 44: 10000010 00101100. Small integers, such as gaps, therefore need fewer bytes than a fixed 32-bit representation.

Week 13 · Self-authored practiceReview lecture notes

Quantize a BM25 contribution

Independent practice question

Index-wide BM25 contributions range from 0 to 20. With b = 8 bits and the mapping 2^b·(s − s_min)/(s_max − s_min), what is the impact of a contribution of 7.5? What goes wrong at the maximum?

Show answer · Self-authored explanation and worked example

256 × 7.5 / 20 = 96. The maximum maps to 256, which needs nine bits, so implementations scale by 2^b − 1 = 255 or clamp the top value. Quantization also merges nearby scores, a small loss of precision accepted for compression and speed.

Week 13 · Self-authored practiceReview lecture notes

Decide which documents need scoring

Independent practice question

Lists a, b and c have upper bounds 4, 7 and 3, and the current top-k threshold is 9. Which of the documents {a, c}, {a, b} and {b, c} must be scored? Which lists are non-essential for MaxScore?

Show answer · Self-authored explanation and worked example

Bounds: {a, c} = 7 < 9, skip; {a, b} = 11 and {b, c} = 10, both must be scored exactly. Sorting bounds gives c (3), a (4), b (7); c and a together reach only 7 < 9, so they are non-essential and only list b generates candidates.

Week 13 · Self-authored practiceReview lecture notes

Explain why learned sparse retrieval is slower

Independent practice question

A learned sparse model gives the token for a very common word a high weight in almost every document. Why does this slow down dynamic pruning compared with BM25?

Show answer · Self-authored explanation and worked example

BM25 gives long lists low impacts, so their upper bounds are small and many documents can be skipped. A long list with high learned impacts has a high upper bound that rarely falls below the threshold, so more documents are scored and more blocks are decompressed. Expansion also makes queries longer, adding more lists to traverse.