Study guidance only. Verify current requirements in the official notebook and rubric. Personal submissions, output tables and reports are not included.
Assignment 1
End-to-end A1 workflow
A1 is not just a BM25 function. It is a reproducible IR experiment with validation, tuning, comparison, and analysis.
Why separate train and test?
Train qrels select k1 and b. Test qrels are used once for unbiased final evaluation. Repeatedly changing parameters after inspecting test results leaks test information into training.
How to discuss tuned BM25
Do not write only “the score improved.” Explain how k1 changes tf saturation and b changes length normalisation, then use gain/loss queries to diagnose the effect.