How do you know
the number is real?
An evaluation is a measurement system. It deserves the same scrutiny as the product it measures.
My Arbol and Nullius work explores a deceptively simple question: does an improvement survive a test that was not designed around the answer?
Three habits worth keeping
- Hold something out. Keep data that the configuration search never sees.
- Count what you tried. The best of many experiments is different from one experiment chosen in advance.
- Inspect the questions. A retrieval system can look strong when its test questions borrow the wording of the documents.
Why this interests me
This connects data engineering, applied AI, and product decisions. A dashboard can be beautifully built and still answer the wrong question. The work is to make the evidence useful enough that someone can make a better decision.
Nullius is presented here as a research direction based on local design and evaluation notes. This page makes no benchmark-performance or customer-result claims.