What’s new todayLast update
108 use cases added today Lake Merritt Guide 4: Replicating a Real Benchmark (BBQ fairness evals)
Transcript reviewed · recording and outcomes not independently reproduced
What the transcript explains
Dazza Greenwood adapts the BBQ fairness benchmark to Lake Merritt with a custom scorer and eval pack. The demonstrated run uses an age-related subset and compares predicted labels with expected labels. The workflow illustrates why reproducing a benchmark needs its metadata and scoring rules as well as a list of questions.
Limits and human checks
- The demonstrated subset cannot establish overall model fairness.
- The scorer uses benchmark label comparisons, not an LLM judge; do not compare its scores directly with a different judge setup.
Our notes are based on the caption transcript, reviewed 2026-10-15. Caption errors and differences from what appears on screen remain possible.
Evidence at a glance
Transcript reviewed- Source author
- Dazza Greenwood
- Original account
- Read the source ↗
- Published
- Source link checked
- Transcript reviewed
- . Recording not visually reviewed; results not independently reproduced.
Report an issue with this case
Open correction emailOpens your email app with the case link included. Nothing is sent automatically.
About this source
Caption transcript reviewed by Agents In Use on 2026-10-15; the account remains author-reported.
Agent not identified