Bayesian tools for design and user research.
Say what a small study can and cannot support: how many problems remain unseen, whether raters agree, and what to ask next.
User research runs on small samples on purpose: five interviews, two coders, one round of testing. The useful question is not whether the sample is large, but what it can support. These tools estimate how many problems a study probably missed, check whether coders agree beyond chance, and rank the next question by how well it separates your explanations.
What do you need to find out?
We found these problems with five users. How many are we missing?
An estimate of the problems no participant hit, from how often the found ones repeat. Capture–Recapture Estimator · Are five users enough for a usability test?
Do our reviewers agree beyond chance?
Raw and chance-corrected agreement per label, from your coded records. Inter-Rater Reliability Audit · How many experts do you need for an expert review?
Which question would best tell our explanations apart?
Candidate questions ranked by how well each separates your hypotheses. Collection Planner
How do we turn several people's guesses into one prior?
One prior pooled from each person's range, with the disagreement kept visible. Expert Prior Builder · How do you turn expert opinions into a prior?
How sure are we of a rate, given what the team believes beforehand?
A posterior for the rate that combines the team's prior with the observed count. Beta-Binomial Calculator
Worked examples (synthetic data)
- Three people's guesses at trial conversion
- Four early ratings of a prototype
- A new feature that may go unused
- Onboarding, before and after a change
Instruments
- Capture–Recapture Estimator
- Inter-Rater Reliability Audit
- Collection Planner
- Beta-Binomial Calculator
- Expert Prior Builder
- DAG Sketchpad
- Sample Size Checker
- Usage Count Estimator
- Rating Estimator
- Competing Hypotheses
- Learning Value
- Learning Backlog
- Rollout Readout
Articles
- How do you turn expert opinions into a prior?
- Are five users enough for a usability test?
- How many experts do you need for an expert review?
- How many ratings before an average means anything?