A Practical Workflow for Experimentation Teams

The Hypothesis Bank Playbook

How to generate, score, and compound experiment hypotheses with Claude + Airtable — so your team always has 30+ high-quality ideas ready to test.

Step-by-step workflow Claude + Airtable ICE scoring Proactive + Reactive generation

30+

hypotheses generated in the first two weeks of a new programme

12%

average A/B test win rate — more shots is the only reliable route to more wins

4–5×

experiment ideas generated per hypothesis in Airtable once the system is running

THE PROBLEM

Most experimentation programmes stall — not because teams can't run tests, but because they run out of good ideas.

Generating and prioritising hypotheses manually is slow, subjective, and doesn't scale. The three bottlenecks that kill programmes before they compound:

BOTTLENECK 01

Hypothesis generation is extensive and slow

Manual research across session replays, analytics, UX audits, and customer reviews takes weeks. Most teams generate 3–5 ideas and run out of steam.

BOTTLENECK 02

Prioritisation is subjective

ICE scoring done in a room ends up biased toward whoever speaks loudest. There's no objective check on whether the score reflects real business impact.

BOTTLENECK 03

Results don't feed back into new ideas

When a test finishes, learnings stay in a slide deck and never become the next hypothesis. Each sprint starts from scratch instead of building on the last.

Step One

Build your Hypothesis Bank in Airtable

Your Airtable is the source of truth for everything. Claude knows which fields to populate — but you need to set up the structure first. The system uses four linked databases.

📋

Hypothesis Bank

The core table — every idea lives here, whether submitted by Claude or your team

Hypothesis Title Hypothesis Statement Evidence Link Page / Funnel Step Primary Metric Directional Lift Source Device Target Status Submitted By

📊

Scoring DB

ICE scores — human and Claude separately

Human Impact Score Human Confidence Human Ease Claude Impact Score Claude Confidence Claude Ease Score Difference Priority Decision

🔬

Results DB

Experiment outcomes — feeds back into new hypotheses

Experiment Name Primary Metric Lift Decision (Ship/Kill/Iterate) Key Learning Revenue Impact Segment Findings Child Hypotheses Generated

📝

Meeting Notes DB

Captures nuances from weekly syncs — context that doesn't live in your analytics platform

Meeting Date Linked Hypothesis Nuance / Context Captured Stakeholder Feedback Priority Change Rationale Action Owner

Why the Meeting Notes DB matters: Stakeholder decisions, past test context, and business constraints live only in conversations — not in Amplitude or Notion. When your note-taker captures these and feeds them into the database, Claude has the full picture when it scores and generates hypotheses. This is how the system avoids blind spots a pure-data approach would miss.

Step Two

Generate hypotheses with Claude

Two modes of generation. Both feed directly into the Hypothesis Bank in Airtable. Together they create a self-sustaining supply of ideas.

Proactive Arm

Starting fresh — or opening a new programme

Claude pulls from multiple data sources simultaneously to generate hypotheses. At programme start, this typically produces 15–50 hypotheses in the first two weeks.

Amplitude funnels and drop-off points

Session replays and rage click data

UI/UX audit findings

Customer reviews and feedback sources

Business KPIs and goal documents

Reactive Arm

Triggered automatically when an experiment finishes

When a result lands in the Results DB, Claude reads the outcome — what won, why, what the metrics showed — and generates new child hypotheses automatically. This is the compounding mechanism.

Primary and secondary metric results

Segment-level findings (US vs UK, mobile vs desktop)

Customer feedback during test period

Parent/child hypothesis relationship mapping

Standard Hypothesis Format — What Claude populates

"We believe that [change] for [audience] will cause [primary metric] to [increase/decrease] because [evidence source + observation]."

Real example from a client: "We believe that adding security trust badges next to the pay button for all checkout users will cause payment success rate to increase because session replays show hesitation at the payment form, and customer reviews mention security concerns as a reason for drop-off."

Step Three

Score and prioritise with ICE

The ICE scoring workflow removes subjectivity — but only if you do it in the right order. The sequence matters.

Before Claude scores anything, your team scores the hypothesis independently — based on your own business knowledge, development capacity, and traffic constraints. This happens first, before Claude touches the data.

Why? Because if Claude scores first, teams unconsciously anchor to its numbers. You want two independent scores so you can see where the gap is — and the gap is the conversation that matters.

Once both scores are in, the difference column highlights exactly where human judgement and data-driven scoring disagree. That disagreement is a signal, not a problem to resolve.

Impact

How much will this move the metric if it works?

Confidence

How strong is the evidence backing this idea?

Ease

How much dev effort and time does this require?

The Scoring Sequence

You

Human scores

Impact, Confidence, Ease based on business knowledge

→

Claude scores

Same three dimensions using its own rubric independently

→

△

Compare the diff

Score gap per dimension surfaces the real prioritisation debate

Real example: A client scored a trust badge hypothesis low on Confidence because they'd previously tested Trustpilot reviews and it hadn't moved. That context wasn't in any database — it came up in a weekly sync. The meeting note taker captured it, and from then on Claude knew to weight that type of evidence lower for this client. The system learns.

THE COMPOUNDING EFFECT

How it all connects into one loop

Each phase feeds the next. Once the system is running, hypothesis generation becomes automatic — not a bottleneck before every sprint.

Phase 01 · Planning

Generate & Score

Claude pulls from Amplitude, session replays, UX audits

Hypothesis Bank populated in Airtable

Human scores first, Claude scores second

Weekly sync aligns on top 3–5 to run

Phase 02 · Evaluation

Run & Evaluate

Experiment runs in Amplitude or Statsig

Results auto-populated into Results DB

Claude drafts Ship / Kill / Iterate decision

Revenue impact and key learning documented

Phase 03 · Documentation

Learn & Compound

Learnings stored in Results DB

Reactive arm triggers — generates new child hypotheses

Parent/child tree grows with each completed test

Next sprint starts with 30+ ideas already ranked

↻

Every result makes the next sprint smarter.

Teams using this system typically hold 30–35 active hypotheses across all clients, with 4–5 experiment ideas per hypothesis in the backlog. The bottleneck shifts from "we don't have enough ideas" to "we don't have enough traffic to run them all." That's a good problem to have.

BEFORE YOU START

What you need to make this work

This system is powerful once it's running — but it needs clean foundations. These are the prerequisites Zain checks before setting up the workflow with a new client.

Clean analytics with validated events

Claude pulls from your analytics platform to generate hypotheses. If events are misfiring or your funnel tracking is incomplete, the hypotheses will be built on bad signals. Fix the foundation first.

Enough traffic to run meaningful tests

As a rough guide, you need 20,000+ monthly sessions on the page you're testing to reach statistical significance in a reasonable runtime. Below that, tests take too long and teams lose patience.

Agreed primary metrics before generating

Claude needs to know what success looks like for your business before it can score hypotheses correctly. Define your primary KPIs and guardrail metrics upfront — this goes into the skill as context.

A weekly cadence to review and prioritise

The system handles generation automatically — but a human still makes the final prioritisation call. A 30-minute weekly sync to review the scoring differences and confirm the sprint backlog is all it takes.

TAKE IT FURTHER

This is one workflow. The full system has five.

The Hypothesis Bank is Workflow 2. The complete Adasight experimentation system layers in AI-powered prioritisation, result evaluation, meeting sync capture, and variant design — turning experimentation into a compounding growth engine.

Watch the Webinar