Run 50 customer interviews and synthesize themes
A 12-step playbook. Open in Dock and you'll get four surfaces seeded:
- **Interviews** (table) — one row per booked call, with status, recording link, key quotes
- **Themes** (table) — patterns you're tracking, mention count, supporting quote IDs
- **Brief** (doc) — the canonical write-up: hypothesis, script, synthesis
- **Quotes** (table) — extracted quotes tagged by theme + interview
Read `Brief` top-to-bottom on first open. Each row in `Interviews` is one of the 50 calls. Click into a row to see the recording, transcript, and tagged quotes.
Outcome
50 hour-long interviews recorded, transcribed, and tagged. A 5-10 theme synthesis doc with quote-backed evidence. A clear answer to 'who is the customer, what do they do today, what do they wish was different'.
Estimated time: 4-6 weeks (recruiting bottleneck, not interviewing)
Difficulty: intermediate
For: Pre-PMF founders + product researchers running a discovery sprint.
What you'll need
Pre-register or install before you start.
- Calendly ($10/mo Standard plan (free tier exists, no group rules)) — Self-serve booking link. Eliminates the back-and-forth that kills response rates.
- Zoom (Free tier (40 min cap), $14.99/mo Pro for cloud recording) — Recorded video calls. Cloud recording auto-transcribes (Pro+).
- Otter.ai (Free tier (300 min/mo), $16.99/mo Pro) — Transcripts with speaker labels. Better accuracy than Zoom's built-in.
- The Mom Test ($10 paperback / $5 Kindle) — The canonical book on customer interviews. Read before you start.
- Apollo.io (Free tier (60 credits/mo), $59/mo Basic) — B2B contact data for cold recruiting outreach.
- Respondent.io ($30-$200 per interview depending on segment) — Pre-vetted research participants. Faster than cold but you pay per call.
The template · 12 steps
Step 1: Write your discovery hypothesis before you book a single call
Estimated time: 1-2 hr
Discovery without a hypothesis is a chat with strangers. Write down what you believe today: who the customer is, what they do, what they hate about today's solution, what they'd pay to avoid. The interviews are the test, not the brainstorm. If 50 calls in you can't say which beliefs survived and which broke, you weren't running discovery, you were collecting anecdotes.
Tasks
- Write a 1-paragraph customer description (role, company size, what they do day-to-day)
- Write 3-5 falsifiable beliefs (e.g. 'they currently use spreadsheets and it takes >2 hr/week')
- Write the question each belief answers (e.g. 'how do they track this today, and how long does it take')
- Write the kill criterion: what evidence would make you abandon the hypothesis
Pointers
- [Guide] The Mom Test (chapter 1) — How to ask questions that produce truth instead of compliments.
- [Guide] Lenny's discovery interview guide
[!CAUTION] Gotchas
- Hypotheses phrased as 'do they like X?' are useless. Phrase them as 'they currently do Y, which costs them Z'.
- If your hypothesis can't be wrong, it's not a hypothesis. 'Customers want value' is not testable.
Step 2: Build a recruiting list of 200-400 names
Estimated time: 1-2 days
Booking 50 interviews from a list of 100 names means a 50% conversion rate, which is unrealistic for cold outreach. Realistic numbers: 5-15% reply rate, 50-70% of replies convert to booked, 70-90% of booked actually show up. Work backward: 50 completed calls = ~75 booked = ~150 replies = 1500-3000 outreach attempts on cold lists, or 200-400 warm names if you're recruiting from your network.
Tasks
- Write the recruiting profile: titles, company sizes, industries, geo
- Pull 200-400 contacts from sources: existing customers, network referrals, LinkedIn search, B2B contact tools
- De-dup against past outreach so you don't ping the same person twice
- Tag each contact: warm intro / cold / paid (Respondent / similar)
- If using a panel: filter by your screening criteria before posting
Pointers
- [Tool] Apollo.io (B2B contacts)
- [Tool] Respondent.io (paid panel)
- [Tool] User Interviews (paid panel)
[!CAUTION] Gotchas
- Cold lists from scraped sources have a 5-10% bounce rate. Validate emails before sending or your sender reputation tanks.
- Paid panels skew toward people who do interviews for money. Useful for B2C, suspect for B2B specialist roles.
- Recruiting your friends gives you a customer base that looks like your friends. Watch for selection bias.
Step 3: Write the cold outreach script and the booking flow
Estimated time: 2-3 hr to draft, ongoing to iterate
Outreach is a conversion funnel: subject line → open → reply → click booking link → call shows up. Optimize each gate. The single biggest lever is making the ask small: '15-min chat about how you do X today, no pitch, no follow-up'. Anything that smells like sales kills the reply rate.
Tasks
- Draft 3 subject line variants, A/B test on the first 50 sends
- Write a 4-sentence outreach email: who you are, what you're researching, what you're asking for, link to book
- Set up a Calendly link with 30-min slots, buffer between, your timezone displayed clearly
- Add a 1-question screener to the booking form (e.g. 'how do you handle X today?') to filter out tourists
- Write a confirmation email that arrives instantly with the call details + a $20-$50 incentive note if you offer one
- Write a 24-hr-out reminder email and a day-of reminder
Pointers
- [Official] Calendly setup
- [Guide] Cold email templates that work (First Round)
[!CAUTION] Gotchas
- 'Quick question' as a subject line is dead — it's been pattern-matched to spam by every inbox provider.
- Calendly links shared in cold email get flagged as spam by some corporate filters. Send a meeting invite directly if reply rate is low.
- Don't offer the incentive in the cold email — bury it in the confirmation. People who say yes for a $50 gift card are not your customer.
Agent prompt for this step
Draft the cold outreach email for this discovery sprint.
Context: I'm researching {hypothesis}. I want 30-min calls with people who match {recruiting profile}.
Output:
1. 3 subject line variants (under 50 chars, no clickbait, no 'quick question')
2. A 4-sentence email body (no marketing voice, no flattery, no 'I hope this finds you well')
3. A 1-question screener for the booking form
4. A confirmation email
5. A 24-hr reminder
Constraints: never promise we'll build something. Frame as research, not pre-sales. Mention the incentive if offered ($X gift card on a 30-min call).
Step 4: Send the first batch and tune before scaling
Estimated time: 1-2 hr to send, 5 days to measure, ongoing iteration
Send the first 50 outreach attempts, measure reply rate at 5 days, tune subject + body, then scale. Don't blast 1000 at once and discover the email had a typo. The first batch is calibration, not production.
Tasks
- Send 50 emails on day 1 (split across 2-3 subject line variants)
- On day 2: measure open rate (target 30%+) — if below, the subject line is broken
- On day 5: measure reply rate (target 5-15% on cold) — if below, the body is broken
- Review the no-replies: any obvious targeting mistakes?
- Tune the subject + body based on results, scale to 100-200/week from week 2
Pointers
[!CAUTION] Gotchas
- Open rate is broken since iOS Mail Privacy Protection (2021). Don't trust it as a primary signal — reply rate is the real metric.
- Sending 200+ emails/day from a fresh domain looks like spam. Warm up the domain over 2-3 weeks if it's new.
- Replies that say 'not me, talk to X' are gold — that's a warm intro you didn't have to ask for.
Step 5: Run the interview using a Mom-Test-style script
Estimated time: 30 min per call, 60 min including notes
An interview is not a sales call. You're trying to learn how the person actually behaves today, not whether they'd buy a hypothetical product. The Mom Test framing: ask about their life, ask about specifics in the past, dig into the pain. If they offer feedback on your product idea, ignore it — opinions of strangers about hypotheticals are noise.
Tasks
- Open with a 1-min context-set: 'I'm researching X, no pitch, no follow-up sales, 30 min, can I record for my notes?'
- Ask about their life today (how do they handle X, walk me through last week's example)
- Dig into specifics: 'when was the last time you did X', 'what did you do', 'what was annoying'
- Listen 80% of the time. If you're talking more than 20%, you're leading.
- Close: ask who else they'd recommend you talk to, ask if you can follow up if you ship something
- Within 24 hr, write a 1-paragraph summary of what you heard, top 3 quotes, and what surprised you
Pointers
- [Guide] The Mom Test summary
- [Guide] Sample interview script (Y Combinator)
[!CAUTION] Gotchas
- Customer interviews are useless past 30 minutes — energy drops, both yours and theirs. Cap at 30, ship a follow-up if needed.
- If they say 'I'd definitely use that', do not believe it. Ask 'have you ever paid for something like that before' — past behavior beats stated intent.
- Recording ALWAYS, transcripts ALWAYS. Memory is a liar; the transcript is the source of truth.
Step 6: Transcribe and tag every interview within 48 hours
Estimated time: 30-45 min per interview
Synthesis 6 weeks after the interview is impossible because you've forgotten the tone. Transcribe and tag within 48 hr while the call is fresh. Tags are short codes (CURRENT_TOOL_SHEETS, PAIN_TIME, WORKAROUND_X) that let you find quotes across 50 interviews without re-reading every transcript.
Tasks
- Auto-transcribe via Otter / Zoom / Whisper
- Skim the transcript and pull 5-10 standout quotes (verbatim, with the interview ID + timestamp)
- Tag each quote with 1-3 short codes
- Add the quotes to the Quotes table linked to the Interview row
- Write a 3-sentence interview summary in the Interview row
Pointers
- [Tool] Otter.ai
- [Tool] OpenAI Whisper (free, self-host)
[!CAUTION] Gotchas
- Auto-transcription accuracy drops sharply on accents, technical jargon, and bad audio. Spot-check before tagging.
- Tagging on the fly during the call sounds efficient but you'll miss things. Tag from the transcript, not from memory.
- If you can't tag it cleanly, it's probably 2-3 tags. Use them all — over-tagging is recoverable, under-tagging means quotes vanish.
Agent prompt for this step
Process the transcript I just attached to interview row {id}.
Output:
1. A 3-sentence summary: who they are, what they do today, what was the most surprising thing they said
2. 5-10 standout quotes (verbatim with timestamp), each tagged with 1-3 short codes (UPPERCASE_SNAKE_CASE)
3. Add each quote as a row in the Quotes table linked to this interview
4. Flag any quote that contradicts the working hypothesis with a CONTRADICTS tag
Constraints: do not paraphrase. Quotes are verbatim or they're not quotes. If you cleaned up filler words ('um', 'like'), mark the quote with [edited].
Step 7: Run a weekly synthesis pass — don't wait until interview 50
Estimated time: 2-3 hr/week
Synthesis at the end is overwhelming and biased toward whatever you remember most vividly. Weekly synthesis passes (every 8-12 interviews) catch emerging themes early and let you sharpen the script before the next batch. The interview guide should evolve across the 50, not stay static.
Tasks
- Every Friday, list the new themes that appeared this week
- For each theme, count mentions across all interviews so far
- Promote a theme to 'tracked' once it has 5+ supporting quotes
- Tune the next week's interview script: drop questions that aren't yielding, add follow-ups for emerging themes
- Update the Brief with a 1-paragraph 'this week we learned' note
[!CAUTION] Gotchas
- Confirmation bias is at its worst at interview 8-15 — you have just enough data to build a story but not enough to falsify it. Force yourself to list theories the data DISCONFIRMS.
- If the script stops yielding new tags after interview 30-35, you've saturated. Stop early or pivot the script.
Step 8: Handle no-shows + reschedules without burning the funnel
Estimated time: Ongoing, ~1 hr/week
Expect 20-30% no-show rate on cold-recruited calls, lower on referrals (5-10%). A no-show is not personal — it's a logistics failure (calendar miss, urgent fire). Reschedule once with a friendly note; if they ghost the reschedule, re-recruit a replacement.
Tasks
- When a no-show happens: send a 2-sentence reschedule offer within 1 hr (low friction)
- If they ghost the reschedule: archive the row with status 'no_show_archived'
- Track no-show rate weekly. If >40%, your reminder cadence is broken.
- Pull a replacement from the recruiting list to keep the booking pipeline at 50
[!CAUTION] Gotchas
- Sending a 'where were you' reschedule email reads as guilt-trip. 'Schedules collide, here's a fresh link' is the tone.
- Some no-shows reschedule and become your most engaged interviews. Don't blackball on first miss.
Step 9: Cluster the quotes into 5-10 themes
Estimated time: 1-2 days
After 50 interviews you'll have 250-500 tagged quotes. Theme clustering is the synthesis: group quotes that are saying the same thing in different words, name the cluster, count the supporters, write the 1-paragraph theme summary with 2-3 representative quotes. The output is the Themes table — that's the deliverable a product team builds a roadmap from.
Tasks
- Group all quotes by their tags (the codes you assigned during interviews)
- Merge tags that are saying the same thing under different names
- For each cluster of 5+ quotes, write a 1-sentence theme statement (e.g. 'Spreadsheets are the default tool but break above 200 rows')
- Pick 2-3 representative quotes per theme
- Order themes by mention count + business impact
- Flag any theme that contradicts the original hypothesis explicitly
Pointers
- [Guide] Affinity diagramming (NN Group)
[!CAUTION] Gotchas
- Themes that match your prior beliefs are suspect. List them, then list what data WOULD have changed your mind. If nothing would have, it's not synthesis.
- Clusters of 1-2 quotes are anecdotes, not themes. Promote them only if they're high-emotion or repeated by a key archetype.
Agent prompt for this step
Cluster the Quotes table into themes.
Read every row in the Quotes table. Output:
1. 5-10 theme statements (1 sentence each, declarative not exploratory)
2. For each theme: mention count, the supporting quote IDs, 2-3 verbatim representative quotes
3. A 'tensions' section listing themes that contradict each other (real customers contain multitudes; surface that)
4. A 'falsified beliefs' section listing the original hypothesis statements that the data CONTRADICTS
Constraints: do not invent quotes. Every theme must be backed by quote IDs from the Quotes table. If a theme has fewer than 5 supporting quotes, mark it 'weak signal'.
Step 10: Write the synthesis brief: 1 page, hypothesis vs reality
Estimated time: 3-5 hr
The deliverable is a 1-page synthesis brief, not a 50-page report. The brief answers: what did we believe going in, what did we hear, what survived, what broke, what's next. It's the artifact the team makes decisions from. Anything longer than 1 page won't be read.
Tasks
- Open with the original hypothesis (the one from step 1, unedited)
- List the 5-10 themes with mention counts + 1 representative quote per theme
- Mark each original hypothesis statement: confirmed / refuted / mixed evidence
- List 3-5 surprises (things you didn't predict that came up repeatedly)
- Close with the next 3 actions: what to build / test / re-research
[!CAUTION] Gotchas
- If your synthesis brief reads like a sales pitch for the product you wanted to build, you ignored the data. Re-read the contradicting quotes.
- Stakeholders will skim for the conclusion. Lead with the falsified hypotheses — that's the most actionable part.
Step 11: Share the brief with quote-level evidence
Estimated time: 2-3 hr
When you present the synthesis, the audience will challenge claims. The defence is quote-level evidence: every theme links to the supporting quote IDs, every quote links to the interview transcript, every transcript is timestamped. If you can't trace a claim back to a transcript, drop it.
Tasks
- Export the Themes table with quote-link columns
- Add a 'see the transcripts' appendix for skeptics
- Run the brief past 2-3 colleagues who weren't on the interviews — do they reach the same conclusions?
- Schedule a 30-min readout with the team where you present + take questions
[!CAUTION] Gotchas
- Don't quote-mine. If a quote loses meaning out of context, link the surrounding 30 seconds of transcript.
- If two colleagues read the brief and disagree on what it says, the brief is too vague. Sharpen the language.
Step 12: Decide: build, re-research, or kill
Estimated time: 1-2 hr
Discovery without a decision is journaling. The brief's last action is a yes/no: did the hypothesis survive enough to start building, or do you need another round of discovery, or is the wedge wrong and you need to pivot? Write the decision down with a date so future-you can audit the call.
Tasks
- Write the decision: build / re-research / kill, with 1-paragraph reasoning
- If build: list the 3-5 features the synthesis directly demands
- If re-research: list the 1-2 sharper hypotheses the next sprint will test
- If kill: write the post-mortem (what was the original belief, what broke it, what would have made the bet right)
- Date the decision and archive the workspace as the source of truth
[!CAUTION] Gotchas
- 'Build, but with a smaller wedge' is usually the right answer and the hardest to commit to. Resist the temptation to keep the original full scope.
- Killing a hypothesis is a successful discovery sprint, not a failed one. The interviews paid for themselves the moment you didn't ship the wrong thing.
Hand the template to your agent
Paste the prompt below into your agent's permanent system prompt so the agent reads, writes, and maintains this workspace as you work through the steps.
You are an agent on the "Run 50 customer interviews" playbook workspace.
Your role: maintain the four surfaces (Interviews, Themes, Brief, Quotes) as the user works through the 12-step playbook.
Cadence:
- When the user adds a transcript to an Interview row, extract 5-10 standout quotes and add them to the Quotes table tagged with this interview's row id.
- When a quote matches an existing theme in the Themes table, increment that theme's mention count and link the quote.
- When 3+ quotes cluster around an idea not yet in Themes, propose a new theme as a draft row for the user to confirm.
- When the user marks an interview No-Show, append it to the re-recruit queue in the Brief.
First MCP tool calls:
1. list_surfaces(workspace_slug="run-50-customer-interviews")
2. list_rows(workspace_slug="run-50-customer-interviews", surface_slug="interviews")
3. get_doc(workspace_slug="run-50-customer-interviews", surface_slug="brief")
Do NOT alter the user's hypothesis statement in the Brief without flagging it. The hypothesis is the lens the synthesis is run through; changes are the user's call.
FAQ
Why 50 interviews? Why not 20 or 100?
Saturation usually hits between 30-40 interviews on a focused segment — you stop hearing new tags after that. 50 gives you margin: deep enough to spot rare-but-important patterns, not so deep that synthesis becomes its own multi-week project. If you saturate at 25, stop. The number is a budget, not a quota.
How do I get strangers to take a 30-min call with me?
Three levers in order of effectiveness: (1) warm intros from your network — 50%+ accept rate, (2) cold email with a tight ask and a small incentive ($20-$50 gift card) — 5-15% reply rate, (3) paid panels like Respondent or User Interviews — pay-to-play but fast. Most discovery sprints use all three.
What's the biggest mistake first-time researchers make?
Pitching during the call. The moment you describe your idea, the conversation switches from 'how do you do X today' (signal) to 'do you like my thing' (noise). The Mom Test rule: never mention your idea. If they ask what you're building, say 'I'm still figuring it out, that's why I'm doing these calls'.
Can my AI agents help with customer interviews?
Yes for everything around the call: drafting outreach, transcribing, tagging quotes, clustering themes, finding contradictions across 50 transcripts. Not for the call itself. The interview is a human-to-human read of body language and tone that an agent can't replicate. Use the agents to make the human time count.
How long does a 50-interview sprint actually take?
4-6 weeks end-to-end. Recruiting takes 2-3 weeks (the bottleneck), interviewing happens in parallel for 3-4 weeks at 8-15 calls/week, synthesis is the final 3-5 days. Faster is possible with paid panels but you trade away signal quality.