Kickstart

Research & Validation

Every Research & Validation post from the Kickstart blog, condensed to its key points – skim the bullets, and read the full article when you want the detail.

How to user test your app with just 5 people

  • What the model claims: With an assumed 31% per-user discovery rate, Nielsen’s five-user model estimates that five similar users reveal about 85% of the usability issues that setup can expose – not 85% of bugs, accessibility needs, or problems across distinct audiences.
  • One tester beats zero: A single relevant participant already surfaces about a third of what there is to learn, because “the difference between zero and even a little bit of data is astounding.”
  • Small rounds, not big studies: Later participants increasingly repeat issues already observed, so test, improve, and test again – and give distinct groups like novices and VoiceOver users their own coverage.
  • Apple’s three questions: Ask “Do you know how to XYZ?”, “Is it easy to XYZ?”, and “How can we make this better?” rather than “do you like it?”, and while watching, don’t argue, defend, or dismiss.
  • Give them a task, not a tour: Set one specific task (“sign up for the app, then log your first workout”) and find your app’s equivalent of Mercury Weather’s Two-Second Glance Test.
  • Treat findings as hypotheses: Fix blockers and high-severity patterns first, weigh severity and recurrence before changing deliberate workflows, and give more time to anything users uniformly praise – that’s how Halide’s reduced-processing option grew into Process Zero.
Read the full article: How to user test your app with just 5 people

App Store competitor analysis: how to spy on your rivals (ethically)

  • Download three direct competitors: search your most important term on a real device and study three apps serving the same customer and job – everything else in the stack depends on this.
  • “What’s New” archaeology: every product page’s Version History shows the 25 most recent release notes – read them bottom to top to watch a competitor’s strategy unfold, and note their release cadence.
  • Take monthly metadata snapshots: save each rival’s product page, subtitle, screenshots, and prices into a dated folder, then for every change write what you observed, your best hypothesis, and the cheapest test that would settle it.
  • Read the four-star reviews: these are users who liked the app but still named one or more problems with it – sort by most critical, read 20 or 30 per competitor, and if three apps get hammered for a common complaint then you might have found a gap the whole market is leaving open.
  • Capture paywalls and ads: screenshot every onboarding step and paywall (trial length, plan count, pre-selected plan), then check the Meta Ad Library – free and public – to see which benefit each competitor’s adverts lead with.
  • Be the best at something: write a plain SWOT for each competitor, apply Chesterton’s Fence to features you don’t understand, and treat zero competition as a warning – it often means no market.
Read the full article: App Store competitor analysis: how to spy on your rivals (ethically)