Beta Testing in Software Testing: A Four-Week Run-Book
Beta testing explained through one four-week beta for a mobile app feature: who tests, what QA does each week, how to triage feedback, exit criteria, and alpha vs beta testing.
Last updated 2026-11-01 · 8 min read · By SoftwareTestPilot
Most explanations of beta testing in software testing stop at a definition. This one is a run-book: a single four-week beta for a new "split the bill" feature in a payments app, with what QA does in each week. The app and numbers are an illustration, not a real case study, but the steps are the ones teams actually follow.
What is beta testing?
Beta testing is acceptance testing done by real users outside the development organisation, in their own environments, before the product is released to everyone. The ISTQB glossary groups it with other forms of user acceptance testing done at the customer's side.
The point is not to find basic bugs. Those should already be gone. The point is to see what happens on devices, networks, data and habits that your test lab can't copy.
Open, closed and staged betas
- Closed beta: invited users only. Easier to manage and gives cleaner feedback.
- Open beta: anyone can join. More devices and more load, but noisier feedback.
- Staged rollout: the release goes to a small percentage of real users first, then grows. Both app stores support this, along with dedicated testing tracks such as Apple TestFlight and Google Play's closed and open testing tracks.
Our example uses a closed beta that widens each week.
The four-week plan
Before week 1, the build must have passed system testing, a full regression run and a smoke test. Beta testers are volunteers; a broken build loses them fast.
| Week | What happens | QA's job |
|---|---|---|
| 0 (prep) | Pick testers, write the beta brief, set up crash reporting and a feedback form | Confirm the build passed regression and smoke tests; write 4-6 tasks for testers |
| 1 | Small group (for example 30 users) gets the build | Watch crash reports daily; reproduce every crash and log it with steps |
| 2 | Fixes shipped in a second build; group widened | Retest the fixes, run a quick regression on the new build |
| 3 | Full beta group; short survey sent | Triage feedback into bugs, usability issues and feature requests |
| 4 | Freeze; check exit criteria | Report open issues by severity and give a go / no-go recommendation |
The tasks you give testers matter. "Try the app" gets little back. "Split a dinner bill of ₹1,240 between three friends, one of whom isn't on the app" gets specific feedback about exactly the flow you changed.
Turning beta feedback into something useful
Beta feedback is short and vague. QA's real value is turning it into decisions. Every item goes into one of four buckets:
| Feedback received | Type | What QA does |
|---|---|---|
| "App closed when I tapped Pay on my old Android phone." | Bug (crash) | Find the matching crash log, reproduce on that OS version, log as high severity |
| "Couldn't find where to split the bill." | Usability | Count how many testers said the same; pass to product with the count |
| "Please add UPI autopay." | Feature request | Tag and pass on; not a defect, not a launch blocker |
| "It's slow." | Unclear | Ask which screen and network; check performance data before logging |
When several testers report the same thing in different words, merge them into one issue and record the count. "7 of 30 testers couldn't find split" is far more convincing to a product manager than seven separate comments. For logging the real bugs, follow the same rules as any good bug report: steps, device, OS version, expected and actual result.
Exit criteria: when is the beta done?
Write these down in week 0, agreed with product and development. A typical set:
- No open critical or high-severity defects in the beta flow.
- Crash-free sessions at or above the team's agreed target, measured over the last build.
- Every usability issue reported by several testers has a decision: fix now, fix later, or won't fix.
- The planned beta period has ended, or every tester task has been completed by enough people.
If the criteria aren't met at the end of week 4, QA reports that plainly with the list of open issues. The release decision belongs to the product owner; QA's job is to make the risk visible.
Alpha vs beta testing
Both are forms of acceptance testing, and interviewers often ask for the difference.
| Question | Alpha testing | Beta testing |
|---|---|---|
| Who tests? | Internal people: QA, developers, product, other staff | Real users outside the company |
| Where? | In a controlled test or staging environment | On users' own devices, networks and data |
| When? | After system testing, before any outside release | After alpha, before general release |
| Main goal | Find defects before outsiders see the product | Check the product works and makes sense in real-world use |
| Who fixes what's found? | Fixed quickly, often in the same cycle | Triage first; only release-blocking issues are fixed before launch |
| Typical feedback | Detailed bug reports with steps | Crash logs, short comments, ratings, "I couldn't find X" |
A one-line answer: alpha testing is done internally in a controlled environment; beta testing is done by real external users in their own environment, after alpha and before release.
Where beta fits with other testing
Beta does not replace planned testing. It sits at the very end of the cycle and catches a different kind of problem. Some issues only surface through real use, which is why exploratory testing before beta is a good rehearsal: it's the closest a tester gets to behaving like an unscripted user.
If you're preparing for interviews, practise explaining one beta you took part in, even as a user, and one bug it found. You can swap beta testing stories with other testers in the QA Community.
Key takeaways
- 1Beta testing = real external users, real environments, just before release.
- 2Alpha testing is internal and controlled; beta testing is external and uncontrolled.
- 3Beta is not a replacement for system or regression testing; the build should already be stable.
- 4QA's main beta jobs: turn vague feedback into reproducible bugs, triage, and give a go / no-go recommendation.
- 5Decide exit criteria before the beta starts, or it will never end.
Common mistakes to avoid
- Sending an unstable build to beta testers and burning their goodwill.
- Collecting feedback with no plan to read or triage it.
- Treating every feature request as a bug that blocks release.
- Running a beta with no end date or exit criteria.
- Not setting up crash reporting, so crashes arrive as "it closed" with no details.