Lesson 1 of 87 min read
Why report and data testing matters
Managers make decisions from dashboards: revenue, sign-ups, stock levels. A wrong number on a report can be more expensive than a broken button, because nobody notices until a bad decision is made. This lesson explains what report testing covers and where the bugs usually hide.
Key takeaways
- 1A report is a chain: source data, transformation (queries, ETL), storage, then display. Bugs can enter at every step.
- 2Main checks: correct numbers, correct filters, correct dates and time zones, correct formatting, fast enough to load.
- 3Common bugs: double counting after joins, missing rows, wrong time zone, rounding, stale data and filters that are ignored.
- 4Your 'oracle' is usually the source data: recompute the number yourself with SQL and compare.
- 5Ask who uses the report and which decisions depend on it; test those numbers first.
A real-looking report bug
Dashboard tile: 'Orders today: 1,248' Source check:
SELECT COUNT(*) FROM orders
WHERE created_at >= '2026-11-03' AND created_at < '2026-11-04';
-> 1,190
Cause found: the report joined orders to order_items, so orders with 2+ items were counted more than once. Fix: COUNT(DISTINCT o.id).
Go deeper
Expert notes beyond the basics
- 01
Follow the number
Pick one tile and trace it backwards: which API returns it, which query builds it, which tables feed that query and where those tables are loaded from. Each hop is a place to test.
- 02
Risk-based focus
Not every number is equal. Revenue, payouts and stock usually matter most. Ask the report owner which three numbers would hurt most if wrong, and test those deeply.
Common mistakes to avoid
- - Checking only that the chart loads.
- - Testing with an empty or tiny dataset.
- - Ignoring data freshness.
- - Not asking who uses the report.
Try it yourself
List five things that could make 'Sign-ups this week' wrong.
Write your answer on paper or in a sheet first, then compare.
Interview questions
How to answer like a confident QA engineer
- Q1
How would you test a sales dashboard?
Model answer
I start with the numbers people decide on, recompute each from the source tables with SQL for a known period and compare. Then I test filters and date ranges, time zones, empty periods, rounding and currency, drill-downs adding up to totals, exports matching the screen, data freshness, permissions so users only see their own region, and load time with realistic data volumes.
- Q2
What is a test oracle for reports?
Model answer
An independent source of the expected value, usually a SQL query on source data or a hand-calculated small dataset.
- Q3
What is stale data?
Model answer
Data that has stopped updating, so the report shows old numbers that look valid.
Knowledge check
+5 points
What is usually the best way to verify a report total?