digi.ai.in
← digi.ai.in / blog
2026-08-198 min readPerformance

What Changes About a Codebase Once Real Users Show Up

A codebase that passes every test in development can still fail in production, because real users bring devices, networks and traffic that dev environments don't.

A codebase that works perfectly in development can still fail once real users arrive, because real traffic exposes conditions — slow networks, unindexed queries, edge-case devices — that no local test replicates. This is not a vague risk. It is a measured, documented gap that the web performance industry has built entire tools around, and understanding it changes how you should plan, test and budget for a build.

Lab data and field data are not the same thing, and Google treats them differently

Most teams building a website check it with tools like Lighthouse before launch. That's "lab data" — a controlled, repeatable test on a fixed device and network profile. It is useful, but it is not what determines your search performance.

Google's own developer documentation formalizes a second category: Real User Monitoring (RUM) data, also called field data, which captures the performance experienced by a site's actual users, and this field data is what Google actually uses to judge whether a site meets its recommended Core Web Vitals thresholds. Lab data is a rehearsal. Field data is the performance.

For a founder or ops manager commissioning a site, the practical takeaway is: ask what the plan is for measuring performance after launch, not just before it. A pre-launch report is a snapshot of one condition. It is not evidence of how the site will behave for your actual customers.

Why "real user data" isn't even one single number

Once a site has enough traffic, you'd expect a single, authoritative picture of how it performs. It doesn't quite work that way, and the disagreement here is well documented.

Google's public Chrome User Experience Report (CrUX) aggregates real-user data, but only over a rolling 28-day window, whereas dedicated RUM tools update immediately after a code change is deployed. That means CrUX is always describing the past month, not today.

More importantly, independent monitoring vendors report their own real-user numbers frequently don't match Google's published CrUX figures for the same page and metric, because CrUX only reflects a subset of users — those signed into Chrome, syncing history, with usage statistics enabled, and without a corporate policy disabling it — while independent RUM tracking isn't restricted the same way. So two dashboards can both be "correct" and still disagree, because they're counting different users.

This is why serious web performance research pairs both data types rather than relying on either alone. HTTP Archive's Web Almanac combines lab crawls with CrUX field data across metadata from over 16 million websites tested monthly, specifically because CrUX is used to work around some of the inherent limitations of a lab dataset. If you commission a site and only ever see a pre-launch Lighthouse PDF, you are seeing less than half the picture the industry considers standard.

Technical debt: the shortcuts that were invisible until traffic arrived

Performance metrics are one lens. The other is structural: code that "worked" in development starts costing real money once it has to survive real usage patterns.

One practical proxy engineering teams use is the rollback/hotfix rate after a release. A Back Market engineering post describes how, once debt accumulates, incident resolution times skyrocket and backlog items get picked up last — the team spends more time firefighting than building.

A separate practitioner analysis of 100 startup codebases (self-described as anecdotal, not peer-reviewed, so read it as a pattern rather than a statistic) found the failures that showed up once real traffic arrived were rarely exotic. Commonly it was things like missing database indexes — a problem invisible with ten test records that becomes a full page timeout with ten thousand real ones. The same analysis noted these were preventable with about two weeks of upfront architecture planning.

This is now being studied at a larger scale too. An empirical study built a dataset of over 300,000 verified AI-authored commits across more than 6,000 GitHub repositories, specifically to check whether code-quality issues get fixed quickly or accumulate as technical debt after the code goes into production — and found that real-world behavior diverges from what shows up in controlled lab settings.

The pattern across all three sources is consistent: the things that break in production are usually boring, checkable things (missing indexes, unhandled edge cases, no rollback plan), not exotic bugs. A short architecture review before launch is cheap compared to firefighting after.

India-specific reality: your "real users" span a wider gap than most playbooks assume

Most performance advice online assumes a fairly narrow spread of conditions — decent Wi-Fi versus "average" 4G. That assumption doesn't hold for an Indian audience.

As of 2025, India has over 1 billion internet users, with 70% internet penetration, and average mobile data consumption reached 19.5GB per user per month in 2022. TRAI itself only raised the minimum broadband download-speed requirement from 512 kbit/s to 2 Mbit/s in 2023 — a reminder of how recently "broadband" in India meant something quite slow by global standards. TRAI's own filings also show 284.64 million broadband users in rural India, where rural data consumption is around 45% of overall mobile data usage.

Network speed in India is also unusually split rather than converging on a single average. Historical OpenSignal testing found India had the slowest average 4G speeds among 88 countries measured, at roughly 6.07 Mbps, attributed to congestion rather than lack of coverage. More recent Ookla data shows this has moved: median 4G speed had risen to about 18.93 Mbps by Q4 2023, while median 5G speed had already passed 300 Mbps in the same period. TRAI's latest quarterly report confirms 4G is still carrying the bulk of traffic even as 5G grows, with 2G, 3G, 4G and 5G contributing about 0.07%, 0.13%, 60.13% and 39.68% of total wireless data volume respectively.

Put simply: your real-user base likely includes a congested-4G majority and a very fast 5G minority within the same population, at the same time. A site built and tested only on office broadband is being validated against neither of these groups.

A short checklist for what to ask before and after launch

Question Why it matters
Do we have real user monitoring set up, not just a pre-launch Lighthouse report? Lab data and field data routinely disagree
Whose "real user" numbers are we trusting — CrUX or an independent RUM tool? They measure different, non-overlapping user subsets
Has the database been checked for missing indexes under realistic data volumes? This was one of the most common production failures in the 100-startup analysis
What is our rollback/hotfix rate after each release? A practical proxy for accumulating technical debt
Has the site been tested on constrained 4G, not just office Wi-Fi or 5G? 4G still carries ~60% of India's wireless data volume
Have we planned two weeks of upfront architecture review? Cited as sufficient to prevent common preventable failures

What this means practically

None of this means development testing is pointless — it's the necessary first pass. But it means a launch date shouldn't be treated as the finish line for performance and stability work. The gap between lab and field, the split between rural and urban network conditions, and the slow accumulation of technical debt are all things that only become visible once actual traffic starts hitting the code. Budgeting time and attention for that phase, rather than assuming a clean pre-launch test means the work is done, is the difference between a site that holds up and one that quietly degrades for a chunk of its users without anyone noticing.

Questions people also ask

Does a 100/100 Lighthouse score mean the site will perform well for real users?

Not necessarily. Lighthouse is lab data — a single simulated test run. Google's own documentation notes that lab and field data can disagree in either direction, so a high lab score is not proof of good real-world performance.

Why do two different analytics tools show different real-user performance numbers for the same site?

Because they're not measuring the same users. Google's CrUX dataset only includes Chrome users signed in, syncing history, with usage reporting enabled, while independent RUM tools track a different, unrestricted population.

What's the single most common production issue that doesn't show up in development?

Across the sources reviewed here, missing database indexes stand out as a recurring, preventable cause of production failures once real traffic volume arrives, according to an analysis of 100 startup codebases.

Written by the studio behind digi.ai.in. Questions? hi@digi.ai.in or WhatsApp +91 98605 96905
WhatsApp us