HomeinsightsApplication Performance: Data First, AI Second
data & AIquality assurance & testing
Application Performance: Data First, AI Second
Didier Nizard, Directeur d'Itecor Paris et Franck Jarre, QAT Practice Representative · November 19, 2000
What if the main challenge of applying AI to performance were not the algorithm, but the quality of the data fed into it?
THE KEY IDEA
Value does not depend on the volume of data collected. It stems from reliable, contextualised and correlated data capable of informing a decision. AI speeds up analysis; it does not replace this foundation.
Response times, errors, CPU, memory, Kubernetes metrics, logs, traces, deployment events: information systems already generate a vast amount of data. The challenge lies in making sense of it.
A transaction takes 2.5 seconds instead of 800 ms during a load test. The slowdown is measured, but its cause remains unknown: is it CPU saturation, the database, an external dependency, or a regression introduced by the latest version? To understand this, the data must be cross-referenced. This is where NeoLoad and the Grafana ecosystem complement each other: one measures performance under controlled conditions, whilst the other provides the context needed for diagnosis.
NeoLoad: measuring under controlled conditions
With NeoLoad, we don’t just observe the system: we define the test conditions. Concurrent users, load scaling, endurance, stress testing, business workflows, data volumes and critical transactions all form part of reproducible test campaigns. Their history enables us to answer specific questions: is this version slower? At what load level does performance degradation occur? Has an optimisation actually worked?
However, you must compare like with like. Rule number one: examine the p95 and p99 percentiles, not just the average. It is often the highest response times that compromise a service objective. Rule number two: maintain the same dataset, the same platform state and the same warm-up phase, then repeat the tests to measure natural variability. Two isolated runs with a 10 per cent deviation are not enough to draw a conclusion; three stable runs followed by a comparable deviation constitute a much stronger signal of regression.
When integrated into the CI/CD pipeline as quality gates, these metrics become a delivery criterion: a release that fails to meet the service level objectives (SLOs) for a critical transaction will not proceed to production. Representativeness remains essential. This applies not only to scenarios but also to the scale of the platform, its data and its dependencies, including simulated ones. Without a credible environment, results are difficult to translate into production.
Grafana: understanding what lies behind the figures
NeoLoad indicates when performance is deteriorating. Grafana helps to understand what is happening at the same time: metrics in Mimir, logs in Loki and traces in Tempo, all brought together in a single interface. An increase in response times, coupled with the saturation of a component or an abnormally slow dependency, becomes a useful diagnostic lead. Endurance tests illustrate this benefit well. A memory leak or a garbage collector drift may only become apparent after several hours of stable load. It is the trend in the graphs, rather than an isolated measurement, that reveals this.
For this analysis to be useful, three conditions are essential. Firstly, instrument the test platform with the same rigour as the production environment; you can even retain all traces there when the production environment only samples them. Secondly, collect application metrics (connection pools, threads, the garbage collector, queues) beyond just CPU and memory. Finally, adopt a common language: OpenTelemetry enables you to track a transaction across services without relying on a proprietary format.
Value is created through correlation
QA, development, infrastructure, SRE and business teams each hold part of the explanation. It is not a matter of adding a dashboard, but of linking signals around a single event, from the load injected right through to the business impact.
THE ANALYSIS CHAIN
NeoLoad load → business workflow → response time → metrics → logs → traces → anomaly → probable cause → business impact. Measurement and observability then inform human analysis, potentially assisted by AI.
Four prerequisites make this chain possible: synchronising the clocks of the load generators and the platform; annotating campaigns, their stages and the versions tested in Grafana; exporting NeoLoad data series (active users, throughput, response times) to Mimir, via API or export; and finally, propagating the trace context. By injecting W3C Trace Context headers, NeoLoad enables simulated calls and their identifiers to be retrieved in Tempo.
The consequence then becomes clear: a shopping basket that cannot be confirmed, a delayed payment or processing outside the SLO. We move from a technical symptom to an identifiable business impact.
ILLUSTRATIVE EXAMPLE
NeoLoad gradually scales up to 3,000 users. At 2,200 users, the p95 for ‘basket validation’ rises from 600 ms to 1.8 s, whilst the other stages remain stable. In Grafana, this plateau corresponds to 100 per cent saturation of the database connection pool. Tempo traces show that slow transactions are waiting more than one second for a connection. After resizing the pool and rewriting a query, a new test measures 650 ms at the 95th percentile with 3,000 users. This correlation guided the investigation towards the correct cause, rather than blaming the database as a whole.
AI depends first and foremost on data quality
AI can analyse telemetry, detect anomalies, correlate signals and speed up investigations. However, it cannot reconstruct information that has never been collected. Unrepresentative scenarios, poorly defined transactions or traces lacking context simply lead to misleading analyses being produced more quickly.
An AI strategy applied to performance therefore begins with decisions about data: what data to collect, at what level of detail, for how long, and using which user experience metrics? On this basis, tools such as Grafana Assistant and Assistant Investigations, within Grafana Cloud, make it easier to query telemetry using natural language. The expert remains responsible for the context, causal interpretation and decision-making.
Our belief
The future of performance testing is not simply a matter of injecting more load or multiplying the number of dashboards. It rests on the ability to make the data speak for itself. Itecor Paris supports its clients throughout this entire process: relevant scenarios and representative environments, reliable measurements with NeoLoad, structured observability with Grafana and OpenTelemetry, signal correlation and AI-assisted use case preparation.
Three questions for your organisation
Does your testing platform adequately reflect the production environment in terms of scale, volume and dependencies?
Can your metrics, logs and traces be viewed together and linked to a single transaction?
Are your load campaigns, their stages and the versions tested annotated in your dashboards?
Three ‘yes’ answers: you have a solid foundation for leveraging AI. One ‘no’ answer: you have identified a concrete priority.
IN A NUTSHELL
Before asking AI why an application is slowing down, let’s give it the right data, at the right time and in the right context. AI can become the co-pilot; data remains the fuel.
Would you like to assess the maturity of your performance and observability pipeline? Our NeoLoad and Grafana experts are offering a 30-minute consultation.
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie
Duration
Description
__hssrc
Session
This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.
_GRECAPTCHA
5 months 27 days
This cookie is set by the Google recaptcha service to identify bots to protect the website against malicious spam attacks.
cookielawinfo-checkbox-advertisement
1 year
Set by the GDPR Cookie Consent plugin, this cookie is used to record the user consent for the cookies in the "Advertisement" category .
cookielawinfo-checkbox-analytics
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional
11 months
The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
CookieLawInfoConsent
1 year
CookieYes sets this cookie to record the default button state of the corresponding category and the status of CCPA. It works only in coordination with the primary cookie.
JSESSIONID
session
New Relic uses this cookie to store a session identifier so that New Relic can monitor session counts for an application.
viewed_cookie_policy
11 months
The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Cookie
Duration
Description
__cf_bm
30 minutes
This cookie, set by Cloudflare, is used to support Cloudflare Bot Management.
__hssc
30 minutes
HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Cookie
Duration
Description
__hstc
1 year 24 days
This is the main cookie set by Hubspot, for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).
_ga
2 years
The _ga cookie, installed by Google Analytics, calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognize unique visitors.
_ga_*
1 year 1 month 4 days
Google Analytics sets this cookie to store and count page views.
_ga_JYCPSB48B8
2 years
This cookie is installed by Google Analytics.
_gat_gtag_UA_*
1 minute
Google Analytics sets this cookie to store a unique user ID.
_gid
1 day
Google Analytics sets this cookie to store information on how visitors use a website while also creating an analytics report of the website's performance. Some of the collected data includes the number of visitors, their source, and the pages they visit anonymously.
CONSENT
16 years 2 months 25 days 10 hours
YouTube sets this cookie via embedded youtube-videos and registers anonymous statistical data.
hubspotutk
1 year 24 days
This cookie is used by HubSpot to keep track of the visitors to the website. This cookie is passed to Hubspot on form submission and used when deduplicating contacts.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Cookie
Duration
Description
IDE
1 year 24 days
Google DoubleClick IDE cookies are used to store information about how the user uses the website to present them with relevant ads and according to the user profile.
test_cookie
15 minutes
The test_cookie is set by doubleclick.net and is used to determine if the user's browser supports cookies.
VISITOR_INFO1_LIVE
5 months 27 days
A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.
YSC
Session
YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.
yt-remote-connected-devices
never
YouTube sets this cookie to store the user's video preferences using embedded YouTube videos.
yt-remote-device-id
never
YouTube sets this cookie to store the user's video preferences using embedded YouTube videos.