Visa has introduced a specialized benchmark, the Visa Agentic Commerce Benchmark, designed to assess how effectively AI agents handle real-world product purchasing scenarios. The results of the first round of testing fell far short of ideal and clearly demonstrate the gap between the ambitions of autonomous commerce and the current level of technology.
The trials involved 22 models from 12 labs — an impressive sample covering a wide range of leading players in the artificial intelligence market. The average success rate was only 57.2% of attempts passed. In other words, nearly every second product-selection task ended in failure.
What lies behind the 57.2% figure
A figure just above half is not merely a statistical error but a systemic signal. Agentic commerce assumes that AI independently interprets the user's request, navigates catalogs, compares parameters, and makes a purchase decision. It is precisely along this multi-step path that the models reveal their vulnerability: errors accumulate at each stage, reducing overall reliability.
For the payments industry and e-commerce, this is a fundamentally important conclusion. Autonomous agents promise to radically transform the user experience, but at the current level of accuracy, mass trust in them is premature. Business models built on fully delegating procurement and purchasing functions to AI cannot yet rely on consistent results.
Visa's benchmark sets an important reference point for the entire industry: it moves the discussion of agentic commerce from the realm of marketing promises into the realm of measurable metrics. Further rounds of testing will show how quickly models can overcome the reliability threshold required for real transactions.
Cryptalist analysis: The 57.2% result reflects the fundamental immaturity of agentic AI solutions for commerce: the gap between demonstration capabilities and production reliability remains significant. For the market, this is a signal that the infrastructure for autonomous payments and procurement will be built gradually, and demand will shift toward hybrid models involving humans. Investors and payment platforms should factor in a long horizon for the adoption of agentic commerce.