Global networks are increasingly implementing computer vision solutions. The rapid growth of the market is supported by retailers integrating advanced analytics and generative models into pricing, merchandising, and customer interactions.
According to Statista, the number of stores worldwide with fully autonomous checkouts has grown from about 350 in 2018 to a projected 10,000 by 2024. AI is transforming the way people shop, from the way products are displayed on shelves to the methods of payment. Let’s explore some of the most exciting real-world innovations and applications that are transforming retail operations today.
Cashier-less stores
Stores use a combination of computer vision, sensor systems, and real-time shopping behavior analysis to identify items a customer is picking from a shelf and automatically charge them as they leave the store. This eliminates the need for checkout lines and queues, making the shopping experience as fast and contactless as possible.
With CV systems and motion-tracking algorithms, cashierless stores can accurately identify each visitor’s actions, recognizing products even in partial darkness or changing lighting. Companies like Amazon widely implement such solutions.
In October, Amazon took another step towards full logistics automation by introducing the robotic system Blue Jay and the AI agent Project Eluna.
These systems use CV to automate giant fulfillment centers. This means the “grab-and-go” technology we know from cashierless stores is now scaled to the entire infrastructure. Thus, CV and AI help robots find, pick, and pack items faster, significantly accelerating delivery.
Amazon is not only focused on speed, but also on sustainability. Packaging optimization systems help reduce waste, while investments in clean energy and energy-efficient technologies support the environmental efficiency of logistics. As a result, warehouse automation combines high productivity with ecological care, creating a more sustainable logistics ecosystem.
Smart shelf monitoring
This retail trend combines CV, the Internet of Things (IoT), and AI to automate the monitoring of products on shelves. Automated systems enable real-time monitoring of product availability, control of inventory, and dynamic price updates, ultimately solving the problem of automated customer service and optimizing staff productivity.
For this, various types of computer recognition are employed: object recognition to identify products on the shelf, product classification to confirm the category or brand, monitoring of empty spaces to signal the absence of a product, and recognition of barcodes and QR codes for integration with the inventory management system. The primary goal is to ensure constant access to products, minimize empty shelves, and automate routine inventory tasks.
In June, at the VivaTech exhibition, Carrefour announced a partnership with VusionGroup to test a new generation of “connected stores” by digitizing shelves. Carrefour became the first grocery retailer in Europe to implement this technology. The project is based on VusionGroup’s advanced EdgeSense technology platform, an integrated system that combines an innovative rail system, electronic shelf labels, CV, AI, and ultra-accurate data.
The EdgeSense platform was created to achieve four key objectives: merchandising control, i.e., ensuring product display consistency through automated visual monitoring of shelves; product availability on shelves through real-time monitoring and automatic detection of missing products; price compliance through automated control of electronic labels; and precise geolocation of products to improve the shopper’s journey in the store and optimize e-Commerce picking.
Cameras constantly scan the shelves, and as soon as the system detects a shortage of goods, it sends notifications to employees for replenishment of stocks, price changes, or promotions. These technologies are currently being tested in a pilot Carrefour store in Villabé, France.
Fraud prevention
As self-service systems in retail continue to grow, the challenge of preventing both accidental and intentional errors in scanning products also increases. To solve this problem, many chains are turning to computer vision and artificial intelligence, which enable real-time transaction monitoring and analysis.
One of the largest UK retailers of groceries, Sainsbury ‘s, has introduced a new visual AI system at its self-service checkouts, which the media, including The Independent, have dubbed “VAR-style” – like the video assistant referee system in football.
The system operates according to the following method: cameras placed above the self-service checkouts analyze shoppers’ actions. If the product enters the packing zone without proper scanning, the system records the discrepancy.
At the same time, it does not immediately call an employee and does not sound a loud alarm. Instead, a pop-up message appears on the checkout screen: “Looks like that last item didn’t scan. Please check that you scanned it correctly before continuing.” Along with the message, the shopper is shown a video replay of their actions, showing how the item entered the packing area.
This approach can provide a gentle reminder of mistakes, helping to correct accidental errors (for example, one customer noted that the system was triggered because his bag of basil was “too light”), while also deterring deliberate theft attempts.
Agentic commerce
This is a new approach in retail, where AI agents take over some of the user’s business actions, performing them autonomously and with minimal human intervention. Instead of having to search for products, compare prices, and place an order, the user can specify an intent, such as “Find me the best smartwatch under $300 with delivery tomorrow.” The agent will guide the entire process, including comparing options, placing an order, and making a payment.
In retail, agent commerce is used to automate purchases, personalize offers, predict needs, integrate with platforms, and save users time. The agent can independently find products, evaluate prices, and choose the best option, taking into account purchase history, preferences, and budget. Additionally, it orders products when they run out, automatically replenishing stocks. It works through APIs and payment standards of trading platforms, performing actions without additional human intervention, thereby significantly reducing the time for shopping.
OpenAI takes another step in implementing agent commerce by integrating food and grocery delivery services directly into ChatGPT. Consumers will soon be able to order groceries and restaurant delivery by talking to a chatbot. Integration highlights:
- App connection: the user links their delivery account to ChatGPT, allowing the agent to act on their behalf.
- Launch via chat: the agent recognizes the intention to order food or groceries, offers the appropriate service, displays the menu, and shows available options.
- Checkout and payment: while the selection of products takes place in the chat, order completion, payment, and confirmation are performed through the app or the service API.
The role of quality labeling
AI models learn most effectively when trained on a diverse range of examples. The more varied the data, the more situations the model can recognize and process correctly. For instance, in a retail environment, the system encounters thousands of real-world scenarios – shelves arranged in different ways, products under various lighting conditions, packaging designs that change over time, and customers interacting with products.
Companies like Keymakr specialize in creating large, well-annotated datasets tailored to the needs of computer vision models and LLMs used in AI agents. Such datasets include product recognition, product categorization, defect labeling, and other specific tasks that enable AI systems to accurately interpret information from shelves and customer behavior.
In such cases, using poorly annotated data can lead to errors, confusion between similar products, and overall loss of efficiency. That’s why investing in professional data annotation is a strategic decision for retailers striving for stable automation. Only well-labeled and diverse data can ensure:
- The ability of AI to accurately recognize products even under challenging conditions
- The model’s adaptability to changing store environments without the need for full retraining
- Improved the capability of the system to detect anomalies
- Reduced the number of false detections and streamlined the automation of shelf monitoring, stock management, and price updates
Looking ahead, computer vision will continue to evolve, and AI agents will play an increasingly significant role in daily retail operations. Multimodal data, which combines text, images, and video, will become increasingly important. Companies that understand how to work with these new data types and technologies will be better prepared for the next wave of smart retail innovation.
