IBM introduces app that brings augmented reality to brick and mortar shopping experience

by Ron on March 26, 2013

At the recent CeBIT trade fair in Hannover Germany, IBM introduced a prototype app that offers shoppers a way to get access an abundant supply of information about products on a shelf simply by pointing their smartphone camera at the merchandize.

The system, which IBM is calling Augmented Reality Shopping Assistant, combines the smartphone camera and hardware features, augmented reality technology and data derived from a variety of sources. For instance, if you point the camera at a shelf full of different cereals you might see nutritional information derived from the products themselves or government sources of nutritional data.

The IBM app superimposes information about products on the shelf over what you're seeing through the smartphone camera The IBM app superimposes information about products on the shelf over what you’re seeing through the smartphone camera


While the demo used food products on a shelf, it could apply to any products.

In this way, the shopper could find exactly the products they want very quickly from many choices with the kinds of attributes they are looking for such as how many calories per serving or how much sugar the product has.

The idea said Amnon Ribak from IBM Research in Haifa, Israel, who demoed the application for journalists at the trade fair, is to use the smartphone to provide a similar experience users could get shopping online with access to the world of information they find on the Internet when searching for products, but within a brick and mortar store experience, giving the shopper the best of both worlds.

You can see a range of information about each product. You can see a range of information about each product.


And this could have tremendous value for midmarket retailers trying to find ways to lure people into the store when online offers such a rich experience.

Shoppers can see the physical products, but also interact with a variety of information about the products through the phone. Imagine, for instance, pointing the phone at a shelf full of food items and finding which is fair trade, eco-friendly or fat-free.

As Wayne Rash (who was on the press tour with me) wrote on eWeek, “The [app] works by actually recognizing the package. There’s no need to scan a bar code on the box. The app can also integrate with social networks to see user reviews of the product or locate more detailed information,” Rash wrote.

Stores like Best Buy have tried to provide more information about a product by using a QR Code. If you have a QR code reader app on your smartphone, you can point it at the code and go to the to get more information about the product.

But this is not nearly as elegant as the IBM solution. Instead of pointing at a code, the user points the smartphone at the shelf, and it processes the information and superimposes information about the products they’re seeing.

While this app is still in the prototype stage, if it were to come to market, it could give consumers the a great combination of online and brick-and-mortar store shopping. And it could provide a way midmarket businesses to get shoppers into the stores where they can sell directly to them.

Note: This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I’ve been compensated to contribute to this program, but the opinions expressed in this post are my own and don’t necessarily represent IBM’s positions, strategies or opinions.

web counter

Leave a Comment