It's Watching What You Eat: Machine Vision and the Future of Consumer Products Manufacturing

By: Mark Osborn, VP, Strategy and Business Planning, SAP

Mark Osborn, VP, Strategy and Business Planning, SAP

The cameras are watching. Everywhere. All the time. I’m not referring to the streets and subways of London, Shanghai or New York, though. Rather, this new reality applies to the cameras enabled with machine vision capabilities that are increasingly being placed inside advanced production facilities. They are radically transforming manufacturing processes, while simultaneously boosting productivity, efficiency and quality.

"In process manufacturing, new, advancing machine vision technologies such as hyperspectral imaging can be applied to food products and ingredients for functions like measuring pH, color, tenderness, ripeness and more"

Machine vision refers to the process of using an image to extract information, then leveraging the information to confirm presence or absence, check position or orientation and to spot patterns or exceptions resulting from the analysis of multiple sequential images over time. These capabilities are being applied in a variety of ways to revolutionize manufacturing and production processes.

For example, machine vision can serve as a basis for automation and collaboration with robots deployed for manufacturing processes. In discrete manufacturing, a machine vision-enabled process can provide a visual inspection of materials, parts, and labels from a given bill-of-materials (BOM) prior to assembly. This ensures the proper location and installation of any given part or material, as well as the capability to detect any potential material defects pre-production. Combined with robotics solutions, these machine vision capabilities can alert robots when to initiate an assembly process or when to remove parts from a process in the event of an issue or defect.

As manufacturing processes get closer and closer to consumers, the combination of machine vision and robotics can be applied to increasingly complex and variable make-to-order scenarios that would be impossible in more traditional manufacturing environments relying on economies of scale to be cost effective.

Automated artificial intelligence-enabled software ‘bots’ could “learn” millions, if not billions, of potential combinations of products and parts. It’s not unreasonable to expect them to apply that knowledge to pick components down to the individual order level, present them to the manufacturing process for visual inspection and then, once confirmed and approved, pass them on to an industrial robot for final assembly. This process would occur in seconds, and with no requirement for human intervention. At scale, technologies like these could be applied to produce huge volumes of configurable, and even customizable, products but with accuracy and cost-efficiency levels infinitely greater than what’s been previously possible.

In process manufacturing, new, advancing machine vision technologies such as hyperspectral imaging can be applied to food products

and ingredients for functions like measuring pH, color, tenderness, ripeness and more. Hyperspectral imaging combines computer vision with spectroscopy, a technique that can analyze the chemical makeup of foods and other ingredients from the light of a single pixel. This means that companies can apply the technology to take images of food and other ingredients, and immediately understand chemical composition, moisture levels, nutritional content and more.

These technologies can be applied to move food quality and safety beyond the realm of measuring small representative samples and then applying the results of these tests to the total available lot of a given ingredient or food product. Instead, companies would gain the ability to assess the entire lot for food and other ingredients—every side of beef, every strawberry in the carton, every head of lettuce, every ounce of dye.

In doing so, companies can ensure that effectively all the ingredients and food products they are producing or processing are fresh and help to eliminate waste resulting from over or under-ripe food products. They can also help ensure the safety of all ingredients and food products by checking to confirm that lots are free from foreign objects like plastics, metals or unsafe chemical compounds.

Finally, these technologies can help to eliminate fraud from production processes by helping to confirm that food products that are marked ‘fresh’ have never been frozen, or that the Atlantic cod that’s going in to those fish sticks is, in fact, Atlantic cod—and not a different variety of fish packaged as cod.

Much like the configurability and personalization that machine vision could help enable in discrete manufacturing, it’s within the realm of possibility that similar capabilities might be enabled by machine vision and other advanced technologies for process manufacturing, too. These could, for example, be combined to regulate quantities of specific nutrients added to foods for specific consumer segments, or the amount of dye blended into a cosmetic to produce a color specifically tailored to an individual consumer.

Extending the implications even further, while machine vision is being deployed successfully in manufacturing and production processes, there are also examples of how it is being deployed in complementary processes to improve production planning and optimize output.

For example, beverage and snack companies are placing computer vision cameras inside of coolers in stores and vending machines. These cameras monitor the stock inside the cooler to:

• Ensure merchandising compliance by monitoring to ensure only authorized products are placed in the cooler and that all products are placed properly, alerting the retailer and the consumer products company in the event of any exceptions.

• Monitor consumer demand at the individual cooler level, taking periodic images of available stock to measure the velocity of consumer demand for the products in the cooler.

Machine learning capabilities applied to this data help companies anticipate and mitigate compliance risk before issues occur, and fine-tune replenishment and the variety of products inside of a given cooler down to an individual level. In turn, this helps to maximize future sales based on actual consumer demand. Armed with this data, companies can update short-term forecasts and manufacturing production capacity plans in real-time, helping to ensure that capacity is balanced with demand and that production output is optimized based on predictions of future demand—thus maximizing consumer sell-through while simultaneously eliminating waste from over-production and spoilage.

These are just a few of many examples of machine vision application, but each represents a bright future for the technology, and the manufacturing processes it’s helping to transform.