We live in the digital age. Some reports (8) indicate the average American adult spends half of their waking hours in front of a digital screen. In the workplace, most business processes are or will be transitioning to digital systems. The original motivation for digitization was to increase the efficiency and accuracy of existing processes over those of paper-based systems. As the volume, variety, velocity, and veracity of digital data grew, savvy entrepreneurs invented new ways to create business value. The growth in data availability is now fueling a wave of interest in artificial intelligence (AI) from companies seeking to turn their data into a competitive advantage. In a recent cross-industry survey (6), 47% of business executives reported that their companies had embedded at least one AI capability in their business processes, and 30% were piloting AI initiatives. AI is becoming a practical reality in a growing number of fields, touching ever more dimensions of daily life, from smart speakers in the home to driver-assistance–enabled cars on roads. Some tasks, such as handwriting recognition, are routinely accomplished using AI tools. Others, like designing new flavor experiences, are pushing the frontiers of AI research.
This article presents evidence of the potential for AI—in particular an interdisciplinary subfield related to AI called computational creativity—to assist human product developers in creating new flavor and food products. Our experience is based on a multiyear collaboration between McCormick & Company Research and Development (R&D) and IBM Research. The first publicly available results of our efforts in computational creativity are three new AI-enabled McCormick products that are available in online markets and brick-and-mortar stores (1,5).
Future of Flavor Product Development and Innovation
Flavor plays a critical role in the success of a food product in the market. Product developers at McCormick have thousands of ingredients from which to choose in developing flavor formulations. For example, there may be dozens of garlic powders available, varying by particle size, country of origin, drying process, and flavor profile. Furthermore, the set of available ingredients is changing and expanding, with new ingredients being introduced in the market at a rapid pace. Selecting the right garlic or combination of garlic ingredients for a formulation in the right ratios taxes a human developer’s ability to remember each of the individual characteristics of all available ingredients and consider all possible combinations and their interactions.
In the coming decade, the food and beverage industries will face seismic changes. The unprecedented growth of e-commerce already has disrupted the entire retail supply chain from end to end. In U.S. markets, the new generation of consumers, led by Generation Z and millennials, are encouraging consumer packaged goods companies to expand their focus on ingredient characteristics related to transparency, natural, organic, genetically modified organisms (GMOs), sustainability, and social responsibility. Because of these demands, flavor product developers face a large and ever-growing set of objectives and requirements as they select which ingredients to use when formulating new products.
Designing new flavor experiences is both an art and a science. It requires many years for a person to become proficient. Product developers gain expertise over years of hands-on experimentation, iteratively creating candidate formulas, manufacturing samples, conducting laboratory and consumer tests on the samples, and learning from the results. Flavor formulation is a resource- and time-intensive process. Yet, speed is of the essence in the demanding and competitive environment of product development.
Development of iconic products, such as the Oreo cookie (Nabisco) or Philadelphia cream cheese (Kraft), is an optimal outcome; however, the trend in the consumer packaged goods industry has been toward faster product turnover. A majority of the new food and flavor products launched will disappear from store shelves in less than 5 years. Online shelf space, however, is virtually unlimited, removing the traditional limitations on the number of products that can be offered. Products offered online can be introduced faster and with smaller initial manufacturing runs. Because most products do not survive in the market very long, they need to be replaced frequently. This increases the demand on product developers to create better products faster.
AI, Machine Learning, and Computational Creativity
The terms AI, machine learning, and computational creativity are defined in different ways by different authors. We include our definitions here for clarity and completeness.
The term “artificial intelligence” refers to the capability of a machine to imitate the natural intelligence of human behavior, such as visual perception, learning, reasoning, understanding natural language, and problem solving. The concept of AI is not new. The idea of creating an artificial brain can be traced back in the literature to ancient Greek mythology. However, it was not until 1956, after the advent of modern computing, that the field of AI was established as an academic discipline. The explosion of current interest in AI stems from the confluence of three trends: the availability of massive amounts of data; increased computing power, in particular the availability of graphical processing units; and advancements in algorithms (2).
While AI encompasses the broad science of mimicking human abilities, “machine learning” is a specific subset of AI that trains a machine how to learn (7). More specifically, machine learning is a subfield of AI centered on the study of the algorithms and statistical models that computer systems use to perform a specific task effectively without explicit instructions, relying on patterns and inference instead (10). The term was coined by Arthur Samuel, a computer pioneer at IBM. In 1959, Samuel created a checkers-playing program that is widely considered the world’s first self-learning computer program (9).
Machine learning has been applied successfully to a wide variety of tasks and fields. Examples of everyday consumer applications of machine learning include movie recommendations based on viewing patterns made by streaming services, such as Netflix; targeted advertisements shown by search engines, such as Google, based on web browsing history; and personalized playlists, such as Discover Weekly, offered by the music streaming service Spotify. In industry, machine learning is used by financial services companies to prevent fraud, by manufacturing companies in robot-driven assembly lines, by health care companies to detect chronic disease, and by agriculture companies to determine sowing dates, to name just a few. In the food and beverage industries, the use of AI is becoming increasingly attractive. Massive amounts of data, such as data accumulated in sensory and consumer testing, is now available and accessible. In fact, a session dedicated to AI was held at the 2019 annual meeting of the Institute of Food Technologists in New Orleans, LA.
The Association for Computational Creativity (http://computationalcreativity.net/home) defines computational creativity as a multidisciplinary endeavor that is located at the intersection of the fields of AI, cognitive psychology, philosophy, and the arts. Like intelligence, creativity is innately understood but is challenging to define rigorously. Although academic research continues to expand the definition of what it means to be creative, the basic tenets are novelty, differentiation, surprise, and usefulness. McCormick’s efforts in computational creativity are focused on one central question. Can computers help human product developers be more creative?
McCormick’s vision for bringing AI to its organization was threefold: 1) elevate the capability and impact of every member of the more than 500 member team to the level of the best member; 2) accelerate the rate of innovation by at least three times; and 3) significantly enhance new product staying power in the market. The vision of how “big data” can be harnessed to deliver a paradigm shift in innovation capability is illustrated in Figure 1. Simply stated, the goal is to usher in a new chapter in food and flavor innovation.
After studying the leading companies in the field of AI, McCormick chose IBM Research as its partner. IBM Research’s decades-long leadership in AI systems development, its cognitive solution technology, and its pioneering work in the development of AI technologies in the areas of natural language understanding, machine learning, and deep learning made it the natural choice for the collaborative project.
A Computational Creativity System to Assist Product Developers
McCormick and IBM Research launched an initiative in early 2013 with the objective of assessing the potential for the use of AI in creating new flavors and foods by combining more than 40 years of sensory science and taste data from McCormick R&D with the latest cutting-edge AI technology and capability from IBM Research.
Utilizing AI technology was a good fit for designing new flavor experiences at McCormick because of the wealth of available operational and research data accumulated over decades, including data on historical flavor formulas, raw material components, experimental results, consumer testing results, recipes, market success data, and more. When expanded globally by 2021, the system will contain more than 1 billion data points, 500,000 formulations, and hundreds of millions of external data points.
The historical data, however, was stored mostly in silos and legacy systems scattered around the world. Worse, it was stored in a variety of formats and languages and rarely shared. The relationship and significance of various data sources, such as sensory versus culinary versus chemistry, were unclear.
Building on previous experiences using AI to pair flavors, as well as proprietary IBM Research AI for Product Composition (3), IBM Research created a system that uses new and advanced machine-learning algorithms to sift through McCormick’s data and identify patterns and novel combinations that fit specific design objectives.
As illustrated in Figure 2, this AI system is designed to learn and predict the following attributes:
- Possible alternative raw material complements and substitutes for a formula
- Appropriate ratios of raw materials (based on usage patterns)
- Market success (based on consumer liking)
- Novelty of system-generated flavor formulas, as measured by a derived distance function in the space of possible flavors
- A definition of what it means to be a particular product (e.g., a barbecue sauce versus a marinade)
At the start, the challenges seemed insurmountable. A computer cannot taste or smell. Creating a flavor experience involves the function of at least three senses: taste, smell, and sight. Consumer preference for a flavor or taste is the outcome of numerous inputs, including emotional, physical, and spiritual experiences, some of which go back to early childhood. Additionally, the physiology of taste receptors on the human tongue is very complex. Our understanding of how they work and where they exist continues to expand (4). There is also a genetic component that helps explain why some people experience certain food items as delightful, while others experience the same food as unpleasant. The flavor experience domain also is highly complex, and our knowledge of it is still evolving.
The AI being developed by McCormick and IBM is a learning system, not a deterministic software program. It incorporates feedback from developers into its next suggestion. Over time, with more data and proper and organized feedback, it will improve and be better able to predict human liking and market success.
To be a useful tool in the daily workflow, the AI system needs to provide various levels of creativity to support product developers. In some scenarios, the goal is to optimize a flavor formula by tweaking it to perfection, such as optimizing the sensory profile of a vanilla flavor. In other scenarios, the goal may be to develop a truly novel flavor experience. Here, the variation or distance between ingredients (e.g., vanilla versus strawberry) is likely more significant than the choice within any one ingredient family (e.g., which vanilla to use). The AI system learns and uses a distance model to suggest desirable flavor formulas.
Because AI is data driven, the insights it delivers evolve as the data set changes. Product developers have an overwhelmingly large number of combinations and proportions of ingredients from which to choose. As they gain more experience, they develop a suite of favorite flavors that they have learned work well in their formulations. At the same time, ingredient companies are launching new and exciting ingredients (including flavors) every day. For example, there are thousands of dairy or cheese flavors available globally today. Most developers have a go-to solution of a select number of dairy flavors they know and trust. They may develop a practice of testing these first in their formulations. This is not the case with AI. Having an AI system that can expeditiously and intelligently explore all options helps developers avoid using habitual standbys when evidence suggests better alternatives may exist. Humans rely on their personal experiential success or failure to determine the path forward. AI relies on the experiential success and failure of all developers, providing a much broader prospective.
Lessons Learned
One take-away from the application of computational creativity to product design is the importance of data. Machine learning requires massive amounts of data to be successful. Collecting and curating vast quantities of data require significant effort. We were fortunate that McCormick had the foresight to store more than 40 years of product development data. That said, getting the data entered into the computational creativity system required considerable effort. Data from multiple systems needed to be merged, and some data that was saved only on paper needed to be scanned into the system and transcribed. Standardizing data capture across organizational boundaries and capturing as much data as possible will improve future results and create other applications for AI.
The central goal of the AI project was to help product developers create better products faster. However, product developers need to see the value in the AI system and want to use it in order for the company to derive value from it. Involving developers from the start of the project was key. Developers want tools to help them work more efficiently and effectively and to deal with the challenges they face. However, we learned that experienced developers were able to provide valuable directional objectives for building the system—how to interpret data, what data to trust, how to evaluate the results—and provide feedback on how the system could be improved.
Applying AI to product design is still a research endeavor. Although our initial success is promising, use of computational creativity for product design is still largely unexplored. Progress takes time, and there will be more challenges, as well as successes, ahead. Applying computational creativity to product design is an investment in the future that has great potential, but also significant risk.
The Future
All that is visible today is the proverbial tip of the iceberg. In the future, AI will be a ubiquitous and seamless technology used in new product development in leading flavor innovation labs. A company’s access to AI and its ability to shape and harness it for competitive advantage will determine who is on the new product playing field and, to a large extent, who will win. Today, we would not think of developing products using beam balances and slide rules. In the future, it will be very difficult to attract top technical talent to an R&D organization that is using traditional iterative processes, such as Stage-Gate (Stage-Gate International), rather than AI. Technology innovation using AI is revolutionizing product innovation, offering new capabilities and workflow processes that will create a seismic change in the speed and effectiveness of product development.
As Chief Science Officer Hamed Faridi, Ph.D., heads a global team of more than 500 scientists and flavorists responsible for thought leadership, scientific research, product development, health and wellness, and sustainability to support the global businesses of the world’s largest flavor company—McCormick and Company, Inc. Hamed is the voice of science for McCormick. His vision led to the formation in 2006 of the McCormick Science Institute (MSI), a research-driven organization working to advance scientific understanding of the potential health benefits of culinary spices and herbs. Hamed has published six books and dozens of scientific articles on food science and technology. He is an AACCI past president (1995–1996) and was elected an AACCI Fellow (2011). He has received numerous awards, including the 2016 Ellis Island Medal of Honor, given to “Americans in recognition for their inspired service to the nation,” and the 2017 Upwardly Global’s Outstanding Leadership Award, which recognizes individuals “who have advanced the promise of an interdependent world and global economy by championing immigrants and refugees.” Hamed received his M.S. (1973) and Ph.D. (1975) degrees from Kansas State University, and an MBA from Fairleigh Dickinson University (1987). He is a Certified Food Scientist (CFS).
Richard Goodwin, Ph.D., is a distinguished research staff member and manager of the Computational Creativity Research group at the IBM T.J. Watson Research Center in Yorktown Heights, NY. The group’s research focuses on applying AI for product formulation. His research interests include machine learning, deep neural nets, computational creativity, intelligent agents for planning, scheduling, and decision support. Richard holds a B.S. degree in electrical engineering from the University of Waterloo and M.S. and Ph.D. degrees in artificial intelligence from Carnegie Mellon University. He was awarded the Wagner Prize in operations research for work on agent-based scheduling and received the Decision Science Paper of the Year Award for work on applying option theory to project planning. Richard holds 28 U.S. patents and has authored more than 45 research publications.
Robin Lougee, Ph.D., is the IBM Research lead for Consumer Products & Agriculture. She is a senior industrial research scientist with a strong track record of delivering innovation to IBM and its customers. Robin pioneered the creation of COIN-OR, an open-source foundry for computational operations research, and led its growth to an independent nonprofit that has served the scientific and business communities for more than 15 years. She was elected to the board of INFORMS, the largest society in the world for professionals in the field of operations research and analytics and served as chair of the INFORMS Computing Society and president of the Forum for Women in OR/MS. Robin was awarded the 2016 INFORMS George E. Kimball Medal and 2014 INFORMS Impact Prize. She is the founding chair of the Syngenta Crop Challenge in Analytics Prize and a co-author of the “Science Breakthroughs to Advance Food and Agriculture Research by 2030” report from the National Academies of Sciences, Engineering and Medicine. She currently serves as the Steering Committee chair of the CGIAR Platform for Big Data in Agriculture and an Advisory Board member for the Cornell University Department of Food Science. Robin earned a Ph.D. degree in mathematical sciences from Clemson University in 1993.
Jim Martin is the director of Global Creative Intelligence for McCormick and Company, Inc. in Hunt Valley, MD. He oversees McCormick R&D’s efforts to utilize computational creativity and machine learning to better understand consumer flavor palettes and to more quickly deliver better tasting products to its customers and consumers. Jim has been with McCormick R&D for 20 years and has more than 30 years of experience in various product development disciplines, including seasonings, coating systems, flavor applications, and condiments.
