How Big Data and Business Intelligence Are Changing the Manufacturing Industry

December 1, 2014 Ben Kerschberg


Manufacturing has never been sexy. However, the United States has the world’s largest manufacturing sector, with a consistent 20% international market share since the 1980s. One in six American jobs is still directly or indirectly tied to manufacturing, a result of the sector’s tremendous multiplier effect. The effect is especially high in sectors such as high-tech, telecommunications, wholesale and retail, finance, and accounting. [1] Each of these related jobs and sectors depends on a strong manufacturing base.

According to the National Association of Manufacturers, U.S. manufacturing relies on a far more educated workforce and pays higher wages and better benefits than other sectors in the U.S. economy. United States manufacturing alone would comprise the eighth largest economy in the world. In current dollars, manufacturing GDP makes up over 12% of the United States’ total GDP. [2]

Even these stats should not leave us with the impression that everything is rosy in the manufacturing industry. It isn’t. Numerous factors challenge its health. Domestic pain points include corporate tax rates, rising healthcare costs, regulatory compliance, and energy prices. Those costs add a stunning 18% to manufacturers’ costs relative to our major trading partners.

The question then becomes: How do we fix potholes in the manufacturing industry? The use of analytics and big data is a critical answer.

The Need For Big Data-Spurred Business Intelligenceshutterstock_154614725

Among the manufacturing industry’s most forward-looking initiatives has been the application of analytics against large pools of data in order to infer actionable business intelligence in real time. This data can be captured, communicated, aggregated, stored, and analyzed to find hidden patterns and insights that, if unaddressed, would otherwise result in inefficiencies.

Zettabytes (10 to the power of 21 bytes) of information are now created annually. Most of the world’s data was produced in the past several years and is unstructured in nature — ubiquitous data that cannot be organized in neat rows and columns and is difficult to leverage without the right analytics tools. Very little of that data (0.5% worldwide) has been analyzed in any way.

It is now clear that “[l]arge-scale data gathering and analytics are quickly becoming a new frontier of competitive differentiation.” [3]

According to the McKinsey Global Institute:

The use of Big Data is becoming a key way for leading companies to outperform their peers. Across sectors, we expect to see value accruing to leading users of Big Data at the expense of laggards, a trend for which the emerging evidence is growing stronger. Forward-thinking leaders can begin to aggressively build their organizations’ Big Data capabilities. This effort will take time, but the impact of developing a superior capacity to take advantage of Big Data will confer enhanced competitive advantage over the long term and is therefore well worth the investment to create this capability. But the converse is also true. In a Big Data world, a competitor that fails to sufficiently develop its capabilities will be left behind. [4]

One cannot overstate the value of investing in infrastructure focused on extracting business intelligence from data. This includes analytics and cloud infrastructure to do massive data storage demands.

It is not easy to identify the “correct” data to analyze. Even technologically savvy organizations have important data that resides in antiquated legacy systems not integrated into its whole. Much information is siloed and resides in isolated datasets with different custodians or in enterprise resource planning (ERP) tools. Yet without significant investment, manufacturers will miss business-critical opportunities to leveraging one of their most important corporate assets: information (data). The next section examines these opportunities in detail.

The manufacturing industry, like others, must (i) find hidden data—and ideally real time data; (ii) apply advanced computational methods to extract that data’s secrets; (iii) allow its big data and analytics teams to find the business intelligence they can use, and then (iv) optimize traditional manufacturing processes accordingly.

Mapping the process doesn’t make it easy. According to Gartner:

The quality of the[se] outcomes is strongly linked to the quality of the data being collected and the manner in which it is stored, analyzed, visualized, and converted into meaningful and valuable sustainable business intelligence.

The ability to perform the processes named by Gartner depends on three variables:

  1. Data transparency that allows data from different manufacturing functions to be integrated.
  2. Process visibility that allows managers to see how processes unfold as they happen, which allows for real-time adjustments.
  3. Data visualization. As analytics are applied to big data, the output must be analyzed mathematically and represented visually so as to allow end users such as plant managers to actually see the hidden data and its value. This is especially important in light of the fact that the data is dynamic in real time.

Manufacturers with better information realize continuous improvement that allows them to focus their efforts on return on investment. These improvements take many forms, including:

  1. Margin Recovery. Savings are regularly found in reduced materiel costs and system capacity recovery—for example, yields in packaging.
  2. Product Quality and Safety. Teams are better equipped to understand and eliminate the true root causes of product risk. They can thereby actually address and fix these shortcomings at the start of processes rather than merely discarding low-quality output based on post-production testing. Product quarantines can be established based on specific, objective data rather than subjective approximations.
  3. Eliminating overlapping investment and personnel support. Identifying overlap should result in the proper allocation of resources—both people and technology. The human element of the manufacturing processes cannot be decoupled from even the most advanced analytics. On the contrary, is essential to BI, as discussed below.
  4. Measurable ROI. Moving from the sort of traditionally unstructured approach that characterized previous manufacturing reporting processes to a modern solution built for data collection and analysis enables management to develop consistent, new methods that can result in greater efficiencies, yields, and production flexibility. Such gains allow manufacturers to take new products to market up to considerably faster than before, a remarkable achievement with significant effects on profits and losses.
  5. Collaboration. The synergy of BI and big data provides both macro and granular views of information that allows management, operators, and engineers to work together based on quick feedback in a data-driven environment. This is not the siloed manufacturing industry of the past.
  6. Monetizing Assets. Corporate attitudes with respect to monetizing corporate assets such as intellectual property have changed dramatically over the past five years. In manufacturing, this move towards using BI to mine big data has shifted the view of those assets and the systems that generate them as profit-enabling centers rather than just insurance and a cost of doing business. This is a profound change with significant effects on an enterprise’s bottom line.

The Relationship Between Human and Business Intelligence

shutterstock_115846333The relationship between technology-assisted processes and the people needed to make them succeed has spurred vigorous debate in numerous businesses. Let’s cut to the chase. No matter how advanced technology may be—in this case highly advanced mathematical algorithms that produce business intelligence—it is a costly mistake to divorce the processes that it drives from the specialized, tacit human knowledge and expertise that exists in any enterprise.

Decoupling the two is a serious step in the wrong direction. This is not to say that relying on humans alone to guide manufacturing processes suffices. Likewise, analytics alone cannot be the sole catalyst of this industrial knowledge. Just as corporations have discovered hidden, siloed data, so too must they tap into and then incorporate into their BI the extraordinary amount of tacit knowledge that resides (often hidden) in their workforce.

To some degree you look as much as possible to leverage data that is coming out of equipment or sensors—data that is inherent to the process. This is the basis for BI that should guide process improvements and standards. But the way that people interact with that process tells you a lot about what they know and believe to be true. For example, if a skilled, experienced operator overrides temperature settings in a cooking process or who shortens a cycle time, you have to ask why. The key is to be able to sit down with those operators to understand why they’re making the choices that they’re making, to capture that (tacit) knowledge, and then to combine their human expertise with the underlying objective data in order to create new operating procedures and processes. [5]

Capturing this knowledge is even more important when one considers natural employee attrition and retirement. Expertise that walks out the door cannot easily (if ever) be recovered and can significantly affect operations and business in the long term.

Expertise gained through BI must be “socialized” among managers, engineers, and other workers so that it may be transferred to manufacturing processes. In this respect, one must consider the types of workers one finds in manufacturing today to dispel the common visual that people have of highly repetitive, task-oriented types of jobs and replace it with the reality of manufacturing’s high level, analytical problem solvers.

Manufacturing operators today are not the stereotypical industrial types anymore, and working in a modern factory is intellectually engaging, not just rote work. It’s not a job one does because one has no other choice in life. Modern manufacturing attracts workers and analysts who might otherwise get a graduate degree in math but find it just as rewarding to be part of a team that’s doing process analysis and continuous improvements, whether that relates to redesigning transmission lines for a major car manufacturer or making baby formula in a food plant.

Big Data’s Return on Investment

Analyzing Big Data for business intelligence assets has a measurable return on investment (“ROI”). Moving from traditionally unstructured approaches often found in older processes to a modern structured solution designed for data collection and analysis to empower enterprises can result in 14% higher production yields. Software empowers teams of operators, engineers, and management to work collaboratively and identify areas of improvement in a data-driven culture predicated on quick and accurate feedback. A 14 percent (14%) ROI gives companies competitive advantages that were not possible before when one considers how those savings can be used:

  1. launching products more quickly;
  2. reinvestment in the company’s workers and product lines;
  3. job-promoting expansion;
  4. as a buffer against price pressures from competition and other market factors; and
  5. proper capital allocation based on new business plans and process strategies.

Perhaps most interesting, newly found efficiencies can help companies on the supply side both accelerate their speed to market and provide more accurate data to their buyers, which results in improved customer loyalty and credibility within their supply chain. A large portion of this stems from analyzing big data.


The manufacturing sector has come far in its use of data and analytics, but not far enough. MIT Professor Eric Byrnjolfsson wrote only this month: “Too many managers are not opening their eyes to this opportunity and understanding what Big Data can do to change the way they compete.” [6] Manufacturers who do not have a long-term holistic vision will be at a significant disadvantage vis-a-vis their competition. They will find it necessary to keep many jobs abroad rather than bring them back to local markets and more highly skilled workforces. Critical information will remain hidden, making it difficult if not impossible to drive improvements or increase production without adding equipment or people. They will become seen as less reliable supply chain partners. The parade of very real horribles goes on — at the end of the day, they will lag far behind their competition.

data_intelligence (1)

End Notes

[1] See Ross DeVol & Perry Wong, Jobs For America: Investments and Policies for Economic Growth and Competitiveness (Milken Institute Jan. 2010).

[2] Id.

[3] See Jacques Bughin, John Livingston & Sam Marwaha, Seizing The Potential of Big Data (McKinsey Quarterly Oct. 2011).

[4] Big Data: The Next Frontier of Innovation, Competition, and Productivity (McKinsey Global Institute May 2011).

[5] See The U.S. Sustainable Business Market: Spending Patterns in (IT) Solutions and Architectures Currently Supporting Sustainability and GHG Management by Manufacturers (Gartner Group Report. No. G00213092 Aug. 30, 2011).

[6] Eric Byrnjolfsson, Jeff Hammerbacher & Brad Stevens, Competing Through Data: Three Experts Offer Their Game Plans (McKinsey Global Institute Oct. 2011). (Brad Stevens is the coach of the Boston Celtics.)

Previous Article
Reinventing Strategic HR around Service Delivery
Reinventing Strategic HR around Service Delivery

Even in the best of times, the place of HR in many organizations is never quite assured. To address the per...

Next Article
5 Rookie Mistakes in Marketing Automation
5 Rookie Mistakes in Marketing Automation

By Jane Gibson The licenses are purchased, and the Marketing team is excited. Ready to run with a new autom...