Playing with Fire: The Sensor Analytics Explosion

Will Goodrum

[fa icon="calendar"] March 31, 2017


Recent articles have touted the boundless value locked in sensor data, if businesses would only strike the Machine Learning match to set it all ablaze. Any organization not thinking seriously about Sensor Analytics is at risk of being burnt by the disruption to structure, competition, and workflows that this nascent technology brings. New entrants will need to ask themselves “What industry are we in?

Data Explosions

A broad spectrum of industries are being wracked by data explosions on a scale previously unseen. The Oil and Gas sector is building up stores of data from tens of thousands of sensors and transducers spread across entire oil fields. Device manufacturers are embedding sensors in their products to track maintenance intervals in real-time, predict faults before they occur, and study product usage patterns. Each such installation creates petabytes of data that must be stored and analyzed to gain actionable insight. No matter your industry, the Internet of Things (IoT) and the Industrial Internet has come, or is coming soon, to you.

Playing with Fire

One of my first cars was a used 1987 Buick Grand National. The car would start, but after idling at a high level, the engine would shut off. As my stepfather and I stood there next to it, scratching our heads, he offered this wisdom:

"There're only three things it could be: air, fuel, or spark. If one of them isn't there, it isn't going to work."

The Buick wasn't running because these three components of air, fuel, and spark were out of balance. We figured out we had a bad fuel pump. Once that was replaced, everything ran as new.

As I look at the data explosions taking place around Sensor Analytics and IoT, I keep coming back to that old Buick and what it takes to make an engine run well; getting value from sensor analytics requires having the right components meet in the right proportions. In our experience, there are three key component challenges to balance to achieve value from analytics:

  • Business Context (Is it the right data?): The data may be big, but are they the right data to address questions of interest or to advance existing strategy? Available data may initially appear to align with important questions, but on further analysis lack the granularity, prevalence, or relevance to the business to generate value. Our Data Discovery consulting service is designed to address this issue.
  • Data Access and Availability (Is there too much or not enough data?): Many industries are ingesting massive volumes of data, but volume alone does not equate with value. Does the company have sufficient systems and infrastructure to store and analyze the data generated? Although there may be more data available than has previously been used by engineers or analysts in these fields, are there enough examples of the problem of interest to be able to derive usable insights? Our agile data science methodology is designed to quickly arrive at insights and iterate toward more refined solutions, based on information we find in your data.
  • Technical Resources (Are the data in the right format?): Sensor data tends to be stored in long log formats as a function of time. Predictive algorithms require that this data be transformed from long logs into transactional tables for analysis. This data transformation requires significant storage space and carefully constructed Extract-Transform-Load (ETL) processes. Elder Research has a dedicated team of software engineers with experience developing customized analytics solutions, from infrastructure to user-interface to get data out of warehouses and into the hands of your analysts.

These are not the only challenges facing companies that are embracing Sensor Analytics, but they are fundamental ones. Handling increased data velocity, and having the right talent and capabilities to analyze these new data are crucial as well. At Elder Research, we have extensive experience assessing these many facets that contribute to analytic success for our clients, across industries.

Applying Sensor Analytics requires commitment to address these challenges, while setting and managing appropriate expectations on the part of leadership. There is value in sensor data but it is not easily harnessed. How can companies succeed at finding it?

Starting a Fire with Sensor Analytics

 Throughout the last two decades, Elder Research has seen industry after industry catch fire with advanced analytics. Starting a fire requires the right mixture of components (fuel, air, and spark), at the right time, to harness the potential energy stored in the fuel. Too little or too much of any one element will throw off the equilibrium required for the reaction to take place. There is significant potential energy to be found in data, as well, if you balance the three components (as shown in Figure 1):

  1. Business Need/Context – is it the right data?
  2. Data Availability/Accessibility – is it enough data?
  3. Technical Resources – are the data in the right format?

Getting value from sensor analytics.png

Figure 1. The three factors that must intersect in the right proportions to achieve business value from Sensor Analytics and IoT initiatives.

Business Need/Context – is it the Right Data?

All analytics initiatives are aimed at discovering knowledge that will lead to right action.  So, they start at the end – at the business context -- in order to be successful. Consider the example of predictive maintenance for medical devices. The business need is clear: replacing parts or returning a device for service to maintain high quality parts in use.  Taking action requires expensive downtime, in addition to the cost of the parts and labor. But, the cost of a device failing in the surgical theater is even higher.   

Even with a clear business need, how to capture value from analytics is not always clear. Just because a model is capable of predicting a maintenance interval does not mean that maintenance scheduling will occur. Are the necessary resources available to respond to the model at the time required? If a model recommends that a device needs maintenance, the hospital must take action to return the device. Just having an accurate estimate of the time until failure is not sufficient to successfully resolve the business problem.

More generally, analytics results must be operationalized in order to make insights accessible to key stakeholders and enable them to take action. But change management is difficult. Are managers prepared to reorganize their teams because an algorithm suggests that there is efficiency to be captured? Are subject-matter experts willing to confront long-held beliefs when sensor data undermines those beliefs?

Elder Research has extensive cross-industry experience that repeatedly shows that starting small and achieving quick wins helps to build momentum and excitement within organizations regarding the real value accessible through advanced analytics. Our Analytics Assessment service helps organizations identify and prioritize opportunities for advanced analytics with a specific focus on actionability and return on investment.

Data Access and Availability – is it Enough Data?

Sensors produce Big Data. In the medical device example above, a lot of the early excitement in this area focused on the sheer volume of sensor data being collected, and the analytic opportunities it might make possible. But volume alone is not enough; big data is not inherently valuable if its information does not align with the business problems or the questions the business is trying to answer.

We have found much of the power of big data has to do with a rich variety of data sources.. Unleashing its power comes from the synthesis of previously disparate, siloed, or neglected sources of data. Unfortunately, there are often intentional or unintentional access constraints that lock data into traditional verticals or strict ownership policies. When data remains inaccessible, its value is severely constrained. With open data access, analysts can fill gaps with data from other internal sources, solving previously insoluble problems.

Sensor Analytics involving IoT is especially dependent on data access and availability. It isn't simply Big Data from one type of sensor, it's the information streaming in from sensors on all manner of components.

For example, physicians may have previously only had data on patients with diabetes from relatively infrequent lab tests. Wearable health devices like FitBits now produce ample information on population activity level. When wearable data is coupled with data from connected glucose level monitors, physicians are now able to see data on both diet and exercise at a much more granular level, leading to a higher quality of care. If patients are failing to follow a prescribed activity plan, physicians can proactively intervene to encourage more healthy living. However, this level of care requires physicians are able to access the activity and monitoring data. If either are tied up in proprietary siloes, the potential value is greatly diminished.

Technical Resources – is it Too Much Data?

Capturing value from sensor data analytics requires an investment in technical resources. Powerful and robust computing hardware must be available to ingest, manage, warehouse, and deliver data for analysis, with sufficient velocity and volume to meet the needs of decision makers.

Where possible, these systems should be automated to minimize the potential for human error in Extract-Transform-Load and data warehousing procedures to convert sensor data into a transactional table before it is fed into a predictive algorithm.

Elder Research helped a large oil and gas firm use sensor data to predict which gas wells would require costly intervention to prevent the wells from freezing. Pressure, temperature, fill level, and flow rate data from thousands of sensors on over 1700 wells had to be summarized and integrated at the appropriate level for predictive modeling. As is often the case with sensor data, the data were large, diverse, noisy, and often erroneous or incomplete. Despite all these challenges of scale, Elder Research developed a model that was 70% accurate at estimating well problems 4-6 months in advance, and capable of saving 600,000 MSCF in annual deferred gas production – a huge win for the client.

Achieving value in such sensor analytics initiatives requires significant hardware, software, domain experience and analytical expertise. Investment in each of these areas, along with the necessary infrastructure and business process changes, substantially increases the likelihood of analytic success, and consequent revenue increases.


The excitement around the potential of the Internet-of-Things to unlock real value through data is pervasive and ubiquitous. More and more industries are facing data explosions as information pours in from sensors in increasingly interconnected devices. However novel the Internet-of-Things may seem, the keys to harnessing the explosive power from advanced Sensor Analytics are tried, true, and tested. Just as with my old Buick, it's a matter of getting the right mixture of necessary components in place at the right time. Across many diverse industries Elder Research has seen that when business need, data availability, and technical resources all converge, the disruptive energy of advanced analytics can fuel real and lasting value and sustainable competitive advantage.

Our services are designed to help you identify real value opportunities in your data. From the comprehensive Analytics Assessment, to the targeted Data Discovery service, and even complete Predictive Model Building, we work alongside you, helping you to unleash the potential value in your data warehouses.

Request a consultation to find out more about our services and experience in sensor analytics.


Learn more about Sensor Analytics

Download the case study Using Sensor Analytics to Predict Natural Gas Well Freezing

About the Author

Will Goodrum Data Scientist Will Goodrum has a decade of experience applying numerical analysis and engineering to solve practical problems and generate value for customers. Previously, Dr. Goodrum worked in an engineering software firm, helping medium-to-large scale customers across industrial sectors develop superior products and reduce their time-to-market. As a graduate student, he applied statistical modeling and physics-based simulation to estimate the impact of policy decisions on lifetime maintenance costs for a regional transportation authority. Will holds a B.S. in Mechanical Engineering from the University of Virginia, and a PhD in Engineering from Cambridge University.