Despite the lucrative potential of data analytics, many companies struggle to produce a return on an initial investment. Why? Because of the challenge of getting all the needed components working in concert. After more than two decades helping clients deploy successful analytics solutions Elder Research has learned that five technology components are essential to applying analytics.
Each component builds on the others so we have organized them into the Analytics Value Pyramid and will review each of them from the bottom up.
The base layer of IT infrastructure includes virtual machines, network bandwidth, disk drives, and processing power. These components are necessary but not often discussed in the context of analytics success. We have seen multiple projects stall, however, due to insufficient resources for databases and processing. One of the major stumbling blocks is that analytics does not fit within the usual IT paradigms leading to potentially contentious interaction when the data science team asks for processing power or disk space that is beyond the usual IT baseline requirements. Any substantial increase in resources needed for analytics can lead to conflicts between analytics and IT teams.
Relational databases, “Big Data”, and Data Warehousing are topics that fall within the data storage layer. Data storage resource needs can also lead to conflict between business units. Because analytics requires historical data, IT data loads usually grow after starting an analytics project. This can lead to significant space constraints on preconfigured storage area networks (SANs) or database servers. “Big Data” storage solutions such as Hadoop, Spark, or MongoDB are popular, but are not one size fits all. We find that massively parallel Big Data solutions are only worthwhile if your data is changing rapidly or exceeds tens of terabytes of data.
The data transformation layer is concerned with the ability (and software required) to cross-walk data from different systems, clean data, and prepare data for analytics. Data transformation is necessary because data are stored in a format convenient for storage and retrieval, not one that is most efficient for analytics. Most analytic applications require combining multiple tables and databases into a single “analytic base table” or ABT. It cannot be pre-configured as its exact content and desired format depends on the problem and chosen algorithmic approach.
Specialized software is needed for the most effective analytics. Excel is an extremely flexible tool but lacks the most powerful data mining algorithms. There are many options ranging from free open-source tools such as R or Python, to enterprise solutions such as those from SAS or IBM. Although the software vendors will attempt to convince you otherwise, there is no “one-size fits all” software tool. The variety and quality of tools available today means there are useful tools for every budget and IT situation.
Visualization and Delivery
Visualization and delivery is the “action platform” for the users of the analytics solution. This consists of software, reports and data products that communicate the results of analytics in a consumer friendly manner. As analytics permeates the organization, the value of visualization and discovery increases. The most successful organizations even enable non-technical, non-data-savvy users to access and apply complex analytic results to their work. Visualization and delivery bridges the gap between the technical staff and the operational staff. Many early adopters focus too much on the models themselves, and ignore visualization and delivery; avoid this common mistake to deliver increased value.
The Keys to Success
In addition to understanding and implementing these critical types of technology, sustained success from analytics requires a complete cultural shift within a business to institutionalize analytics-based decision management. Elder Research has found that having an “analytics culture” is one of the strongest indicators of future analytics success. Using an analytics assessment to benchmark an organization’s current analytic activities relative to industry best practices can identify opportunities to better leverage analytics, and helps prioritize data-centric business goals to maximize return on investment. The results of the assessment can then inform the process of developing a successful analytics strategy and tactical roadmap.
We help our clients craft an internal analytics culture that delivers value long after our first consultation, and we serve as a partner to help our customers find the right match between the available data science technologies and their specific business challenges.