Blog

Extract Meaningful Data from Software Usage Logs

Anna Godwin

June 2, 2017

BLOG_Extract-Meaningful-Data-From-Nonstandard-Software-Logs.jpgSoftware usage logs are a valuable data source that can reveal insights into customer experience, user configurations, workflows, and software stability. Parsing meaningful attributes from log data is a critical step to understanding user behavior and driving product improvements. However, log data comes in many formats, some of which are far from standardized. Here, we review nonstandard log formats and how log analytics can extract meaningful product usage information.

Can I Use an Out-Of-The-Box Solution?

Many organizations hope to employ an out-of-the-box solution for log parsing. These products are worth considering if your logs are in a standardized format with time stamped lines. For example, Logstash works well with standard logs produced by Java, JavaScript, PHP, Ruby, Python, and Windows.

If your logs don't meet the baseline requirements of a plug-and-play product, Elder Research can provide an efficient, but tailored solution. We have experience addressing the following challenges when parsing nonstandard logs:

  • Absence of timestamps
  • Pseudo XML-like tagging
  • Complex unicode character strings
  • Excessive line redundancy in a single log
  • Absence of logging specifications for standardization

Elder Research has overcome these challenges and more to deploy effective log analytics solutions for our clients. In cases where timestamps are not available, we extract and maintain a sequence of logged events data to support temporal and workflow analysis. Where clear tagging is not available, we work with the client to define criteria to the desired data. Unicode strings as well as international characters are cleaned and codified into a useful format for use in predictive models.

Use Case Driven Solutions

Analyzing log data provides powerful insight into user behavior that can improve software design, enhance user experience, and improve sales outcomes. Our analyses have allowed clients to:

  • Explore usage behavior to inform software design and accelerate adoption rates
  • Understand product stability coupled with crash data to improve future releases
  • Leverage past customer sales data to prioritize current leads

Elder Research works with our clients to understand the development, history, and intent of their logging systems, and the business use cases that will benefit from analyzing their data.  Our Product Usage Analytics platform is a powerful combination of proprietary tools and consulting services employed to extract valuable data from software usage logs. The platform utilizes a robust ETL process to parse, clean, and store log data to develop models optimized for the client’s environment.

Software Usage Analytics Platfom Architecture.png

Our technology agnostic solutions work within existing client architectural frameworks, taking into consideration compute speed, available memory, and model accuracy.  The end result is a parsing pipeline built to meet business needs and constraints.

Elder Research provides tailored solutions for each business case — whether the complete platform or individual service — based on project budget and business goals. What sets us apart is our flexibility to provide exactly the services necessary to drive product and customer service excellence for your organization.

Elder Research has deep experience fielding applications — from product user behavior analysis to improving user experience — impacted by the issues described here.  Contact us for a consultation.


Related

Download our Product Usage Analytics brochure.

Download the case study Using Product Usage Analytics to Improve User Experience


About the Author

Anna Godwin Data Scientist Anna Godwin enjoys working closely with customers to assess needs and deliver custom analytic results to generate business value. She values teaming and brainstorming with clients and colleagues. Prior to joining the Elder Research Commercial Data Science team, Anna worked as an engineer in medical device product development, analyzing controlled experiments on production processes to drive decisions in research, development, and manufacturing capacities. A North Carolina native and Wolfpack fan, Anna holds three degrees from NC State: an MS in Analytics, BS in Biomedical Engineering and a BS in Textile Engineering.