Automating Network Entity Detection

Author:

Ryan McGibony

Date Published:
April 5, 2019

Elder Research developed an automated data pipeline to cleanse data and feed a data visualization tool used to identify and explore document preparer network relationships. The solution enabled the client to automate significant portions of work, make data-driven decisions, prioritize resources, and gain new business value from the data.

The Challenge

Elder Research was tasked to identify networks of document preparers who worked together in a given year. Documents with particular information in common indicated a possible network connection. The goals were to enhance the client’s capabilities in two ways:

Improve the efficiency of finding documents with incorrect preparer identification and re-assigning them to the correct preparer identification when possible.
Enable the analysts to consider preparer networks instead of only individual preparers to better deploy investigative resources. The

Solution

The project required several interrelated stages of data analysis using multiple data sources and formats. Since it was common for the documents to have typographical and other errors preparers could appear to be linked when they did not actually work together. Elder Research used extensive data validation procedures to account for missing data and ensure that the documents identified the proper preparer.

For the preliminary phase of network identification Elder Research quantified the strength of the relationship between each pair of document preparers. The idea behind this technique came via Elder Research’s extensive experience with text mining, as exemplified by the award-winning book, Practical Text Mining, co-authored by Dr. John Elder and five others (Elsevier, 2012).

Further phases of network analysis techniques were used to sort through the web of connections and boil down the links to the most likely networks of preparers who worked together, as shown in the example below.

To make it easy for end-users to search and explore preparer relationships Elder Research deployed its proprietary browser-based network visualization tool. This enabled the client to interactively explore and visualize relationships among preparers and to also explore preparer connections based on the raw document data (without the pre-processing analytics).

Results

Advanced analytics and data visualization automated 40% of the cases being investigated for improper preparer identification, reducing case investigation from 20 minutes per case to less than a minute per case, significantly improving investigative asset utilization.

Learn more about data engineering and automating data pipelines

We employ a disciplined multi-phase data engineering process to properly organize and prepare the data to provide the highest potential for success from the Data Science.
Download Case Study