As analytics professionals, we can work days or weeks building and validating predictive or descriptive models. However, the success of an analytics project does not just depend on the model building. To truly be a success, we need to deploy our models, integrate them into workflows, and use them to change business processes. Operationalizing the analytic assets can be the most rewarding part of the process, but how do we know if we are successful? How can we ensure that once analytics are deployed in production that the environment will remain stable and continue to produce the desired results?
On a recent client engagement, we had the opportunity to work with an established team of data scientists and software engineers. This client built most of their analytic models using SPSS Modeler in conjunction with an SPSS Collaboration and Deployment Services (C&DS) repository. C&DS is a platform for the management and deployment of analytic assets, which provides a centralized repository for storing and sharing assets, controlling version, automating processes, and sharing the results.
This customer's typical C&DS configuration housed both development and production analytic models on a single production server. Without a true development environment, models were simply labeled “development” or “production” using different folders, filenames, and tags within the same production repository.
The challenge with this configuration was there was no way to test changes before they affected the production system. Any development change made to the server could potentially take down the production system and cause the models to fail. As the number of analytic models grew and more users were expecting daily results, the limitations of this design became critical and we knew we needed to provide a more stable environment.
The solution involved adding new C&DS servers for each deployment area. Since C&DS provides features to manage the process of promoting analytic assets from one environment to the next, we set up distinct development and production analytic environments with reusable “promotion policies” to deploy analytic models into production.
Security roles were created to limit functionality and keep the production environment clean and stable. The development environment was open to data scientists for creating, sharing, and storing streams and jobs, but the server and database configuration, promotion policies, and ability to promote models were managed by a small number of administrators. This provided the advanced analytic developers with the flexibility to do their job—creating big data solutions—but also kept the environments stable. When server changes were required, they were first applied to the development environment. If problems were encountered, we had the ability to fix them before they affected the production jobs. This solution prevented nearly all of the problems previously experienced in the client’s production environment.
The client has now been successfully using this new topology for more than a year. The downtime for the production analytics has been minimal, and it can be scheduled so that it does not interfere with the delivery of the analytic results to the users. Although this seems like a simple solution, it is our experience that not all data scientists—especially SPSS Modeler users— are aware of the practice of using separate environments in C&DS. We recommend establishing up a distinct "sandbox" development space for building and testing analytics, and then using promotion policies to push well-tested models to a separate production environment.
At Elder Research, we are committed to excellence in the entire process of building, validating, and operationalizing analytic assets, to deliver results that will add value to the business. Whether a client is using commercial tools, open-source options, or a custom software solution to build and deploy models, our analytics experts are able to work within unique environmental constraints to solve their problems. Our success depends on our client’s success and satisfaction.
Request a consultation to see how Elder Research can help with model deployment or other analytics consulting services.