A modern approach to building a data science framework delivery pipeline using DevOps practices

Abstract

For data science, the potential for commercialization is significant. Recent work indicates that the philosophy of DevOps is a perfect way of addressing the challenges of software development. And both data science and software engineering from a product perspective need to provide customers with digital services. The feasibility of using DevOps in data science must therefore be studied. This article outlines a method for creating a framework for the use of DevOps methods within a data science program. I used four pipeline practices: version control, platform server, containerization and continuous integration and delivery. DevOps is, however, not a theory specifically designed for data science. This means that the DevOps methods which are currently available cannot solve all the problems of production data science. I have used DevOps' practice to address such a problem with a data science practice. I've learned and engaged in the process of transfer learning. This paper describes a parameter-based method to move parameters from one dataset to another. I have examined the impact of model transmission learning on a new dataset. The result demonstrates the adaptation of the learning process to modify the model without re-training from scratch when the dataset changes but is identical to the old one. This is a safe idea to freeze the convolutive layer if the current model is to reach the same degree of efficiency as the previous one. When the new model will achieve higher than the original model, the loading of weights is better choice. In conclude, the current methods of DevOps do not need to be restricted if we use DevOps in data science.