Dubravko Dolic & Hüdaverdi Cakir
Industrializing your Data Science capabilities
#1about 2 minutes
The challenge of industrializing data science models
A tire recall incident highlights the gap between a data scientist's local Python script and a scalable, production-ready solution.
#2about 3 minutes
Building the initial concept for a data science factory
The journey began with a demand forecasting use case, leading to the concept of a lab for experimentation and a factory for industrialization.
#3about 5 minutes
Establishing processes and a cloud-agnostic tool stack
A standardized process with dev, QA, and prod stages was created, supported by a cloud-agnostic tool stack including Git, Jenkins, and Kubernetes.
#4about 5 minutes
Technical architecture for a multi-stage deployment environment
The architecture uses Kubernetes and containerization to create reproducible dev, QA, and prod environments with immutable builds and stage-specific configurations.
#5about 11 minutes
Live demo of deploying and promoting application versions
A command-line interface is used to deploy a new version of a Shiny application to the dev environment and promote an existing build from dev to QA.
#6about 5 minutes
Monitoring applications with logs and metrics
The platform provides developers with access to Elastic Stack for log aggregation and Prometheus with Grafana for metrics to monitor application performance.
#7about 2 minutes
Providing a simplified lab environment for data scientists
A web-based frontend offers pre-configured templates for RStudio and Python, abstracting away infrastructure complexity for data scientists.
#8about 4 minutes
Real-world use cases from tire manufacturing
Several applications are showcased, including real-time tire monitoring for mining trucks, optimizing material mixing, and deploying scrap prediction models to edge devices.
#9about 6 minutes
Integrating the factory into a larger analytics ecosystem
The Data Science Factory is part of a broader ecosystem that includes a telemetry backbone, image recognition pipelines, and predictive maintenance platforms.
Related jobs
Jobs that call for the skills explored in this talk.
Matching moments
01:55 MIN
Merging data engineering and DevOps for scalability
Software Engineering Social Connection: Yubo’s lean approach to scaling an 80M-user infrastructure
Unlock full access
Log in or set up an account to access this feature and more.
02:34 MIN
Bridging gaps with DevOps and containerization
From Punch Cards to AI-assisted Development
Unlock full access
Log in or set up an account to access this feature and more.
10:54 MIN
Managing massive data scales with the Robotic Drive platform
How to develop an autonomous car end-to-end: Robotic Drive and the mobility revolution
Unlock full access
Log in or set up an account to access this feature and more.
02:18 MIN
Using data management and open source tools for MLOps
MLOps - What’s the deal behind it?
Unlock full access
Log in or set up an account to access this feature and more.
04:25 MIN
From research concepts to production-ready data products
Implementing continuous delivery in a data processing pipeline
Unlock full access
Log in or set up an account to access this feature and more.
05:04 MIN
A platform for self-service infrastructure and environments
Let developers develop again
Unlock full access
Log in or set up an account to access this feature and more.
05:15 MIN
Applying TDD, CI/CD, DevOps, and SAFe at scale
Agile work at CARIAD – Creating a customer web application for controlling the vehicle
Unlock full access
Log in or set up an account to access this feature and more.
02:49 MIN
Q&A: MLOps tools for building CI/CD pipelines
Data Science in Retail
Unlock full access
Log in or set up an account to access this feature and more.
Featured Partners
Related Videos
44:40DevOps for Machine Learning
Hauke Brammer
45:45Effective Machine Learning - Managing Complexity with MLOps
Simon Stiebellehner
47:19The state of MLOps - machine learning in production at enterprise scale
Bas Geerdink
30:49AI beyond the code: Master your organisational AI implementation.
Marin Niehues
34:00Implementing continuous delivery in a data processing pipeline
Álvaro Martín Lozano
45:58Data Fabric in Action - How to enhance a Stock Trading App with ML and Data Virtualization
Andreas Christian
22:41Empowering Retail Through Applied Machine Learning
Christoph Fassbach & Daniel Rohr
30:33A solution to embed container technologies into automotive environments
Falk Langer & Lukas Stahlbock
Related Articles
View all articles
.gif?w=240&auto=compress,format)
From learning to earning
Jobs that call for the skills explored in this talk.
iits-consulting GmbH
Munich, Germany
Intermediate
Go
Docker
DevOps
Kubernetes
SYSKRON GmbH
Regensburg, Germany
Intermediate
Senior
.NET
Python
Kubernetes
evoila Frankfurt GmbH
Mainz, Germany
Intermediate
Senior
Kubernetes
