Project Deep Dive
End-to-End Recommendation System
A production-style ML recommendation stack with Kafka, online serving, retraining workflows, and monitoring.
Overview
This project was built as an end-to-end recommendation pipeline rather than just a notebook model. It covers event ingestion, model training, online serving, operational monitoring, and retraining workflows for a production-style recommendation system.
Key Outcomes
Built a production-grade recommendation pipeline over about 1M user-movie interactions.
Added retraining workflows, runtime monitoring, and A/B testing for production-style evaluation.
Integrated Prometheus and Grafana to make model-serving behavior observable in real time.
Architecture Diagram
User interaction data flows through streaming, training, online serving, and monitoring with retraining closing the loop.
Architecture
Streaming interactions flow through Kafka so the recommendation pipeline can separate data collection from model serving concerns.
Collaborative filtering models such as KNN and SVD power personalized ranking across large user-item interaction data.
Dockerized Python Flask APIs expose inference endpoints for recommendation delivery in a production-style serving layer.
Automated retraining workflows, monitoring, and evaluation close the loop between model updates and runtime behavior.
Stack
- Kafka
- Python
- Flask
- Docker
- Prometheus
- Grafana
- GCP
- Collaborative Filtering