Building CineRAG: A Production-Ready Movie Recommendation Engine with RAG

The Challenge

Building a movie recommendation system sounds straightforward—until you need it to actually understand what users want.

Traditional approaches fall short:

• Keyword search misses semantic intent ("movies like Inception but darker")
• Collaborative filtering requires extensive user history
• Content-based filtering can't handle nuanced queries

I wanted to build a system that could:

• Understand natural language queries like "uplifting movies about unlikely friendships"
• Deliver sub-100ms search latency across thousands of movies
• Provide Netflix-quality UI/UX with responsive design
• Scale to production workloads (1000+ queries per second)
• Be completely containerized for easy deployment

The Goal: Demonstrate that RAG isn't just for documents—it's a powerful paradigm for any semantic search application.

My Approach

I designed CineRAG as a complete, production-ready RAG system following industry-standard patterns. Here's how I built it:

Phase 1: Data Ingestion & Processing

Challenge: MovieLens data is sparse—movie titles and genres don't capture what makes a film compelling.

Solution:

• Ingested the MovieLens 25M dataset (9,742 movies with ratings)
• Enriched data with TMDB API for:
  • Plot summaries and overviews
  • High-resolution poster images
  • Cast, crew, and production details
  • User reviews and metadata
• Created composite text embeddings combining title, overview, genres, and keywords

Phase 2: Embedding & Vector Storage

Challenge: Standard embedding models miss domain-specific nuances in movie descriptions.

Solution:

• Used Sentence-Transformers (all-MiniLM-L6-v2) for 384-dimensional embeddings
• Stored vectors in Qdrant vector database for production-grade similarity search
• Indexed both content embeddings and metadata for hybrid search
• Implemented HNSW indexing for sub-linear search complexity

Phase 3: Query Processing & Retrieval

Challenge: User queries are messy—typos, vague descriptions, mixed intent.

Solution:

• Built an intelligent query processor with:
  • Intent detection (genre, mood, similarity, actor)
  • Query expansion (adding related terms)
  • Synonym handling and normalization
• Implemented hybrid search:
  • Vector similarity for semantic matching
  • Keyword matching for exact terms (actor names, titles)
  • Metadata filtering (year, genre, rating)
• Added reranking layer to refine top-K results

Phase 4: Optimization & Caching

Challenge: Vector operations are expensive. 1000+ QPS requires smart optimization.

Solution:

• Implemented multi-tier caching:
  • LRU cache for hot queries
  • Redis for distributed caching
  • Achieved 40%+ cache hit rate
• Query batching and async processing
• Connection pooling for database efficiency
• Achieved 19-45ms average search latency

Phase 5: Evaluation & Monitoring

Challenge: How do you know if recommendations are actually good?

Solution:

• Implemented real-time evaluation metrics:
  • NDCG (Normalized Discounted Cumulative Gain)
  • MAP (Mean Average Precision)
  • MRR (Mean Reciprocal Rank)
• Built monitoring dashboard for:
  • Query patterns and popular searches
  • Latency percentiles (p50, p95, p99)
  • Cache performance and hit rates
• Continuous A/B testing framework for improvements

Phase 6: Full-Stack Implementation

Challenge: AI demos often look terrible. I wanted Netflix quality.

Solution:

• Built React + TypeScript frontend with:
  • Netflix-style carousel layouts
  • Responsive grid design
  • Real-time search with debouncing
  • Movie detail modals with rich metadata
• FastAPI backend with:
  • Automatic OpenAPI documentation
  • Health checks and metrics endpoints
  • Rate limiting and error handling
  • CORS configuration for production

Architecture

┌───────────────────────────────────────────────────────────────────────┐
│                         CineRAG Architecture                          │
└───────────────────────────────────────────────────────────────────────┘

┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│  MovieLens   │     │   TMDB API   │     │   Raw Data   │
│   Dataset    │────▶│   Enrichment │────▶│   Storage    │
└──────────────┘     └──────────────┘     └──────────────┘
                                                 │
                                                 ▼
┌──────────────────────────────────────────────────────────────────────┐
│                    7-Stage RAG Pipeline                               │
│                                                                       │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐    │
│  │Ingestion│─▶│Embedding│─▶│ Vector  │─▶│  Query  │─▶│Retrieval│    │
│  │         │  │         │  │  Store  │  │Processing│  │         │    │
│  └─────────┘  └─────────┘  └─────────┘  └─────────┘  └─────────┘    │
│                                                            │         │
│  ┌─────────────┐  ┌─────────────┐                         │         │
│  │ Evaluation  │◀─┤Optimization │◀────────────────────────┘         │
│  └─────────────┘  └─────────────┘                                   │
└──────────────────────────────────────────────────────────────────────┘
                           │
                           ▼
┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│   FastAPI    │     │    Redis     │     │   Qdrant     │
│   Backend    │◀───▶│    Cache     │◀───▶│  Vector DB   │
└──────────────┘     └──────────────┘     └──────────────┘
        │
        ▼
┌──────────────────────────────────────────────────────────────────────┐
│                      React Frontend (Netflix-Style)                   │
│                                                                       │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐                   │
│  │   Search    │  │  Carousels  │  │   Details   │                   │
│  │    Bar      │  │  & Grids    │  │   Modal     │                   │
│  └─────────────┘  └─────────────┘  └─────────────┘                   │
└──────────────────────────────────────────────────────────────────────┘

Results

After building and optimizing CineRAG, here are the quantifiable outcomes:

Performance Metrics

Quality Metrics

• Semantic Understanding: Handles complex queries like "thought-provoking sci-fi with philosophical themes"
• Response Quality: Netflix-style presentation with rich metadata
• Code Quality: Type-safe TypeScript + Python with comprehensive typing
• Documentation: Complete README, architecture docs, and API documentation

Business Impact

This project demonstrates:

• Full-stack AI capability: From data pipeline to production deployment
• Performance engineering: Exceeding targets on every metric
• Production patterns: Caching, monitoring, containerization, health checks
• Industry-standard architecture: Reusable patterns for any RAG application

Key Technical Decisions

Why Qdrant Over Pinecone?

Qdrant offers:

• Self-hosting option (no vendor lock-in)
• Rich filtering (metadata queries alongside vectors)
• Better performance for my scale (10K vectors)
• Open source with active development

Why Hybrid Search?

Pure vector search missed exact matches like actor names or specific titles. Combining vector similarity with keyword matching gave the best of both worlds:

• "Movies with Tom Hanks" → keyword match
• "Heartwarming stories about redemption" → vector match
• "Tom Hanks redemption movies" → hybrid

Why Multi-Tier Caching?

Single-layer caching wasn't enough:

• LRU (in-memory): Ultra-fast for hot queries
• Redis: Distributed, persistent, shared across instances
• Combined: 40%+ hit rate, major latency reduction

Why Sentence-Transformers Over OpenAI Embeddings?

• Cost: $0 vs. $0.0001/1K tokens (adds up at scale)
• Latency: Local inference is faster
• Privacy: No data leaves my infrastructure
• Quality: Surprisingly comparable for this use case

Lessons Learned

1. RAG isn't just for documents: Vector search + semantic understanding works beautifully for any content type. Movies, products, music—same patterns apply.

2. Caching is underrated: Most RAG tutorials skip caching. In production, it's the difference between 200ms and 20ms response times.

3. Evaluation metrics matter: Without NDCG/MAP/MRR, you're just guessing if your system is good. Implement evaluation from day one.

4. UI quality signals engineering quality: A polished frontend makes the whole project more credible. Don't skimp on UX.

5. Docker everything: Containerization isn't optional for production. I can spin up the entire stack with one command.

Technology Stack

• Vector Database: Qdrant (self-hosted)
• Embeddings: Sentence-Transformers (all-MiniLM-L6-v2)
• Backend: Python, FastAPI, Pydantic
• Caching: Redis, LRU cache
• Frontend: React, TypeScript, TailwindCSS
• Data Sources: MovieLens 25M, TMDB API
• Deployment: Docker, Docker Compose
• Monitoring: Custom metrics, health checks

Conclusion

CineRAG proves that RAG engineering is a powerful paradigm extending far beyond document Q&A. By applying industry-standard patterns—semantic embeddings, hybrid search, multi-tier caching, and proper evaluation—you can build production-ready AI systems that genuinely understand user intent.

The techniques I used here transfer directly to:

• E-commerce: Product recommendations and search
• Content platforms: Article/video discovery
• Enterprise search: Internal knowledge bases
• Customer support: Intelligent ticket routing

Want to discuss building a semantic search system for your use case? Let's talk about how these patterns can work for you.

View the complete source code on GitHub.