YouTube Channel RAG Tool 🎥

A powerful Retrieval-Augmented Generation (RAG) tool that lets you chat with any YouTube channel's video transcripts using Google's Gemini API and Apify's YouTube scraper.

Features ✨

• 📺 Scrape videos from any YouTube channel
• 📝 Extract video transcripts and subtitles
• ☁️ Upload to Gemini File Search for semantic search
• 💬 Interactive chat interface to query video content
• 🔍 Smart retrieval of relevant information from transcripts

Prerequisites 📋

• Python 3.8 or higher
• Gemini API key (Get one here)
• Apify API token (Get one here)

Installation 🚀

1. Clone the repository

   git clone <your-repo-url>
   cd gemini-api-rag

2. Install dependencies

   pip install -r requirements.txt

3. Set up environment variables

   cp .env.example .env

   Edit .env and add your API keys:

   GEMINI_API_KEY=your_gemini_api_key_here
   APIFY_API_TOKEN=your_apify_api_token_here
   

Usage 💡

Run the tool:

python youtube_rag.py

The tool will prompt you for:

1. YouTube Channel URL - The full URL of the channel (e.g., https://www.youtube.com/@channelname)
2. Number of videos - How many recent videos to process

Example Session

🎥 YouTube Channel RAG Tool
================================================================================

📺 Enter YouTube channel URL: https://www.youtube.com/@example
🔢 How many videos to process (newest first): 10

🔍 Scraping channel: https://www.youtube.com/@example
📊 Fetching up to 10 videos...
⏳ Running Apify YouTube scraper...
✅ Successfully scraped 10 videos

📝 Creating transcript files...
  ✓ Created: 001_Video_Title_One.txt
  ✓ Created: 002_Video_Title_Two.txt
  ...
✅ Created 10 transcript files

☁️  Uploading files to Gemini File Search...
  ⬆️  Uploading: 001_Video_Title_One.txt
  ⬆️  Uploading: 002_Video_Title_Two.txt
  ...
✅ Uploaded 10 files to Gemini

================================================================================
💬 Chat Interface - Channel Name
================================================================================
Ask questions about the video transcripts!
Type 'quit' or 'exit' to end the conversation

🧑 You: What topics are covered in these videos?
🤖 Assistant: Based on the transcripts, the main topics include...

How It Works 🔧

1. Scraping: Uses Apify's YouTube scraper to fetch video metadata and transcripts
2. File Creation: Creates individual text files for each video with title, description, and transcript
3. Upload: Uploads all transcript files to Gemini's File Search for semantic indexing
4. Chat: Provides an interactive chat interface powered by Gemini 2.0 with file search capabilities

Project Structure 📁

gemini-api-rag/
├── youtube_rag.py          # Main application script
├── requirements.txt        # Python dependencies
├── .env.example           # Environment variables template
├── .env                   # Your API keys (not committed)
├── .gitignore            # Git ignore rules
├── README.md             # This file
├── CLAUDE.md             # Project instructions
└── transcripts/          # Generated transcript files (not committed)
    ├── 001_video.txt
    ├── 002_video.txt
    └── ...

API Documentation 📚

• Gemini File Search: https://ai.google.dev/gemini-api/docs/file-search
• Apify Documentation: https://docs.apify.com/
• YouTube Scraper Actor: https://apify.com/streamers/youtube-scraper

Rate Limits & Pricing 💰

Gemini API

• Free tier: 1 GB storage limit
• File size limit: 100 MB per file
• Embedding indexing: $0.15 per 1M tokens
• Queries and storage are free

Apify

• Free tier available with limited usage
• Check Apify pricing for details

Troubleshooting 🔧

"No transcript available"

Some videos may not have subtitles or transcripts available. The tool will still create files for these videos but with limited content.

API Rate Limits

If you encounter rate limit errors, try:

• Processing fewer videos at once
• Adding delays between requests
• Checking your API quotas

Connection Errors

Ensure you have a stable internet connection and valid API keys.

Contributing 🤝

Feel free to open issues or submit pull requests for improvements!

License 📄

MIT License - feel free to use this tool for your projects!

Credits 🙏

• Built with Google Gemini API
• YouTube scraping powered by Apify