ai
A sophisticated RAG (Retrieval-Augmented Generation) chat application with AI governance and deepfake technology. The system combines BM25, fuzzy search, and vector search with BGE Reranker for optimal retrieval. Features include custom guardrails, prompt optimization using Textgrad, and deepfake video generation with lip-syncing.
The system follows a microservices architecture with separate components for RAG pipeline, guardrails, and deepfake generation. The RAG pipeline uses a hybrid retrieval approach combining BM25 (lexical) and BGE embeddings (semantic) for initial retrieval, followed by BGE Reranker for relevance optimization. The deepfake pipeline integrates multiple TTS models and lip-sync services.
User query → Guardrails check → Hybrid retrieval (BM25 + Vector) → BGE Reranker → LLM generation → Response guardrails → User. For deepfake: Text input → TTS models → Audio generation → Wav2Lip → Video output.
Fine-tuned BGE embeddings for enhanced retrieval and reasoning capabilities using FlagOpen/FlagEmbedding framework.
Learn MoreOpen-source lip-sync model by IIIT Hyderabad for synchronizing audio with video.
Learn MoreConditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech.
Learn MoreSimilar to DSPY but better approach for prompt optimization in RAG applications.