AI Error Analysis For PM's & Engineers

AI Error Analysis For PM's & Engineers is a two-hour workshop in Tbilisi focused on monitoring and fixing AI system failures. Unlike traditional software, AI systems fail in distinct ways that require specialized evaluation and iteration practices. The session covers how teams can observe human-AI interactions, evaluate outputs requiring expert judgment, optimize system prompts without introducing regressions, and measure real progress beyond vanity metrics. Participants will explore practical approaches to testing and improvement when starting with limited user data, and learn where to prioritize engineering effort early in development.