For AI Advice to Be Any Good, It Has to Actually Change the Outcome
Salesforce, Thursday, November 12th, 2026
Salesforce AI Research built Interactive Visual Gym to test whether real-time visual AI assistants' guidance actually changes outcomes.
Salesforce AI Research notes that AI assistants are becoming real-time guides for physical work, but most evaluations replay prerecorded video and grade answers against a key, which cannot show whether a warning worked.
Interactive Visual Gym (IVG) is a simulated proving ground that pairs everyday tasks such as cooking or managing smartphone settings with simulated users who react to the assistant's instructions.
Assistants speak in natural language, out-of-plan moments such as mistakes and plan changes can be scheduled precisely, and simulated users vary in skill, trust in AI, execution speed and tolerance for interruption.
The team used it to build IVG-Bench, with 315 scenarios across three environments used to evaluate 15 assistants.