Optima tackles AI benchmarking's biggest flaw by letting users test models against their own data

Artificial Analysis has launched Optima, a platform enabling users to create custom AI benchmarks using their own data and workflows. The platform allows models to be compared not only on quality but also on cost and time per task, providing deeper insights for agent-based applications. This development addresses the biggest flaw in AI benchmarking by allowing users to test models against their own data, enhancing evaluation accuracy and providing a clearer picture of actual performance.