Latest open artifacts (#21): Open model bonanza! Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, GLM-5.1 & others. On CAISI's V4 assessment.

Open Labs, including DeepSeek, released new models this month. The Center for AI Standards and Innovation (CAISI) assessed these open models, noting that the gap between them and American models is widening over time. The report employed nine different criteria, revealing that DeepSeek V4 performed poorly in some areas, impacting its overall evaluation. CAISI utilizes an Elo calculation method based on Item Response Theory, commonly used for model comparisons. However, the gap between open and closed models does not present the complete picture, as evaluations rely on simplistic benchmark settings.