AsgardBench: A benchmark for visually grounded interactive planning

AsgardBench has been introduced as a new benchmark for assessing the ability of artificial agents to adjust their plans based on visual feedback. This benchmark includes 108 tasks across 12 categories, requiring agents to adapt to their observations in the environment. For instance, when tasked with cleaning a kitchen, a robot must determine whether items are clean or dirty and modify its plan accordingly. This process necessitates that agents utilize visual feedback to refine their decisions, presenting a significant challenge.