SWE-bench Multimodal (Vals) leaderboard

SWE-bench Multimodal (Vals) (Vals SWE-bench Multimodal evaluation) is an external benchmark published by SWE-bench Multimodal on Vals AI. JavaScript issue resolution where bug reports depend on screenshots, evaluated by Vals AI with no agent internet access. Its evaluation set is described by the source as 480 JavaScript issues with visual bug reports. The source rates its difficulty as visual software issue resolution.

Each model's result is published by SWE-bench Multimodal on Vals AI as resolved issue percentage. ModelScale prints that value to 2 decimal places, exactly as published, and never converts or rescales it. This page ranks the 0 models that carry a published value, highest value first; a model the source never scored is left out rather than ranked at zero. The source does not state whether a higher value is the better result, so neither does this page.