II.
Benchmark overview
Reference · livebenchmark:os-world
OSWorld overview
OSWorld (Xie et al., 2024) is a scalable real-computer environment benchmark for multimodal agents performing open-ended tasks in Ubuntu, Windows, and macOS with full GUI/keyboard/mouse access; 369 real-world tasks across browser, OS, and productivity apps.
Attributes
displayName
OSWorld
homepageUrl
kind
full-stack
targetsKind
AgentVersion
description
OSWorld (Xie et al., 2024) is a scalable real-computer environment
benchmark for multimodal agents performing open-ended tasks in
Ubuntu, Windows, and macOS with full GUI/keyboard/mouse access;
369 real-world tasks across browser, OS, and productivity apps.
Outgoing edges
covers1
- skill-area:desktop-automation·SkillAreaDesktop Automation
Incoming edges
belongs_to_benchmark1
- test-set:osworld-369-computer-tasks·TestSetOSWorld 369 real-computer tasks
for_benchmark1
scored_against1
- eval-result:os-world.claude-sonnet-4-5.001·EvalResult