Blog / · GH-Proxy Team

CUA: Scaling Computer-Use 2.0 with Virtual Fleets, Drivers, and Synthetic Benchmarks

#Open Source #CUA #ComputerUse #GUI Agents #GitHub Proxy #Automation
Table of Contents4sections

CUA: Scaling Computer-Use 2.0 with Virtual Fleets, Drivers, and Synthetic Benchmarks

Enabling AI agents to control operating systems—viewing screens, typing text, and clicking GUI widgets across native apps—represents a critical leap toward truly autonomous digital workers. However, providing isolated sandboxes, high-fidelity input drivers, and standardized benchmarks across real OS environments has remained a massive engineering challenge.

CUA meets this challenge with a powerful vision: “Give AI agents computers they can use.” Surpassing 26,500+ Stars on GitHub with over 2,240+ Stars gained this week, CUA is establishing the open foundation for Computer-Use 2.0.

🚀 Accelerated Setup via GH-Proxy

Download CUA repositories, hypervisor tools, and drivers at peak speed via GH-Proxy:

👉 Direct GH-Proxy Acceleration for CUA
(Alternative URL: https://gh-proxy.com/https://github.com/trycua/cua)

Installation:

git clone https://gh-proxy.com/https://github.com/trycua/cua.git
cd cua
pip install -e .

🖥 Comprehensive Architecture

CUA delivers an end-to-end framework for desktop automation:

  1. Cua Spaces: Desktop hypervisor orchestration providing isolated Mac and Linux desktop sandboxes on bare-metal and cloud clusters.
  2. Cua Driver: Low-latency, high-precision cross-OS driver capturing window handles, displaying visual cursor trails, and dispatching keystrokes reliably.
  3. Lume VM Engine: Instant sub-second VM provisioning with automated checkpoint rollback for clean-slate retries.
  4. CUA-S1 Decision Models: Fine-tuned decision weights mapping high-level instructions to pinpoint coordinate clicks.
  5. Cua Bench: A comprehensive benchmark measuring end-to-end task completion rates across real productivity software.

🛠 Quick Automation Example

from cua import CuaSpaces, DesktopAgent

# Launch an isolated native desktop space
space = CuaSpaces.create(os="macos-sequoia", headless=False)

agent = DesktopAgent(model="claude-3-7-sonnet")

# Delegate full-fidelity GUI workload
agent.execute(
    space=space,
    goal="Launch Blender, load /assets/cube.obj, trigger GPU render, and save output to Desktop"
)

# Checkpoint and clean up
space.snapshot("blender_render_complete")
space.destroy()

🎯 Target Applications

  • Automating Legacy Desktop Software: Operate CAD, accounting, and proprietary industrial software lacking web APIs.
  • Autonomous Multi-OS QA Testing: Execute end-to-end regression suites across macOS, Windows, and Linux simultaneously.
  • Synthetic Training Trajectories: Generate high-volume, verified computer-use execution datasets to train next-generation multimodal models.

💬 Comments & Discussion