Oriel — AI & technology worth noticing, every day.

vercel-labs/agent-desktop

Repository · https://github.com/vercel-labs/agent-desktop

Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates it — refs stay stable and actions stay safe to retry, instead of guessing from pixels.

First seen by Oriel on .

AI summary

agent-desktop is a cross-platform CLI and C library that lets AI agents observe and operate desktop applications through accessibility trees using JSON element references.

  • Snapshots return JSON trees with actionable refs; agents perform ref-based actions and snapshot again to verify results without pixel matching.
  • The tool runs headlessly by default, avoiding pointer movement and focus changes, while --headed enables physical input when needed.
  • It supports macOS, Windows, and Linux, with platform-specific requirements including macOS Accessibility permission and Linux at-spi2-core graphical sessions.
  • The project warns that snapshots, screenshots, clipboard data, command output, and optional model-assisted descriptions may contain private desktop content.
  • Refs are tied to snapshots, so UI changes can produce STALE_REF or AMBIGUOUS_TARGET errors requiring a fresh snapshot or window selection.

#desktop automation #ai agents #accessibility #rust

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Stars

Total repository stars at each observation.

  • GitHub Trending: 17,583, latest observation ; +9,836 over observed period since

Stars gained (1d)

Daily star gain reported by GitHub Trending. A change compares two daily values; it is not additional stars between observations.

  • GitHub Trending: 2,173, latest observation ; -70 over observed period since

Forks

Total repository forks at each observation.

  • GitHub Trending: 1,252, latest observation ; +699 over observed period since

Aliases

SourceExternal IDURL
GitHub Trendingvercel-labs/agent-desktophttps://github.com/vercel-labs/agent-desktop Read discussion