🇺🇸 English 🇰🇷 한국어 🇯🇵 日本語
Home › Developer Tools
Developer Tools

Cursor Agents Now Drive Desktop Apps

Cursor said Oct. 23 its coding agents can now control desktop apps such as the iOS Simulator, via a macOS helper app and opt-in self-hosted workers on macOS and Linux.

Cursor said on Oct. 23 that its coding agents can now operate a developer's desktop, clicking and typing inside native apps such as Apple's iOS Simulator to check the features they have just built. "Cursor can now control your computer," the company wrote on X, adding that agents "can drive desktop apps like the iOS Simulator to test the features they build."

Xcode on a Mac screen next to an iPhone device simulator running a test app
Xcode and its iOS device simulator running a hobbyist's test app, a file photo. 사진: mareklug / Flickr(CC BY-SA 2.0)

Key takeaways

  • Cursor's agent can now click, type and take screenshots in desktop apps, not only in a browser or a remote cloud machine.
  • On macOS, the feature runs through a helper app called Cursor Computer Use that needs Accessibility and Screen Recording permissions.
  • Self-hosted workers on macOS and Linux must be started with a --computer-use flag; Cursor's docs say the feature is "never enabled by the server."

How Cursor Computer Use works

The agent can pick the tool on its own when a task needs visual interaction, or a developer can call it by typing @ in an agent prompt and choosing "Computer Use," according to guides published by developer Flavio Copes and the newsletter AlphaSignal. It sits next to the agent's other tools, so one run can edit source code, build the app and then tap through the result.

On a Mac, the permissions go to the helper app itself, not to Cursor or the terminal. Cursor's documentation warns that macOS 15 and later periodically asks users to re-approve Screen Recording, so screenshots can stop working until someone clicks "Allow For One Month."

Self-hosted agents on macOS and Linux

For teams that run Cursor agents on their own hardware, the documentation describes a worker command, agent worker --computer-use start, that lets an agent drive that machine's desktop. Linux workers use an X11 display and can start their own virtual desktop if none exists.

A separate Linux-only option, --share-desktop, lets authorized viewers watch or take over an agent's desktop from Cursor. "Both features are explicit opt-ins and are never enabled by the server," the docs say. Enterprise administrators can turn computer use off with a switch in team settings, which is on by default.

Why desktop control matters for coding agents

Until now, Cursor's screen-level testing lived mostly in the cloud. In February the company gave its cloud agents isolated virtual machines with full desktops, and said more than 30% of the pull requests it merged internally were created by agents working autonomously in those sandboxes.

Native local control was a frequent request. A forum thread opened July 17 asked for agents that could operate Adobe, Office and other desktop apps. As recently as Oct. 6, a Cursor staff member replied, "This is still something we're exploring, but I can't share a timeline yet." The latest test-your-work push follows earlier debates over agents that report success without checking, such as LLM bug-fixers that broke working code.

The trade-off is exposure. An agent with screen access can see whatever is open on the machine, which is why the guides recommend test accounts, closing unrelated apps and committing work before a run.

What's next

As of Oct. 25, Cursor's public changelog had no entry for the feature, and the company had not published pricing, plan limits or Windows support for local computer use. Those details, along with any audit or approval controls for the local version, are the main gaps for teams weighing a rollout.

FAQ

Can Cursor control apps on my Mac?

Yes, after you update Cursor and grant Accessibility and Screen Recording permissions to the Cursor Computer Use helper app.

Is Cursor computer use turned on automatically for self-hosted machines?

No. The worker must be started with --computer-use, and enterprise admins can disable the feature for their team.

Does it work on Windows?

Cursor has not said. Its documentation covers macOS and Linux only.

References

#Cursor #Computer Use #Coding Agents #iOS Simulator #Anysphere #AI Agents #Desktop Automation #macOS
S
Seung Jung
AI Newsway · Developer

Editor responsible for AI Newsway. Writes about AI, developer tools, cloud and productivity technology.

More by this reporter ›
Copyright © AI Newsway All rights reserved. Unauthorized reproduction, redistribution, and AI training prohibited.
Share
f X K

Comments 0

Comments are currently disabled.

Be the first to comment.