Hermes Computer Use Skill: How Hermes Agent Controls a Computer

Hermes computer use skill is how Hermes Agent sees a screen and uses apps the way you do: it takes screenshots, clicks, types and scrolls. Hermes does this with its built-in computer_use toolset, powered by Cua’s open-source cua-driver. You can add Cua’s skill pack on top, and since this week you can run it all inside a throwaway Cua Space. In the video below I show Hermes getting its own computer through Cua. Here’s how computer use works in Hermes, how to switch it on and how to keep it safe.

What the Hermes computer use skill does

Hermes’ computer_use toolset gives the agent background desktop control: screenshots, click, drag, scroll, type, key presses, waiting, listing apps and focusing an app. It reads the accessibility tree of any visible window and sends input without bringing the window to the front, switching desktops or moving your real cursor. So you and the agent can co-work on the same machine.

Unlike most computer-use tools, it works with any tool-capable model — Claude, GPT, Gemini or an open model on a local endpoint. It isn’t tied to one provider’s computer-use format.

Toolset vs skill: what’s the difference?

In Hermes, a tool is an ability (like computer_use) and a skill is a set of instructions that teaches the agent how to use abilities well. The computer_use toolset is built in. On top of it, Cua publishes a skill pack: install Cua Driver and run cua-driver skills install, and it lands in ~/.cua-driver/skills/cua-driver.

Hermes doesn’t auto-detect that folder yet — Nous lists it as a planned follow-up — so point Hermes at that directory or symlink it into your skills folder. You can also teach Hermes your own computer-use skills: show it a task, then use /learn to save the steps (see Hermes learn skill).

How to switch on computer use in Hermes

The driver ships with Hermes: the installers and hermes update install cua-driver on macOS, Windows and Linux, and the desktop app bundles it. If it’s missing, any of these installs it:

• Run hermes tools and pick Computer Use.
• Toggle the Computer Use toolset in the dashboard or desktop app.
• Run hermes computer-use install, then hermes computer-use status to check it.

Then grant the permissions for your system:

Finally, start a session with hermes -t computer_use chat, or add computer_use to your enabled toolsets in config.yaml.

Permission modes: how much freedom to give it

Bounded mode is the smart choice for repeat jobs like scheduled reports. You approve exactly what the agent may touch once, and it can’t step outside it. Nous warns that unrestricted mode doesn’t protect against prompt injection, so only use it on a machine you can throw away.

Which computer does Hermes control?

computer_use acts on the machine where the bot’s screen lives. On a local setup that’s your computer. With a sandboxed terminal such as Docker, SSH or Singularity, the driver runs inside the sandbox on its own display, so it can only touch what the sandbox can. Modal, Daytona and Vercel sandboxes can’t host a display yet.

That’s why the new Hermes Cua plugin is such a good fit. A Cua Space is a full computer with its own screen and apps. Set the plugin’s desktop setting to true and it installs what computer_use and the browser need into the Space. Then you can watch the agent click around live, and take over if it gets stuck.

Best uses for the Hermes computer use skill

• Apps with no API — old desktop software, internal tools or dashboards.
• Repeat admin — exporting reports, updating spreadsheets, filing forms.
• Testing your own site or app the way a real user would.
• Checking visual work — reading a design, a chart or a layout on screen.

For web pages, Hermes’ browser toolset is usually faster and more reliable. Use computer use for everything outside the browser.

My verdict on the Hermes computer use skill

Hermes’ computer use works with any model, runs in the background without hijacking your mouse, and comes with proper permission modes. Pair it with a Cua Space and you get an agent that uses a real computer without risking yours. If you want an agent that can handle the apps you use every day, the Hermes computer use skill is the place to start.

About Julian Goldie

I’m Julian Goldie — founder of Goldie Agency and the AI Profit Boardroom. I test new AI agents almost every day for 432K+ YouTube subscribers and run Hermes, OpenClaw, OpenCode, Claude and Codex side by side in my own Agent OS. Claim your free strategy session or join the AI Profit Boardroom.

FAQ: hermes computer use skill

What is the Hermes computer use skill?

Hermes Agent’s computer_use toolset, powered by cua-driver, lets the agent take screenshots, click, type and scroll in apps. Cua also publishes a skill pack you can add.

Which models work with Hermes computer use?

Any tool-capable model, including Claude, GPT, Gemini and local open models.

Does Hermes computer use take over my mouse?

No — it works in the background without moving your cursor or stealing keyboard focus.

How do I turn on computer use in Hermes?

Run hermes tools and pick Computer Use, grant the permissions for your system, then start Hermes with hermes -t computer_use chat.

Is Hermes computer use safe?

Use standard or bounded permission mode, and run it inside a sandbox or Cua Space for real isolation. Only use unrestricted mode on a disposable machine.

Related reading

• Hermes Computer Use
• Hermes Agent Computer Use
• Hermes Cua

Last updated October 2026. This page is a living guide to hermes computer use skill — the facts here move fast and it is updated as they do.