What iscomputer use?
An AI agent clicks and types like a person, even where there is no API.
Get in touch
Let's talk about your project.
First we check whether the project fits your business model. Then you get a proposal with phases and effort.
Talk about AI agents or call: +49 151 1576 5566Computer use means an AI agent operates a program through its interface: it looks at a screenshot, decides on the next click or keystroke, carries it out and checks the next screenshot to see what happened. The software needs no API for this. That is what suddenly makes an older inventory system or a supplier portal reachable.
- OpenAI added computer use to its Agents API on 29 September 2026, and it is already generally available at Microsoft and Anthropic.
- Microsoft says you should not rely on the built-in check-in with a person as a safeguard.
- If you start small, let the agent only read values at first.
How computer use makes software without an API usable
You give the agent a task in plain language, for example copying invoice data from a PDF into an entry form, and from then on it clicks, scrolls and types the way a person would.
An API would have to come from the software vendor. Going through the interface needs nothing from them, but it is slower and makes mistakes an API would not, because the agent interprets an image instead of reading a data field. Microsoft's documentation does note one advantage: it keeps working when buttons or screens change.
Which screen work computer use is worth it for
It suits work that repeats constantly, always runs through the same screens and produces a result you can check afterwards. Anything that needs case-by-case judgement or involves money and contracts is a poor fit. Reading data is the calmest way in (it changes nothing in the other system), and writing comes later.
Which permissions an agent with computer use should get
An agent that clicks can do whatever the account it runs in is allowed to do. The vendors therefore arrive at the same advice independently, a dedicated, isolated machine and an allow list of permitted sites and applications, with Microsoft and Anthropic adding an account with as few rights as possible.
Purchases and destructive changes should be confirmed by a person. There is no guarantee the system will ask first, though, since Microsoft calls the models' behaviour probabilistic. We would put the safeguard into permissions instead. An agent that technically cannot submit an order never needs to ask about it.
Frequently asked questions
Can you automate software without an API?
Yes, that is what computer use is for. Microsoft shows it with examples such as transferring invoice data from a PDF into a form.
Can an AI agent operate any software?
Almost anything a person can operate. If your industry software runs through Citrix, check first, because Microsoft says password fields may not work there.
Terms you should know in the same context
AI agent · Agentic AI · AI assistant · Local AI · Back to the AI glossary A to Z
Which program in your business has no API and still costs someone typing time every day?