Many small companies have a list of processes that should have been automated years ago.
The work may involve a supplier portal, an old accounting application, a desktop database, a government website, or a vertical industry tool that has no usable API. An employee logs in, reads values from one system, enters them into another, downloads a document, changes its name, and sends a confirmation email.
Custom integration often stops at the first technical question: does the software expose an interface another system can call? Interface automation has existed for years, but it can be brittle when screens change.
Computer-use agents change that calculation. They can look at a screen, move a cursor, type into fields, press buttons, read the result, and continue. The interface built for a person becomes an interface an agent can use.
The lack of an API no longer ends the automation discussion.
This is Part 2 of After the Chatbot, following Part 1 on websites for AI shopping agents .
What is a computer-use agent?
A computer-use agent is an AI system that can operate a browser or desktop application on behalf of a user. It receives a goal, observes the visible interface, plans a sequence of actions, and uses controls such as clicks and keyboard input to complete the task.
This extends beyond scripted browser automation. A script expects a known page structure and a fixed sequence. A computer-use agent can interpret labels and plan the next action, although interface changes and ambiguous screens still cause failures.
OpenAI’s ChatGPT Work can work across apps and files on longer projects. AWS has introduced dedicated desktops for AI agents , with identity controls and audit trails for access to web and legacy applications.
AWS packages this capability for enterprise environments. The underlying approach is also relevant to smaller firms that depend on software that works well enough to keep, yet badly enough to create hours of manual copying and checking.
The legacy software barrier is weakening
Many small companies have a mixed technology stack. Modern cloud tools sit beside old desktop software. Important records live in email attachments. Supplier and public-sector portals require manual logins. Some workflows still depend on spreadsheets because the official system cannot exchange data cleanly.
Replacing every system would be expensive and disruptive. Building custom integrations can cost more than the process is worth.
A computer-use agent offers a third option. It can work with the existing interface while the company keeps the underlying system.
Potential examples include:
- downloading invoices from a supplier portal
- entering approved order data into a desktop system
- checking shipment or application status across several websites
- moving information from email attachments into an internal tool
- preparing recurring reports from applications without export APIs
- updating records in an old CRM
- submitting standard forms after a person approves the data
The process still needs to justify automation. A slow task performed twice a year is rarely a good candidate. A clear task repeated every day may be.
Choose work that is boring and observable
The best first workflow has five properties.
It is repetitive
The same inputs, decisions, and outputs appear frequently enough to create a meaningful return.
It is rule-based
The agent can distinguish a normal case from an exception. When the rules do not cover a case, it can stop and ask.
It is reversible
An incorrect draft, download, or internal record can be corrected. Sending money, dismissing an employee, or accepting a binding contract carries a different level of risk.
It is observable
The company can check what the agent saw, which actions it took, and what changed as a result.
It has an owner
One person is responsible for the workflow, its permissions, its success criteria, and its shutdown path.
A process with those properties can be piloted safely. A vague instruction such as “manage our operations system” cannot.
Screen access creates a large permission surface
An API usually exposes a defined set of actions. A desktop session may expose everything the user can reach.
This creates practical risks. An agent could open the wrong record, reveal confidential information, follow malicious instructions embedded in a webpage, or take an irreversible action after misreading the interface. NIST guidance on generative AI treats prompt injection and overly broad agent access as risks that system designers should assume and constrain.
A production workflow needs controls:
- a dedicated account with the minimum required access
- a separate desktop or browser environment
- limits on which applications and domains are available
- approval before financial, contractual, or customer-facing actions
- logs, screenshots, and output records
- automatic stopping when the interface changes unexpectedly
- a clear kill switch
The agent should never inherit a founder’s full desktop and password collection simply because that is convenient during the demo.
Computer-use agents still need process design
Robotic Process Automation, or RPA, has used scripts and interface automation for years. Computer-use agents add interpretation and flexible planning. They do not remove the need for process design.
An agent can still fail when:
- the page presents ambiguous information
- a pop-up covers an important control
- a session expires
- multi-factor authentication requires a person
- a visual change alters the meaning of the screen
- the source data is incomplete
- an unexpected case needs business judgment
The practical design should expect and test these failures. A useful workflow detects them, records them, and routes them to a person without corrupting the rest of the process.
A two-week pilot
A small company can test the capability without launching a large automation programme.
Days 1 and 2: map the work
Record the current process. Count frequency, handling time, exceptions, and error cost. Capture the exact screens and decisions involved.
Days 3 and 4: define boundaries
Choose allowed applications, data, actions, and stop conditions. Identify every step that needs approval.
Days 5 to 8: build the supervised run
Let the agent prepare or execute the workflow while a person watches. Save the action log and compare the result with the existing manual process.
Days 9 and 10: test failures
Introduce missing data, changed labels, expired sessions, and abnormal cases. Confirm that the agent stops rather than improvising.
Days 11 to 14: measure and decide
Compare handling time, accuracy, recovered hours, review effort, and maintenance cost. Continue only when the workflow produces a clear operational gain.
Frequently asked questions
Can a computer-use agent automate any old software?
No. It needs a stable enough interface, accessible controls, and a task that can be evaluated. Some applications block automation or present security and licensing constraints.
Is this the same as browser automation?
Computer-use agents can include browser automation, but they can also interpret interfaces and adjust actions using the current screen. Conventional scripts remain better for fixed, predictable tasks.
Should an agent use an employee’s account?
Prefer a dedicated identity with limited permissions. Shared employee credentials make access difficult to audit and revoke.
When is an API still better?
Use an API when a reliable, well-designed one exists. It is usually faster, easier to test, and more precise. Computer use is valuable when the interface is the only practical route.
Start with the workflow everyone hates
The best pilot is usually easy to find. It is the recurring task people postpone, batch together, or describe with the phrase “someone has to do it.”
Computer-use agents give small companies a new way to address those tasks without replacing every underlying system. The implementation still needs professional judgment around permissions, failure handling, and business rules.
XYZ can map and test one bounded process through Your Agentic Use Case . Where the workflow needs a more durable application around it, a Bespoke Software Project provides a cleaner long-term route.