Google announced the general availability of the Computer Use tool in the Gemini API on October 30, 2026, through the Interactions and GenerateContent interfaces. The feature lets developers build AI agents that interpret screenshots and propose actions in a browser, mobile applications or a desktop environment. The application built by the developer executes those actions.
The release concerns developer infrastructure. General availability of the tool is separate from the features offered to people using the Gemini app. Computer Use fits application development and automation projects that require interaction with graphical interfaces: filling forms, entering repetitive data, checking website user journeys or retrieving product information.
According to Google, the execution environment must be implemented within the application using the API. The model receives the user's objective, the tool configuration and an image of the current state. Its response can contain a function call for a click, a scroll or text entry. The application code takes that proposal and carries out the permitted operation.
After each action, the environment produces a new screenshot, which is sent to the model together with the execution result. The next decision therefore uses the updated interface. This cycle continues until the task finishes or the process stops. The documentation requires the conversation history to retain both model responses and the results of executed functions.
Three environments and several compatible models
The list of compatible models includes Gemini 4 Argon, Gemini 3.8 Flash, Gemini 3.5 Flash-Lite and Gemini 3 Flash Preview. Google recommends Gemini 3.8 Flash for use with the tool. General availability describes Computer Use itself; each model retains its own name and release status, including the Preview label on the model that carries it.
In the browser, the documented operations include single or multiple clicks, scrolling, navigating to an address, returning to the previous page, typing and keyboard shortcuts. Dragging an element and taking a screenshot are also supported. These are instructions for the execution environment, which the developer connects to the application's own automation mechanism.
The mobile environment provides actions optimized for Android. These include opening an application by name, listing available applications, performing a long press, entering text and returning to the previous screen. The desktop configuration offers cursor and keyboard operations, including pressing and releasing buttons or keys and moving elements between coordinates on the screen.
For Gemini 4 and Gemini 3.x models, actions include a field explaining the intent behind the proposed step. The application therefore receives both the operation and the reason the model supplies for choosing it. Normalized coordinates must be mapped to the actual screen size. Google's examples use Playwright to execute actions in the browser.
The current documentation specifies version 2.7.0 or later of the google-genai Python library for its examples. It also presents variants for other languages. The components described cover request configuration, response interpretation and transmission of the new state. Developers assemble them into a multistep agent and can add their own functions or exclude predefined actions.
Separate confirmations for sensitive operations
A safety assessment system also participates in the cycle. Its response can allow an action, request user confirmation or block the step. When confirmation is required, the application obtains it from the person and returns that acknowledgement in the function result. A blocked action requires the application to stop execution or handle the interruption.
Google lists separate categories for financial transactions, sensitive data modification, communication tools and account creation. Other categories cover changes to files and data, consent management and the acceptance of legal terms. These distinctions include, for example, payments, sending messages, deleting information or interacting with prompts concerning privacy and the user's consent.
Policies are configurable, and selected behaviors have override options. The documentation states that the system can still request confirmation even when a developer has supplied such an option. Handling safety decisions therefore remains part of the implementation. The application's own instructions can further specify which actions require human involvement before they can proceed.
The tool also includes an optional check of screenshots to detect hostile instructions hidden in displayed content. Google calls this type of attack prompt injection: a page attempts to redirect the agent with commands unrelated to the user's task. The check is disabled by default and must be explicitly enabled. When it detects such an attack, the mechanism blocks execution.
For the agent's execution environment, Google recommends isolation in a virtual machine, a container or a browser profile with restricted permissions. Lists of permitted or prohibited destinations define where the system can navigate. Technical records should cover requests, screenshots, proposed actions, safety decisions and the operations that the client application actually carried out.
From the 2025 preview to the current tool
Google introduced a specialized Gemini 2.5 Computer Use model in October 2025, when it was available in preview. The original announcement described automation of forms and interactive elements, alongside testing of web interfaces. At that stage, the company specified that it was primarily optimized for browsers. The current documentation describes browser, mobile and desktop environments separately.
Public examples from 2025 included organizing elements on a visual board and transferring information into a demonstration application for appointments. They illustrated operations carried out sequentially through an interface. The October 30, 2026 announcement establishes the tool's new release status in the two APIs, on which developers can build their own workflows and execution systems.
For important tasks, the documentation retains its recommendation for human supervision and advises avoiding critical decisions, sensitive data and actions whose serious errors cannot be corrected. Control applies to the specific operation: the model proposes a step, the system checks permission, and the application executes the action and records the resulting state.
