Google Gemini adds “Select from screen” feature in Chrome

Google added the “Select from screen” feature to Gemini in Chrome 149, enabling users to highlight any text or image on a webpage and send it directly to the AI assistant for contextual responses [1]. The move aims to reduce friction and make Gemini act less like a standalone chatbot and more like an on‑screen assistant that understands exactly what the user is looking at.

Integrated AI assistance in Chrome

The new tool appears in Gemini’s “+” menu and works like a built‑in screenshot utility: after activation, a user draws a box around any visible content, which is automatically attached to the Gemini prompt. This eliminates the need to manually describe a page’s contents, letting the model focus on the exact element the user wants help with. Google is positioning the feature as part of a broader push to embed Gemini across its products, moving the assistant from a pure chat interface toward a context‑aware helper that can act on what users are actively doing on the web.

Broader Gemini upgrades and competitive context

The “Select from screen” launch coincides with the release of Gemini 3.5 Flash, which gives developers direct computer‑use capabilities—allowing AI agents to see, reason, and act across browsers, mobile apps, and desktops without a separate model. Google says this improves long‑horizon tasks such as software testing and enterprise workflow automation [1]. Compared with earlier Gemini versions that required users to type or speak their queries, the new multimodal workflow mirrors moves by rivals like Microsoft’s Copilot, which also integrates screen‑capture functions into its AI suite. By tying Gemini more tightly to Chrome, Google seeks to differentiate its assistant through seamless, in‑browser context rather than relying solely on external plugins or separate apps.

What to watch

• Full rollout timing – Google has begun a phased rollout of Chrome 149; monitoring when the feature becomes universally available will indicate adoption speed.
• Developer uptake of Gemini 3.5 Flash – Early usage metrics for the new computer‑use APIs will reveal whether enterprises adopt Gemini for automation tasks.
• Competitive responses – Watch for announcements from Microsoft, Anthropic, or other AI vendors that may introduce similar screen‑aware capabilities to counter Google’s integration.

By letting users point Gemini at exact on‑screen content, Google blurs the line between browsing and AI assistance, potentially reshaping how everyday web tasks are completed and setting a new baseline for contextual AI in browsers. The open question is whether this tighter integration will translate into measurable productivity gains for users and developers alike.