Microsoft has launched a new feature called Copilot Vision, which allows users to ask the AI-powered assistant questions about websites they’re visiting using Microsoft Edge. The tool can analyze text and images on web pages to answer queries like “What’s the recipe for this lasagna?” It also provides features such as summarizing and translating text, handling discounted products, and serving as a game assistant.
The feature is limited to users who have opted-in to Copilot Labs, an experimental AI program that costs $20 per month. To use Copilot Vision, users simply need to enable it in the Edge browser. Microsoft claims that the tool will delete data after each session, but it’s unclear how long this data is retained.
The feature has some restrictions, such as not being able to view paywalled or “sensitive” content. Microsoft won’t disclose what types of content are considered sensitive, citing legal disputes with news outlets. The company says it will add more websites over time and respect sites’ machine-readable controls on AI.
Microsoft is taking a cautious approach due to concerns from publishers about data usage and server costs. However, the company remains committed to addressing these concerns through user feedback. For now, Copilot Vision may not be available on top news sites, but Microsoft plans to continue working with third-party publishers to improve its capabilities.
Source: https://techcrunch.com/2024/12/05/copilot-vision-microsofts-ai-tool-that-can-read-your-screen-launches-in-preview