Microsoft is expanding its Copilot app on Windows by allowing users to share their screen or apps with the AI assistant. The new feature, called Copilot Vision, was initially limited to the Edge browser but is now being rolled out to all apps on PCs. This update will enable users to receive guidance through popular software like Adobe Photoshop, while also analyzing photos and webpages.
In an early test of Copilot Vision, Microsoft demonstrated its ability to coach users through various tasks. The AI assistant successfully guided a user through a game of Minecraft and optimized settings in the video editor Clipchamp. However, this feature is currently only available for US testers through the Insider beta due to limitations.
The latest update also introduces file search capabilities within Copilot on Windows. Users can ask the AI assistant about the contents of files on their PC, with support for popular document formats such as .docx, .xlsx, and .pdf. These features require only the basic Copilot app on Windows and do not necessitate a full subscription.
Microsoft is testing these new Copilot features with Windows Insiders ahead of a broader rollout to all Windows 11 users in the coming weeks or months.
Source: https://www.theverge.com/news/645666/microsoft-copilot-vision-windows-beta-testing