Microsoft has announced a new interactive skill for its Copilot Studio product, enabling AI agents to conduct research on the web and interact with websites like humans do. The skill, called “computer use,” allows agents to click buttons, select menus, and fill out forms on screens, even if no API is available.
The feature uses mainstream browsers, such as Microsoft Edge, Chrome, and Firefox, and runs on a backend hosted by Microsoft. Agents can adapt to changes in desktop apps and websites, making them more efficient and reducing the need for manual intervention.
To build an AI agent, users don’t need programming or coding skills. Instead, they simply describe what they want the agent to do using natural language at the prompt in Copilot Studio. The agent’s actions are then simulated in a sandbox mode before being deployed.
The new skill has several potential use cases, including automated data entry, market research, and invoice processing. Microsoft has invited users to sign up for early access to test the feature, which is now available through a limited preview.
With this innovation, AI agents can take on more complex tasks, but it’s essential to remember that today’s AI is not perfect and may make mistakes. As with any new technology, it’s crucial to test and fine-tune the skill to ensure its reliability and effectiveness.
Source: https://www.zdnet.com/article/with-copilot-studios-new-skill-your-ai-agent-can-use-websites-and-apps-just-like-you-do