Computer Use Automation

ПродвинутыйdevelopmentМинимальный контекст: 64K

Designs and drives computer-use agents that operate a desktop or browser directly via screenshots and UI actions (click, type, scroll, navigate). Plans multi-step GUI workflows, handles verification and recovery between steps, and structures guardrails for safe autonomous execution. Built for models exposing computer use as a native client-side tool.

computer-usebrowser-automationgui-automationai-agentsagentic-workflowstool-useautomation

Варианты использования

  • Automating repetitive web form submissions across internal tools
  • Driving a browser to gather and reconcile data from multiple dashboards
  • Building an agent that navigates a legacy desktop app without an API
  • Running end-to-end UI flows for QA and regression checks
  • Orchestrating high-throughput agentic search and document processing

Пример промпта

Design a computer-use agent that reconciles invoices between a web ERP and a spreadsheet.

The agent should:
1. Open the ERP dashboard and locate the pending invoices view
2. For each invoice, read the amount and vendor from the screen
3. Cross-check the value against the corresponding spreadsheet row
4. Flag mismatches in a report and never modify records without confirmation

Provide:
- The action loop (observe screenshot -> plan -> act -> verify)
- Guardrails for destructive or irreversible actions
- Recovery logic when an expected UI element is missing
- A stopping condition and human-in-the-loop checkpoints

Рекомендуемые модели

gemini-3-6-flashgemini-3-5-flash-litegemini-3-5-proclaude-opus-4-7

Совместимые инструменты

gemini-cliclaude-codekiroany

Модальности

Вход: text, image
→
Выход: text, code

Похожие Skills

ai-agent-buildermulti-agent-orchestratorbrowser-test-debugging

Автор

OpenModels Community

@openmodelsrun