ClawTrust LogoClawTrust
mac-use

mac-use

by kekejun · v1.0.0

Data Analysis
ClawHub
6.7
/ 10
1 evaluations
1.9k Downloads

Overview

Visually control macOS desktop applications by taking annotated screenshots, locating UI elements via Apple Vision OCR, and then interacting with them through scripted clicks, scrolling, typing, and key presses.

Key Advantages

1.True GUI-level control for macOS apps that lack APIs or plugins, including web, native, and cross-platform apps.
2.Text-aware interaction via OCR with numbered annotations, making it easier to target buttons, links, and labels by stable identifiers (element numbers).
3.Rich interaction set: list windows, screenshot, click by element number, coordinate-based click, scroll, type (with Unicode/CJK support), and key/key-combo presses.
4.Clear, prescriptive workflow (screenshot → inspect → act → rescreenshot) designed to reduce mis-clicks and improve verifiability of actions.
5.Uses on-device Apple Vision OCR and built-in macOS tooling, avoiding external OCR services and reducing latency and privacy concerns.

Use Cases

  • Automating routine workflows in macOS apps that don’t provide APIs (e.g., WeChat, messaging clients, proprietary enterprise tools).
  • Driving web flows in browsers like Chrome or Safari when browser-automation skills are unavailable or insufficient.
  • Filling out forms or navigating menus in GUI-only desktop applications (installers, preference panes, internal tools).
  • Interacting with mini-programs or embedded webviews inside container apps (e.g., WeChat mini-programs, Electron-based tools).
  • Performing moderate-complexity GUI tasks that require visual confirmation, such as verifying a successful navigation or state change via rescreenshots.

Evaluation Scores

6.7
/ 10
Reliability
6.5
Functionality
8.0
Usability
7.0
Safety
6.0
Performance
7.5
Compatibility
4.5

Based on 1 evaluation · Latest: 3/19/2026

Download Trend

Loading...

Evaluation History (1)

6.7/103/19/2026
▼
OS: linux-arm64LLM: google/gemini-3.1-pro-preview
**Judgement:** A capable, visually-driven macOS-only GUI automation skill. Strong choice when you must control Mac desktop apps via their graphical interface, but tightly constrained to macOS with Apple Vision and requires disciplined use. **Recommended scenarios** - When the user explicitly wants to interact with a Mac desktop app’s GUI (click buttons, scroll views, type into fields, trigger shortcuts). - When no API-specific skill exists for the target app or website, and visual interaction is acceptable. - For workflows where you can safely verify every step with before/after screenshots and where occasional OCR misses or layout changes are acceptable. **Key risks / limitations** - **Platform lock-in:** Works only on macOS with Apple Vision and the specified Python tooling installed; unusable on other OSes. - **OCR and layout fragility:** Reliant on text detection; icon-only or stylized controls may require manual coordinate clicks and can be brittle if the UI changes. - **Potential for unintended actions:** Mis-targeted clicks or keypresses (especially with global shortcuts) can trigger destructive actions if used in high-stakes contexts. - **Sensitive data handling:** Typing is clipboard-based; it should not be used for passwords or secrets, and any such use would increase security risk. Use this skill when you need general-purpose, verifiable Mac GUI control, and avoid it for non-macOS environments, security-critical operations, or highly sensitive data entry.

Comments (0)

Post a Comment

No comments yet. Be the first!