Skill Auditor

### Quick judgment Skill Auditor is a **mature, high-coverage security scanner for OpenClaw skills**. It offers a strong baseline with zero dependencies and several powerful opt‑in layers (Python AST dataflow, VirusTotal, LLM semantic analysis, SARIF output). It’s particularly well-suited for **developers and security-conscious operators** who want structured, automatable defenses against malicious or over-privileged skills. ### Capabilities and strengths - **Platform-aware security scanning** - Detects common code-level threats: prompt injection, data exfiltration, shell execution, path traversal, obfuscation, persistence, privilege escalation. - Includes **OpenClaw-specific checks** (MEMORY.md access, sessions tool abuse, gateway control commands, etc.), which is critical for realistic threat coverage in this ecosystem. - Supports **multiple detection modes** (`strict`, `balanced`, `permissive`) to trade off sensitivity vs. false positives. - **Layered analysis design** - **Core static scanner (regex + intent matching)** requires only Node.js and works out-of-the-box, good for CI use and broad compatibility. - **AST dataflow analysis (Python)** traces flows like env vars → HTTP requests, file reads → network sinks, and cross-file flows, improving detection of subtle exfiltration. - **VirusTotal integration** scans binaries against many AV engines, adding important coverage for compiled/machine code. - **LLM semantic analysis** cross-checks findings against the skill’s description, allowing severity adjustment when behavior is legitimate for the stated purpose. - **Strong integration story** - **CLI tools** for scanning single skills, all installed skills, or remote GitHub URLs. - **SARIF output** and `--fail-on-findings` make it easy to plug into GitHub Code Scanning and block risky changes in CI. - JSON and visual reports support both automation and human review workflows. - **Operational usability** - Interactive **setup wizard** detects OS, checks Python, offers tree-sitter installation, configures auto-scan, and persists settings. - Helpful **troubleshooting guidance** and debug mode (`DEBUG=1`). - Backward-compatible with v1.x commands, reducing friction for existing users. ### Risks, limitations, and caveats - **Detection is not exhaustive** - Core scanner is pattern-based and may miss **novel obfuscation**, sophisticated prompt injection, or non-Python dataflow. - AST analysis is currently **Python-only** and described as “basic dataflow,” so complex or multi-language flows may not be fully captured. - Binary scanning requires VirusTotal; without it, executable payloads may be skipped. - **External service implications** - **VirusTotal**: binaries are uploaded to a third-party service; this can expose proprietary or sensitive code. This should be enabled only where that trade-off is acceptable. - **LLM semantic analysis** requires an OpenClaw gateway and network access; depending on deployment, this can introduce **privacy and latency** considerations. - VirusTotal is also **rate-limited**, which may affect large-scale scans. - **False positives vs. false negatives** - `strict` mode intentionally favors detection over precision, which can generate more false positives and may be noisy in CI unless carefully tuned. - `permissive` mode reduces noise but focuses only on critical patterns, increasing the chance of missing subtler threats. - **Scope and dependency constraints** - Advanced features require additional tooling: - Python 3.8+ and tree-sitter for AST analysis. - VirusTotal API key for binary scanning. - Running OpenClaw gateway for LLM semantic analysis. - YARA rule framework is present, but custom rule support is described as **not fully implemented**, so rule-based extensibility is still evolving. ### Recommended scenarios Skill Auditor is **recommended** for: - **Skill developers** who want to automatically check their own skills for security issues before publishing or deploying. - **Platform operators / security teams** running OpenClaw in production, especially in environments with sensitive data or high-privilege skills. - **CI/CD pipelines** enforcing security gates on new or updated skills (via SARIF + `--fail-on-findings`). - **Pre-install checks** of third-party or untrusted skills, including scanning directly from GitHub URLs. It is **less ideal as the only line of defense** in: - Environments relying heavily on non-Python languages for critical logic (since AST dataflow is Python-only). - Highly sensitive IP contexts where uploading binaries to VirusTotal is not acceptable (unless VirusTotal is disabled). Overall, Skill Auditor is a **strong, well-integrated security tool for the OpenClaw ecosystem**, with high functionality and good usability, provided users understand the limitations of pattern-based scanning and the privacy trade-offs of optional external integrations.