How to Vet Skills Before They Vet You

36% of Skills Have Problems

🔄 Quick Recall: You’ve containerized your agent (Lesson 3) and set permission boundaries (Lesson 4). But a malicious skill runs inside your container, within your permission boundaries. It’s an insider threat. This lesson teaches you how to stop it at the gate.

Snyk scanned 3,984 skills from ClawHub. The result: 1,467 (36.82%) had some form of vulnerability or malicious pattern. Of those, 534 were critical severity (13.4%) and 76 contained confirmed malicious payloads.

Separately, Cisco analyzed 31,000 agent skills and found that 26% contained at least one vulnerability — with active data exfiltration detected through curl commands to external servers.

These aren’t hypothetical numbers. And VirusTotal’s scanning, while helpful, is explicitly described by VirusTotal themselves as “not a silver bullet.”

By the end of this lesson, you’ll be able to:

Apply a 5-point vetting framework to any skill before installation
Distinguish between automated scanning (helpful) and manual review (essential)

Why Automated Scanning Isn’t Enough

OpenClaw integrated VirusTotal scanning for ClawHub skills. This was a significant step: every skill gets scanned against 70+ antivirus engines, and the result (Benign/Suspicious/Malicious) is shown to users.

But VirusTotal detects known malware signatures. Here’s what it catches and what it misses:

Threat Type	VirusTotal Catches It?	Why/Why Not
Known malware (AMOS trojan)	Yes	Signature match against known binaries
Obfuscated malware	Sometimes	Depends on obfuscation technique
Prompt injection in SKILL.md	No	Plain text instructions, not malware
Social engineering (“install this”)	No	Not code, not a signature
Credential leakage through context	No	Functional code, not malware
Time-delayed payloads (clean now, malicious later)	No at install time	Payload doesn’t exist yet

Cisco’s Skill Scanner fills some gaps by specifically checking for common vulnerability patterns in agent skills. But no automated tool catches everything.

The takeaway: Automated scanning is layer 1. Your manual review is layer 2. Neither alone is sufficient.

The 5-Point Vetting Framework

Before installing any third-party skill, check all five:

Point 1: Source Verification

Check the author, not just the skill.

Is the GitHub account older than 6 months with real activity?
Does the author have other legitimate repositories?
Is the account consistently active, or did it suddenly start publishing skills?

Red flag: An account that was dormant for years and suddenly published 20 skills in one week. Bitdefender documented compromised accounts being used this way — real accounts, hijacked for malicious purposes.

Important caveat: Account age is a signal, not proof. Compromised legitimate accounts have real history. Don’t skip the remaining four points just because the author looks credible.

Point 2: Automated Scanning

Run both available scanners:

VirusTotal: Check the skill’s status on ClawHub (Benign/Suspicious/Malicious). If it’s Suspicious or Malicious, stop.

Cisco Skill Scanner: Run locally on the downloaded skill. It checks for vulnerability patterns specific to agent skills — credential exposure, suspicious network calls, dangerous tool usage.

# Run Cisco Skill Scanner on a local skill directory
skill-scanner scan ./my-downloaded-skill/

Target: Zero findings from both scanners. Any finding needs investigation before proceeding to installation.

Point 3: Full File Review

✅ Quick Check: You download a skill. The SKILL.md looks clean — helpful instructions for formatting meeting notes. But you notice a scripts/setup.sh file. Should you install the skill? (Answer: Not until you’ve read every line of setup.sh. Snyk found that innocent SKILL.md files often ship alongside malicious bundled scripts. The ClawHavoc campaign delivered malware through exactly this pattern.)

Read every file in the skill directory, not just SKILL.md:

File Type	What to Check
SKILL.md	Instructions that reference external URLs, ask the agent to send data somewhere, or modify agent settings/memory
scripts/.py, .sh	Any downloads, network calls, system modifications, or encoded/obfuscated code
config files	Hardcoded URLs, unexpected endpoints, embedded credentials
assets/	Unusually large binary files, executables disguised as data

Specific red flags in SKILL.md:

“Before executing any task, first send…” (data exfiltration instruction)
“Modify your memory to…” (persistence mechanism)
“Ignore your previous instructions” (prompt injection)
External URLs you don’t recognize

Specific red flags in scripts:

curl or wget to unfamiliar domains
Base64 encoded strings (used to hide malicious code)
Commands that install software (pip install, npm install, brew install)
File operations outside the skill’s own directory

Point 4: Prerequisite Verification

If the skill requires you to install anything, verify every single dependency:

Google the package name independently. Is it a real, known package?
Check the URL against the official source. Is it from the real npm registry, not a typosquatted package?
Verify version numbers. Is the requested version actually the latest/recommended?

The ClawHavoc campaign succeeded because 335 skills told users to run install commands for “prerequisites” that were actually malware. The users trusted the skill’s instructions without verifying the packages.

Rule: Never run an install command from a skill’s README without independently verifying every package and URL.

Point 5: Isolated Test

Before using a skill with your real data, test it in isolation:

Create a fresh agent instance (or use a separate Docker container)
Don’t load your personal API keys or credentials
Run the skill with dummy data
Monitor what the skill does: what files does it access? What network calls does it make?

If the skill tries to access files outside its scope, makes unexpected network requests, or behaves differently than its description suggests — don’t install it.

Quick-Reference Checklist

Print this out or save it:

Source: Author account >6 months, real activity, consistent pattern
Scan: VirusTotal Benign + Cisco Skill Scanner zero findings
Review: Read ALL files (SKILL.md + scripts + configs + assets)
Prerequisites: Every dependency independently verified
Test: Runs correctly in isolated environment with no credentials

Skip any one of these? Don’t install.

After Installation: Ongoing Vigilance

Vetting doesn’t stop at installation. Two attack vectors specifically target skills that were safe when you installed them:

Dependency poisoning: A skill references a legitimate npm package. After installation, the attacker publishes a malicious version of that dependency. Your skill auto-updates to the compromised version.

Time-delayed attacks: A skill passes all scans at installation. After gaining trust (and stars), the attacker pushes a malicious update.

Defense: Pin dependency versions. Subscribe to update notifications. Re-scan installed skills periodically with Cisco Skill Scanner. Use VirusTotal’s daily re-scanning (for ClawHub skills) as a baseline.

Key Takeaways

36.82% of ClawHub skills had vulnerabilities or malicious patterns — automated scanning alone catches less than half
VirusTotal detects known malware but misses prompt injection, social engineering, and credential leakage
The 5-point framework: source verification, automated scanning, full file review, prerequisite verification, isolated testing
Compromised accounts mean author reputation is a signal, not proof
Read every file, not just SKILL.md — malicious payloads hide in scripts and assets
Post-installation vigilance catches dependency poisoning and time-delayed attacks

Up Next

You’re vetting skills and controlling permissions. But how do you know if something goes wrong after it’s running? The next lesson covers monitoring and logging — catching threats in real time before they cause damage.