yilsem/aiturk-hermes-ide
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 6 Wiki Activity

Import AITURK IDE 1.0.0-beta.1 from Hermes 63279301; preserve MIT license

Browse Source
This commit is contained in:
yilsem
2026-09-05 13:26:46 +03:00
commit 03634b1ca3
11340 changed files with 3442369 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
optional-skills/software-development/ast-grep/LICENSE
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Yeongyu Kim
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
optional-skills/software-development/ast-grep/SKILL.md
+288
View File
@@ -0,0 +1,288 @@
---
name: ast-grep
description: "AST-aware structural code search and rewrite via ast-grep."
version: 1.0.0
author: Yeongyu Kim (code-yeongyu), adapted by Hermes Agent
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [ast, codemod, refactoring, structural-search, code-search, rewrite, tree-sitter]
category: software-development
related_skills: [simplify-code, systematic-debugging]
---
# ast-grep
`ast-grep` (binary also named `sg`) is an **AST-aware search and rewrite tool** across 25 languages. It treats your pattern as code, parses it the same way it parses your project, and matches structurally. It is the right tool whenever your question depends on **code shape** rather than text bytes.
This skill ships a Python wrapper at `scripts/ast_grep_helper.py` and platform install scripts at `install.sh` (POSIX) and `install.ps1` (Windows). The helper adds offline pattern validation, the two-pass write trick, and binary auto-resolution. Use it as your default entry point.
Upstream source: vendored from [code-yeongyu/ast-grep-skill](https://github.com/code-yeongyu/ast-grep-skill) (MIT), as shipped in oh-my-openagent's shared-skills bundle.
---
## When to use this skill
Use it whenever the question is about **code structure**, not bytes:
- "Find every function that takes a `Request` parameter."
- "Rewrite every `console.log(x)` to `logger.info(x)`."
- "Strip every `as any` cast."
- "Replace `require(...)` with `import` across the repo."
- "Find empty catch blocks."
- "Migrate `Optional[X]` to `X | None`."
- "Apply this codemod across these 200 files."
- "Run our YAML lint rules and surface violations."
Switch to `search_files` (or plain `rg`) when the question is text-shaped (string literal contents, comments, license headers, file names, cross-language regex). When in doubt, ask: "does the answer depend on the language's syntax tree, or just on the file's bytes?" If the former, ast-grep. If the latter, search_files.
Hermes integration notes:
- Run the helper and `sg` through the `terminal` tool. Single-quote every pattern so the shell never expands `$VAR`.
- For find→read chains around matches, use `--json-out` and process with `execute_code` rather than piping through interpreters.
- This complements (does not replace) Hermes's `patch` tool: `patch` is for targeted edits you author; ast-grep is for pattern-driven bulk rewrites across many sites.
---
## Three things the agent must internalize
### 1. ast-grep is NOT regex
The wildcards are `$VAR` (one AST node) and `$$$` (zero or more nodes). Regex syntax fails silently:
| You wrote | What ast-grep saw | What you wanted |
|---|---|---|
| `foo\|bar` | bitwise-or of `foo` and `bar` | run two separate searches |
| `.*foo` | not parseable | `$$$ foo` (if `$$$` is a list of nodes) or use rg |
| `\w+` | not parseable | `$VAR` to capture any identifier |
| `[a-z]` | character class, not parseable | switch to rg |
The full anti-pattern table is in `references/pitfalls.md` §1. The helper's `validate` subcommand catches these mechanically — call it before debugging "no matches" by hand.
### 2. Patterns must be valid code
The pattern itself must parse. `def $FN($$$):` fails because the trailing `:` makes it incomplete; use `def $FN($$$)`. `function $NAME` without params/body fails; use `function $NAME($$$) { $$$ }`. Full table per language in `references/pitfalls.md` §2.
### 3. `--update-all` and `--json` are mutually exclusive (silently)
This is the single biggest gotcha when scripting. `sg run -p P -r R --json --update-all` returns the JSON but **does not mutate files**. To both preview AND apply, run **two passes**:
```bash
sg run -p P -r R --json=compact . # pass 1: see what would change
sg run -p P -r R --update-all . # pass 2: actually apply
```
The helper does this automatically when you call `replace --apply`. Read `references/pitfalls.md` §9.
---
## The helper script — `scripts/ast_grep_helper.py`
A single-file Python 3 stdlib wrapper. Same on every OS. The agent's default entry point.
### `search` — find all matches of a pattern
```bash
python scripts/ast_grep_helper.py search 'console.log($MSG)' --lang ts src/
```
Validates the pattern offline first. If the pattern looks like regex (`\w`, `.*`, `|`, etc.) the helper exits with a hint and never calls `sg` — saves a round-trip. Pass `--force` to skip validation.
Flags:
- `--lang ts` (or any of the 25 languages; aliases like `js`, `py`, `rs`, `kt` accepted)
- `--globs '!**/*.test.ts'` (repeatable; prefix `!` to exclude)
- `-C 3` (context lines)
- `--json-out` (raw JSON instead of human format)
### `replace` — rewrite by pattern, dry-run by default
```bash
# Dry-run preview (default — no files mutated)
python scripts/ast_grep_helper.py replace 'console.log($MSG)' 'logger.info($MSG)' --lang ts src/
# Actually apply
python scripts/ast_grep_helper.py replace 'console.log($MSG)' 'logger.info($MSG)' --lang ts src/ --apply
```
The helper:
1. Validates both `pattern` and `rewrite` for hint-detectable mistakes.
2. Runs pass 1 with `--json=compact` to collect matches and show a preview.
3. If `--apply` is set, runs pass 2 with `--update-all` to mutate files.
### `scan` — run YAML rules
```bash
# Discover sgconfig.yml from cwd and run all rules
python scripts/ast_grep_helper.py scan src/
# Run a single rule file
python scripts/ast_grep_helper.py scan -r rules/no-console.yml src/
# Apply auto-fixes
python scripts/ast_grep_helper.py scan -U src/
# CI-friendly GitHub annotations
python scripts/ast_grep_helper.py scan --report-style short src/
```
### `validate` — offline pattern check (no `sg` call)
Useful for CI lints, pre-commit hooks, and quick sanity checks:
```bash
python scripts/ast_grep_helper.py validate '\w+' --lang ts
# → exit 2: regex \w not supported. Use $VAR for identifiers.
python scripts/ast_grep_helper.py validate 'console.log($MSG)' --lang ts
# → exit 0: pattern looks plausible for ast-grep.
```
### `langs` / `doctor` / `install`
```bash
python scripts/ast_grep_helper.py langs # list 25 supported languages and aliases
python scripts/ast_grep_helper.py doctor # check ast-grep binary availability
python scripts/ast_grep_helper.py install # delegate to install.sh / install.ps1
```
`new` and `test` subcommands proxy directly to `sg new` and `sg test`.
---
## Direct `sg` use (when the helper isn't enough)
The helper is opinionated. For full control, drop to `sg`. The skill ships a CLI cheat sheet in `references/cli.md`. The minimal idioms:
```bash
# Search
sg run -p 'console.log($MSG)' --lang ts src/
# Search with JSON for scripting
sg run -p 'console.log($MSG)' --lang ts --json=compact src/
# Rewrite, dry-run
sg run -p 'console.log($MSG)' -r 'logger.info($MSG)' --lang ts --json=compact src/
# Rewrite, apply
sg run -p 'console.log($MSG)' -r 'logger.info($MSG)' --lang ts --update-all src/
# Pattern from stdin (great for ad-hoc experiments)
echo 'console.log("hi")' | sg run -p 'console.log($MSG)' --lang js --stdin
# Debug a pattern that returns 0 matches
sg run -p '<your pattern>' --lang <lang> --debug-query=ast --stdin <<< '<sample-code>'
# Run YAML rules
sg scan src/
# Inline YAML rule (one-off)
sg scan --inline-rules '
id: no-todo
language: TypeScript
severity: warning
rule: { pattern: TODO }' src/
```
When using `sg` directly in a shell, **always single-quote patterns** so `$VAR` is not expanded by the shell.
---
## Decision tree — what to use, when
```
USER asks for "find/rewrite/codemod"
│
├─ structural pattern (function shape, call, class, import, control flow)
│ └→ ast-grep (this skill)
│
├─ text pattern (regex, alternation, character classes, file names)
│ └→ search_files / rg
│
├─ semantic question (what variable does this refer to? does this throw?)
│ └→ LSP tools, TypeScript compiler, Pyright, Semgrep with type inference
│
└─ multiple repos / federated search
└→ a search engine + then ast-grep / rg / LSP per-repo
```
If the user says "find all" or "every", default to ast-grep when the target is shaped (function, class, call, import, statement). Default to search_files when the target is text (string content, comment, license header, file name, identifier substring).
---
## Always run dry-run first when rewriting
A bad pattern silently rewrites the wrong thing. The helper's `replace` defaults to dry-run for this reason. The flow is:
1. Search to confirm matches: `helper search '<pattern>' --lang X .`
2. Dry-run rewrite: `helper replace '<pattern>' '<rewrite>' --lang X .` (no `--apply`)
3. Inspect the dry-run summary: number of matches, files affected, the per-location preview.
4. If wrong: refine pattern, go back to step 1.
5. If right: `helper replace '<pattern>' '<rewrite>' --lang X . --apply`.
Never apply a rewrite that you have not first dry-run. After an `--apply` in a git repo, review with `git diff --stat` before committing.
---
## When `sg` returns 0 matches but you know the code is there
In priority order:
1. **Run `helper validate '<pattern>' --lang <lang>`** — catches regex misuse, missing function bodies, Python trailing colons.
2. **Check `--lang`** — `sg` infers from extension; if you pass a `.tsx` file with `--lang ts` (not `tsx`), JSX won't parse.
3. **Inspect the parsed pattern**: `sg run -p '<pattern>' --lang <lang> --debug-query=ast --stdin <<< '<sample>'`. If it shows `ERROR` nodes, the pattern is malformed.
4. **Check the AST of the target file**: `sg run -p '$_' --lang <lang> --debug-query=cst path/to/file | head -40` — find the `kind` you're trying to match.
5. **Try the playground**: <https://ast-grep.github.io/playground.html> — paste code + pattern, see what's happening.
Do not blindly retry with variations. Each failure has a reason; surface it.
---
## When to use YAML rules vs inline `-p` patterns
**Use inline `-p`** when:
- One-off ad-hoc query.
- The pattern is simple (no constraints, no fix template).
- You're exploring.
**Use YAML rules** (file under `rules/`, run via `sg scan`) when:
- The pattern is reused (lint rule, codemod that runs in CI).
- You need `constraints`, `transform`, complex `inside`/`has`, or composite logic.
- You want auto-fix (`fix:` field).
- You want to test the rule (snapshot tests via `sg test`).
The full YAML rule schema is in `references/yaml-rules.md`. Project setup (`sgconfig.yml`, `ruleDirs`, `utilDirs`) is in `references/sgconfig.md`.
---
## Output discipline
- `sg run --json=compact` produces an array of match objects: `{ file, range: {start, end}, text, replacement?, lines, language, ... }`.
- Without `--json`, `sg` produces human-readable colored output suitable for terminals.
- The helper's default output is human-readable (file:line:column + match preview). Pass `--json-out` for raw JSON.
- The helper's `replace` always summarizes: number of matches, number of files, per-location preview.
When summarizing for the user, **always include the count of files affected**, not just the count of matches. Users care about blast radius.
---
## Required reading (in order of priority)
1. `references/patterns.md` — meta-variables, naming rules, strictness levels. Read when you're unsure why a pattern doesn't match.
2. `references/pitfalls.md` — the failure-mode field guide. Read when 0 matches surprises you.
3. `references/recipes.md` — copy-paste patterns by language. Read first when you start a new task.
4. `references/cli.md` — `sg run`, `sg scan`, `sg test`, `sg new`, `sg lsp`. Read when the helper isn't enough.
5. `references/yaml-rules.md` — YAML rule schema. Read when you outgrow inline patterns.
6. `references/sgconfig.md` — project-level configuration. Read when you set up `sg scan` for a real project.
7. `references/install.md` — per-OS install methods. Read only if `install.sh` / `install.ps1` fail.
---
## Invariants (do not break)
- **Validate before searching.** When emitting a pattern programmatically, call `helper validate` first. It catches the regex-misuse class of mistakes that account for ~70% of "0 matches" debug sessions.
- **Dry-run before applying.** Never run `sg run -r ... --update-all` without first inspecting the matches. The helper's `replace` enforces this by default.
- **Two-pass writes.** When using `sg` directly to both preview and apply, run two invocations — `--json` ignores `--update-all`.
- **Single-quote patterns in shell.** `'$VAR'` not `"$VAR"`. The shell expands `$VAR` to the empty string in double quotes, breaking the pattern.
- **Pattern is code, not regex.** When the pattern would need `|`, `.*`, `\w`, or `[a-z]`, switch to search_files instead. Don't try to force ast-grep into a regex shape.
- **`--lang` is required for stdin.** When piping with `--stdin`, set `--lang` explicitly; `sg` cannot infer from extension.
- **Linux: prefer `ast-grep` over `sg`** because `sg` collides with `setgroups` from `util-linux`. The helper handles this; if you call `sg` directly, alias it: `alias sg=ast-grep`.
optional-skills/software-development/ast-grep/install.ps1
+235
View File
@@ -0,0 +1,235 @@
#Requires -Version 5.1
<#
.SYNOPSIS
Install the ast-grep binary on Windows.
.DESCRIPTION
Tries package managers in priority order, then falls back to downloading a
pinned release zip from GitHub into <skill_root>/bin/sg.exe.
Order:
1. Already installed? -> nothing to do
2. Scoop (most common Windows dev tool installer)
3. Winget (Microsoft built-in)
4. Chocolatey (choco)
5. npm (@ast-grep/cli)
6. cargo binstall / cargo install
7. pip (ast-grep-cli)
8. GitHub release zip -> <skill_root>/bin/sg.exe
.PARAMETER Method
Force one method: scoop | winget | choco | npm | cargo | pip | github
.PARAMETER Version
Pin a specific version when downloading from GitHub. Default: 0.45.0
.PARAMETER NoFallback
Don't fall back to GitHub zip; fail if all package managers miss
.PARAMETER Quiet
Suppress non-error output
.EXAMPLE
.\install.ps1
.\install.ps1 -Method scoop
.\install.ps1 -Version 0.42.0 -Method github
#>
param(
[string]$Method = "",
[string]$Version = "0.45.0",
[switch]$NoFallback,
[switch]$Quiet
)
$ErrorActionPreference = 'Stop'
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$CacheBinDir = if ($env:OMO_AST_GREP_BIN_DIR) { $env:OMO_AST_GREP_BIN_DIR } else { Join-Path $ScriptDir "bin" }
function Log([string]$msg) {
if (-not $Quiet) {
[Console]::Error.WriteLine("[install.ps1] $msg")
}
}
function Err([string]$msg) {
[Console]::Error.WriteLine("[install.ps1] error: $msg")
}
function Has-Cmd([string]$name) {
$null -ne (Get-Command $name -ErrorAction SilentlyContinue)
}
function Test-AstGrep {
if (Has-Cmd 'ast-grep') { return $true }
if (Has-Cmd 'sg') { return $true }
if (Test-Path (Join-Path $CacheBinDir 'sg.exe')) { return $true }
if (Test-Path (Join-Path $CacheBinDir 'ast-grep.exe')) { return $true }
return $false
}
if (-not $Method -and (Test-AstGrep)) {
Log "ast-grep already installed"
exit 0
}
function Detect-Arch {
$a = $env:PROCESSOR_ARCHITECTURE
switch -Wildcard ($a) {
'AMD64' { return 'x86_64' }
'ARM64' { return 'aarch64' }
default { return 'unknown' }
}
}
$Arch = Detect-Arch
function Try-Scoop {
if (-not (Has-Cmd 'scoop')) { return $false }
Log "trying: scoop install main/ast-grep"
try { scoop install main/ast-grep; return $LASTEXITCODE -eq 0 }
catch { return $false }
}
function Try-Winget {
if (-not (Has-Cmd 'winget')) { return $false }
Log "trying: winget install --id ast-grep.ast-grep"
try { winget install --id ast-grep.ast-grep --silent --accept-package-agreements --accept-source-agreements; return $LASTEXITCODE -eq 0 }
catch { return $false }
}
function Try-Choco {
if (-not (Has-Cmd 'choco')) { return $false }
Log "trying: choco install ast-grep -y"
try { choco install ast-grep -y; return $LASTEXITCODE -eq 0 }
catch { return $false }
}
function Try-Npm {
if (-not (Has-Cmd 'npm')) { return $false }
Log "trying: npm install -g @ast-grep/cli"
try { npm install -g '@ast-grep/cli'; return $LASTEXITCODE -eq 0 }
catch { return $false }
}
function Try-Cargo {
if (Has-Cmd 'cargo-binstall') {
Log "trying: cargo binstall -y ast-grep"
try { cargo binstall -y ast-grep; if ($LASTEXITCODE -eq 0) { return $true } } catch {}
}
if (-not (Has-Cmd 'cargo')) { return $false }
Log "trying: cargo install ast-grep --locked"
try { cargo install ast-grep --locked; return $LASTEXITCODE -eq 0 }
catch { return $false }
}
function Try-Pip {
$pip = $null
foreach ($p in 'pip3','pip','py') {
if (Has-Cmd $p) { $pip = $p; break }
}
if (-not $pip) { return $false }
Log "trying: $pip install --user ast-grep-cli"
try {
if ($pip -eq 'py') { py -m pip install --user ast-grep-cli }
else { & $pip install --user ast-grep-cli }
return $LASTEXITCODE -eq 0
} catch { return $false }
}
function Triple-For-Windows {
switch ($Arch) {
'x86_64' { return 'x86_64-pc-windows-msvc' }
'aarch64' { return 'aarch64-pc-windows-msvc' }
default { return '' }
}
}
function Try-Github {
$triple = Triple-For-Windows
if (-not $triple) {
Err "no GitHub release asset for arch $Arch"
return $false
}
$asset = "app-$triple.zip"
$url = "https://github.com/ast-grep/ast-grep/releases/download/$Version/$asset"
$tmp = Join-Path $env:TEMP ("ast-grep-install-" + [guid]::NewGuid().ToString('N').Substring(0,8))
New-Item -ItemType Directory -Path $tmp -Force | Out-Null
try {
Log "downloading $url"
Invoke-WebRequest -Uri $url -OutFile (Join-Path $tmp $asset) -UseBasicParsing
Expand-Archive -Path (Join-Path $tmp $asset) -DestinationPath (Join-Path $tmp 'extract') -Force
New-Item -ItemType Directory -Path $CacheBinDir -Force | Out-Null
$candidates = @(
(Join-Path $tmp 'extract/ast-grep.exe'),
(Join-Path $tmp 'extract/sg.exe')
)
$src = $null
foreach ($c in $candidates) {
if (Test-Path $c) { $src = $c; break }
}
if (-not $src) {
Err "no ast-grep.exe or sg.exe found inside $asset"
return $false
}
$dest = Join-Path $CacheBinDir 'sg.exe'
Copy-Item -Path $src -Destination $dest -Force
Log "installed cached binary: $dest"
Log "verify: & '$dest' --version"
Log ""
Log "Add to PATH for direct sg use:"
Log " `$env:Path = '$CacheBinDir;' + `$env:Path"
return $true
} finally {
Remove-Item -Recurse -Force $tmp -ErrorAction SilentlyContinue
}
}
function Run-Method([string]$m) {
switch ($m) {
'scoop' { return Try-Scoop }
'winget' { return Try-Winget }
'choco' { return Try-Choco }
'npm' { return Try-Npm }
'cargo' { return Try-Cargo }
'pip' { return Try-Pip }
'github' { return Try-Github }
default { Err "unknown method: $m"; return $false }
}
}
if ($Method) {
if (Run-Method $Method) { exit 0 }
Err "method '$Method' failed"
exit 2
}
$methods = @('scoop', 'winget', 'choco', 'npm', 'cargo', 'pip')
foreach ($m in $methods) {
if (Run-Method $m) {
Log "installed via $m"
exit 0
}
Log "$m unavailable or failed; trying next"
}
if (-not $NoFallback) {
Log "all package managers failed; falling back to GitHub release"
if (Try-Github) { exit 0 }
}
Err "all install methods failed."
Err ""
Err "Manual options:"
Err " scoop install main/ast-grep # Scoop"
Err " winget install --id ast-grep.ast-grep # Winget"
Err " choco install ast-grep # Chocolatey"
Err " npm install -g @ast-grep/cli # any OS with Node"
Err " cargo install ast-grep --locked # any OS with Rust"
Err " pip install ast-grep-cli # any OS with Python"
Err " https://github.com/ast-grep/ast-grep/releases # manual binary"
exit 2
optional-skills/software-development/ast-grep/install.sh
Executable
+286
View File
@@ -0,0 +1,286 @@
#!/usr/bin/env bash
#
# install.sh - install the ast-grep binary on POSIX systems (macOS, Linux, WSL, Git Bash).
#
# Tries package managers in priority order, then falls back to downloading a
# pinned release binary from GitHub into <skill_root>/bin/sg.
#
# Order:
# 1. Already installed? -> nothing to do
# 2. Homebrew (brew)
# 3. npm (@ast-grep/cli)
# 4. cargo binstall (faster) or cargo install (slower)
# 5. pip (ast-grep-cli)
# 6. nix-env (NixOS / Nix users)
# 7. mise (asdf successor)
# 8. GitHub release tarball -> <skill_root>/bin/sg
#
# Flags:
# --method=<m> Force one method: brew | npm | cargo | pip | nix | mise | github
# --version=<v> Pin a specific version when downloading from GitHub
# --no-fallback Don't fall back to GitHub tarball; fail if all package managers miss
# --quiet, -q Suppress non-error output
#
# Exit codes:
# 0 Installed (or already present)
# 1 Argument error
# 2 All install methods failed
# 3 Network failure during GitHub fallback
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SKILL_ROOT="$SCRIPT_DIR"
CACHE_BIN_DIR="${OMO_AST_GREP_BIN_DIR:-$SKILL_ROOT/bin}"
PINNED_VERSION="0.45.0"
FORCED_METHOD=""
USE_FALLBACK=1
QUIET=0
log() {
if [ "$QUIET" -eq 0 ]; then
printf '[install.sh] %s\n' "$*" >&2
fi
}
err() {
printf '[install.sh] error: %s\n' "$*" >&2
}
usage() {
sed -n '2,/^set -/p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//;/^set -/d'
exit "${1:-0}"
}
while [ "$#" -gt 0 ]; do
case "$1" in
--method=*) FORCED_METHOD="${1#*=}" ;;
--version=*) PINNED_VERSION="${1#*=}" ;;
--no-fallback) USE_FALLBACK=0 ;;
--quiet|-q) QUIET=1 ;;
--help|-h) usage 0 ;;
*) err "unknown argument: $1"; usage 1 ;;
esac
shift
done
# --- detect platform -----------------------------------------------------
detect_os() {
case "$(uname -s)" in
Darwin) echo "darwin" ;;
Linux) echo "linux" ;;
MINGW*|MSYS*|CYGWIN*) echo "windows" ;;
*) echo "unknown" ;;
esac
}
detect_arch() {
case "$(uname -m)" in
arm64|aarch64) echo "aarch64" ;;
x86_64|amd64) echo "x86_64" ;;
*) echo "unknown" ;;
esac
}
OS="$(detect_os)"
ARCH="$(detect_arch)"
# --- already installed? --------------------------------------------------
ast_grep_present() {
if command -v ast-grep >/dev/null 2>&1; then
return 0
fi
if command -v sg >/dev/null 2>&1; then
if [ "$OS" = "linux" ]; then
if "$(command -v sg)" --version 2>/dev/null | grep -qi 'ast-grep'; then
return 0
fi
return 1
fi
return 0
fi
if [ -x "$CACHE_BIN_DIR/sg" ] || [ -x "$CACHE_BIN_DIR/ast-grep" ]; then
return 0
fi
return 1
}
if [ -z "$FORCED_METHOD" ] && ast_grep_present; then
log "ast-grep already installed: $(command -v ast-grep 2>/dev/null || command -v sg)"
exit 0
fi
# --- per-method installers -----------------------------------------------
try_brew() {
command -v brew >/dev/null 2>&1 || return 1
log "trying: brew install ast-grep"
brew install ast-grep && return 0 || return 1
}
try_npm() {
command -v npm >/dev/null 2>&1 || return 1
log "trying: npm install -g @ast-grep/cli"
npm install -g @ast-grep/cli && return 0 || return 1
}
try_cargo() {
if command -v cargo-binstall >/dev/null 2>&1; then
log "trying: cargo binstall ast-grep"
cargo binstall -y ast-grep && return 0 || true
fi
if command -v cargo >/dev/null 2>&1; then
log "trying: cargo install ast-grep --locked"
cargo install ast-grep --locked && return 0 || return 1
fi
return 1
}
try_pip() {
if command -v pipx >/dev/null 2>&1; then
log "trying: pipx install ast-grep-cli"
pipx install ast-grep-cli && return 0 || true
fi
command -v pip3 >/dev/null 2>&1 || command -v pip >/dev/null 2>&1 || return 1
PIP="$(command -v pip3 || command -v pip)"
log "trying: $PIP install --user ast-grep-cli"
$PIP install --user ast-grep-cli && return 0 || return 1
}
try_nix() {
command -v nix-env >/dev/null 2>&1 || return 1
log "trying: nix-env -iA nixpkgs.ast-grep"
nix-env -iA nixpkgs.ast-grep && return 0 || return 1
}
try_mise() {
command -v mise >/dev/null 2>&1 || return 1
log "trying: mise use -g ast-grep"
mise use -g ast-grep && return 0 || return 1
}
# Tarball assets are named like:
# app-aarch64-apple-darwin.zip
# app-x86_64-apple-darwin.zip
# app-aarch64-unknown-linux-gnu.zip
# app-x86_64-unknown-linux-gnu.zip
# app-x86_64-pc-windows-msvc.zip (.zip only on windows)
triple_for() {
case "$OS-$ARCH" in
darwin-aarch64) echo "aarch64-apple-darwin" ;;
darwin-x86_64) echo "x86_64-apple-darwin" ;;
linux-aarch64) echo "aarch64-unknown-linux-gnu" ;;
linux-x86_64) echo "x86_64-unknown-linux-gnu" ;;
*) echo "" ;;
esac
}
try_github() {
TRIPLE="$(triple_for)"
if [ -z "$TRIPLE" ]; then
err "no GitHub release asset for $OS-$ARCH; install via package manager or build from source."
return 1
fi
command -v curl >/dev/null 2>&1 || command -v wget >/dev/null 2>&1 || {
err "need curl or wget for GitHub fallback"
return 1
}
ASSET="app-${TRIPLE}.zip"
URL="https://github.com/ast-grep/ast-grep/releases/download/${PINNED_VERSION}/${ASSET}"
TMP="$(mktemp -d -t ast-grep-install-XXXXXX)"
trap 'rm -rf "$TMP"' RETURN
log "downloading $URL"
if command -v curl >/dev/null 2>&1; then
curl -fsSL "$URL" -o "$TMP/$ASSET" || return 3
else
wget -q "$URL" -O "$TMP/$ASSET" || return 3
fi
command -v unzip >/dev/null 2>&1 || {
err "need 'unzip' to extract GitHub release archives"
return 1
}
unzip -q "$TMP/$ASSET" -d "$TMP/extract"
mkdir -p "$CACHE_BIN_DIR"
if [ -f "$TMP/extract/ast-grep" ]; then
mv "$TMP/extract/ast-grep" "$CACHE_BIN_DIR/sg"
elif [ -f "$TMP/extract/sg" ]; then
mv "$TMP/extract/sg" "$CACHE_BIN_DIR/sg"
else
err "no ast-grep or sg binary found inside $ASSET"
return 1
fi
chmod +x "$CACHE_BIN_DIR/sg"
log "installed cached binary: $CACHE_BIN_DIR/sg"
log "verify: $CACHE_BIN_DIR/sg --version"
log ""
log "Add to PATH for direct sg use:"
log " export PATH=\"$CACHE_BIN_DIR:\$PATH\""
return 0
}
# --- dispatch ------------------------------------------------------------
run_method() {
case "$1" in
brew) try_brew ;;
npm) try_npm ;;
cargo) try_cargo ;;
pip) try_pip ;;
nix) try_nix ;;
mise) try_mise ;;
github) try_github ;;
*) err "unknown method: $1"; return 1 ;;
esac
}
if [ -n "$FORCED_METHOD" ]; then
if run_method "$FORCED_METHOD"; then
exit 0
else
err "method '$FORCED_METHOD' failed"
exit 2
fi
fi
# Try methods in OS-aware priority order.
case "$OS" in
darwin) METHODS=(brew npm cargo pip mise) ;;
linux) METHODS=(npm cargo pip nix mise brew) ;;
windows) METHODS=(npm cargo pip mise) ;;
*) METHODS=(npm cargo pip) ;;
esac
for m in "${METHODS[@]}"; do
if run_method "$m"; then
log "installed via $m"
exit 0
fi
log "$m unavailable or failed; trying next"
done
if [ "$USE_FALLBACK" -eq 1 ]; then
log "all package managers failed; falling back to GitHub release"
if try_github; then
exit 0
fi
fi
err "all install methods failed."
err ""
err "Manual options:"
err " brew install ast-grep # macOS / linuxbrew"
err " npm install -g @ast-grep/cli # any OS with Node"
err " cargo install ast-grep --locked # any OS with Rust"
err " pip install ast-grep-cli # any OS with Python"
err " https://github.com/ast-grep/ast-grep/releases # manual binary"
exit 2
optional-skills/software-development/ast-grep/references/cli.md
+231
View File
@@ -0,0 +1,231 @@
# CLI reference — `sg` / `ast-grep`
Compact reference for the underlying `sg` binary that the helper wraps. Use this when the helper isn't enough or when you want to invoke `sg` directly.
> **Binary name on Linux**: prefer `ast-grep` over `sg` because `sg` collides with `setgroups` from `util-linux`.
---
## `sg run` — one-shot search/rewrite
The default subcommand. `sg -p 'foo'` is shorthand for `sg run -p 'foo'`.
```bash
sg run [OPTIONS] --pattern <PATTERN> [PATHS...]
```
| Flag | Purpose |
|---|---|
| `-p, --pattern <P>` | AST pattern to match. **Always single-quote** in shell to prevent `$VAR` expansion. |
| `-r, --rewrite <R>` | Replacement pattern. Used with `-U` to apply. |
| `-l, --lang <LANG>` | Language. Inferred from path extension if omitted. |
| `--selector <KIND>` | When the pattern is ambiguous, extract only this AST kind. |
| `--strictness <S>` | `cst` \| `smart` (default) \| `ast` \| `relaxed` \| `signature` |
| `--debug-query[=<F>]` | Print parsed pattern. F: `pattern` \| `ast` \| `cst` \| `sexp` |
| `--stdin` | Read code from stdin instead of files. Lang must be set. |
| `--globs <G>` | Include/exclude glob (repeatable; prefix `!` to exclude). |
| `--follow` | Follow symlinks. |
| `--no-ignore <T>` | Disable a class of ignore: `hidden`, `dot`, `exclude`, `global`, `parent`, `vcs`. |
| `-i, --interactive` | Step through matches and confirm each rewrite. |
| `-U, --update-all` | Apply all rewrites without confirmation. **Mutually exclusive with `--json`** (silently). |
| `--json[=<S>]` | Emit JSON. S: `pretty` \| `stream` \| `compact` (compact is best for piping). |
| `--color <W>` | `auto` \| `always` \| `ansi` \| `never` |
| `--inspect <G>` | Detail level: `nothing` \| `summary` \| `entity` |
| `-A, -B, -C <N>` | Context lines after / before / around each match. |
| `-j, --threads <N>` | Thread count (default: heuristic; `0` = auto). |
### `--update-all` + `--json` — the trap
`sg` silently ignores `--update-all` when `--json` is set. To preview AND apply, run **two passes**:
```bash
# Pass 1: preview
sg run -p 'foo()' -r 'bar()' --json=compact src/
# Pass 2: apply
sg run -p 'foo()' -r 'bar()' --update-all src/
```
The `ast_grep_helper.py replace --apply` subcommand does this automatically.
### Examples
```bash
# Basic search
sg run -p 'console.log($MSG)' --lang ts src/
# Search with context lines
sg run -p 'eval($CODE)' --lang js -C 3 .
# Rewrite, dry-run preview as JSON
sg run -p 'console.log($MSG)' -r 'logger.info($MSG)' --json=compact --lang ts src/
# Rewrite, apply
sg run -p 'console.log($MSG)' -r 'logger.info($MSG)' --update-all --lang ts src/
# Pattern from stdin
echo 'console.log("x")' | sg run -p 'console.log($MSG)' --lang js --stdin
# Limit to specific files
sg run -p 'foo()' --lang ts --globs 'src/**/*.ts' --globs '!**/*.test.ts' .
# Debug a pattern that returns 0 matches
sg run -p 'def $F($$$):' --lang py --debug-query=ast --stdin <<< 'def foo(): pass'
```
---
## `sg scan` — YAML rule scanner
Run a configuration of YAML rules across files. Used for project-wide lints and codemods.
```bash
sg scan [OPTIONS] [PATHS...]
```
| Flag | Purpose |
|---|---|
| `-c, --config <C>` | Path to `sgconfig.yml` (default: walk up from cwd looking for one). |
| `-r, --rule <F>` | Run a **single** rule file. Mutually exclusive with `--config`. |
| `--inline-rules <Y>` | Pass YAML rule text inline. Use `---` to separate multiple rules. |
| `--filter <RE>` | Only run rules whose `id` matches this regex. |
| `--include-metadata` | Include rule `metadata` field in JSON output. |
| `-U, --update-all` | Apply fixes from `fix:` automatically. |
| `--report-style <S>` | `rich` \| `medium` \| `short` |
| `--format <F>` | `github` \| `sarif` (CI-friendly outputs). |
| `--error[=ID]`, `--warning[=ID]`, `--info[=ID]`, `--hint[=ID]`, `--off[=ID]` | Promote/demote severity. |
| `-i, --interactive` | Confirm each fix interactively. |
| `--json[=<S>]` | JSON output. |
### Examples
```bash
# Run all rules in sgconfig.yml-discovered ruleDirs
sg scan src/
# Run a single rule file (no sgconfig.yml needed)
sg scan -r rules/no-console.yml src/
# Inline rule (great for one-offs and CI)
sg scan --inline-rules '
id: no-todo
language: TypeScript
severity: warning
rule: { pattern: TODO }' src/
# Apply all auto-fixes
sg scan -U src/
# CI-friendly GitHub annotations
sg scan --format github src/
# SARIF for security scanners
sg scan --format sarif src/ > sarif.json
```
---
## `sg test` — run rule snapshot tests
```bash
sg test [OPTIONS]
```
| Flag | Purpose |
|---|---|
| `-c, --config <C>` | Path to `sgconfig.yml`. |
| `-t, --test-dir <D>` | Test directory. |
| `--snapshot-dir <D>` | Snapshot directory (default: `__snapshots__`). |
| `--skip-snapshot-tests` | Validate test code parses; don't compare snapshots. |
| `-U, --update-all` | Update all changed snapshots. |
| `-f, --filter <G>` | Filter test cases by glob on rule id. |
| `--include-off` | Include rules with severity `off`. |
| `-i, --interactive` | Step through changed snapshots and accept/reject each. |
A test directory looks like:
```
test/
├── no-console.yml # `valid:` and `invalid:` snippets
└── no-console-test.yml # alternative test file format
__snapshots__/
└── no-console-snapshot.yml # expected match locations
```
---
## `sg new` — scaffold
```bash
sg new <COMMAND> [NAME] [OPTIONS]
```
| Subcommand | Creates |
|---|---|
| `project` | `sgconfig.yml`, `rules/`, `utils/`, `__snapshots__/` directory tree |
| `rule` | A new YAML rule file in the first `ruleDirs` entry |
| `test` | A new test file in `testConfigs[0].testDir` |
| `util` | A new utility rule in the first `utilDirs` entry |
```bash
# New project in current dir
sg new project --yes
# New rule
sg new rule no-console --lang typescript
# New test
sg new test no-console --yes
```
---
## `sg lsp` — language server
```bash
sg lsp -c sgconfig.yml
```
Speak LSP over stdin/stdout. Configure your editor (VS Code extension, Neovim `nvim-lspconfig`, Helix `languages.toml`) to spawn this command for live diagnostics.
---
## `sg completions` — shell completions
```bash
sg completions bash >> ~/.bashrc
sg completions zsh > "${fpath[1]}/_sg"
sg completions fish > ~/.config/fish/completions/sg.fish
sg completions powershell >> $PROFILE
```
---
## Useful one-liners
```bash
# Count matches per file
sg run -p 'console.log($_)' --lang ts --json=compact . \
| jq -r '.[].file' | sort | uniq -c | sort -rn
# Find all unique kinds in a file (great for figuring out kind names)
sg run -p '$_' --lang ts --debug-query=cst src/foo.ts \
| grep -oE 'kind: [a-z_]+' | sort -u
# Rewrite only in a subset of files
sg run -p 'foo()' -r 'bar()' --update-all --globs 'src/**/*.ts' --globs '!src/legacy/**' .
# Apply fixes from many rules but only ones matching a pattern in their id
sg scan --filter 'no-' -U src/
# Use ast-grep as a linter in pre-commit
sg scan --format github src/ || exit 1
```
---
## See also
- `references/yaml-rules.md` — rule schema (`pattern`, `kind`, `regex`, `inside`, `has`, `all`, `any`, `not`, `matches`, `transform`, `fix`).
- `references/sgconfig.md` — project configuration.
- Official: <https://ast-grep.github.io/reference/cli.html>
optional-skills/software-development/ast-grep/references/install.md
+166
View File
@@ -0,0 +1,166 @@
# Install ast-grep
The skill ships an `install.sh` (POSIX) and `install.ps1` (Windows) that try every reasonable method in priority order and fall back to a GitHub release download as a last resort. **You usually do not need to read this page.** Run the installer:
```bash
bash install.sh # macOS / Linux / WSL / Git Bash
pwsh -File install.ps1 # Windows PowerShell
```
This page exists for the (rare) case the installer cannot find a working method, or you want to install ast-grep manually.
---
## Per-OS install commands (verbatim, copy-paste)
### macOS
```bash
brew install ast-grep # Homebrew - the primary path
sudo port install ast-grep # MacPorts
npm install -g @ast-grep/cli # if you have Node already
cargo install ast-grep --locked # if you have Rust already
```
### Linux
```bash
# Universal (works on every distro)
npm install -g @ast-grep/cli
cargo install ast-grep --locked
pip install ast-grep-cli
# Distro-specific
nix-env -iA nixpkgs.ast-grep # NixOS / Nix
brew install ast-grep # Linuxbrew
# NixOS shell.nix
nix-shell -p ast-grep
```
> **Linux gotcha**: the binary is named `sg`, but on most Linux systems `sg` is also the **`setgroups` command** from `util-linux`. The shell sees `setgroups` first and ignores ast-grep. Two options:
>
> 1. Always invoke `ast-grep` (full name).
> 2. Add an alias: `alias sg=ast-grep` in your `~/.bashrc` / `~/.zshrc`.
>
> The `ast_grep_helper.py` script in `scripts/` already handles this — when it sees `sg` on PATH on Linux, it runs `--version` and rejects the binary if it isn't ast-grep.
### Windows
```powershell
scoop install main/ast-grep # Scoop (most common on dev machines)
winget install --id ast-grep.ast-grep # Winget (Microsoft built-in)
choco install ast-grep # Chocolatey
npm install -g @ast-grep/cli # any OS with Node
cargo install ast-grep --locked # any OS with Rust
```
### WSL / Git Bash on Windows
Treat as Linux. Use `npm`, `cargo`, `pip`, or `bash install.sh`.
---
## Cross-platform / language-ecosystem methods
These work on every OS:
| Method | Command | Pros | Cons |
|---|---|---|---|
| **npm** | `npm install -g @ast-grep/cli` | Fast, prebuilt platform binaries | Needs Node 18+ |
| **cargo** | `cargo install ast-grep --locked` | Always builds latest from source | Slow (~3-5 min compile) |
| **cargo binstall** | `cargo binstall ast-grep` | Fast (downloads release binary) | Needs `cargo-binstall` first |
| **pip** | `pip install ast-grep-cli` | Works in any Python venv | Needs Python 3.8+ |
| **pipx** | `pipx install ast-grep-cli` | Isolated install | Needs pipx |
| **mise** | `mise use -g ast-grep` | asdf successor, version-pinning | Needs mise |
| **GitHub release** | manual download | Pure binary, no toolchain | Manual PATH setup |
---
## GitHub release manual install
If every package manager fails:
```bash
# 1. Pick the right asset for your OS+arch from the latest release:
# https://github.com/ast-grep/ast-grep/releases/latest
#
# Naming pattern:
# app-aarch64-apple-darwin.zip macOS Apple Silicon
# app-x86_64-apple-darwin.zip macOS Intel
# app-aarch64-unknown-linux-gnu.zip Linux ARM64 (glibc)
# app-x86_64-unknown-linux-gnu.zip Linux x86_64 (glibc)
# app-x86_64-pc-windows-msvc.zip Windows x86_64
# app-aarch64-pc-windows-msvc.zip Windows ARM64
# 2. Download and extract:
VERSION=0.45.0
TRIPLE=aarch64-apple-darwin
curl -fsSL "https://github.com/ast-grep/ast-grep/releases/download/${VERSION}/app-${TRIPLE}.zip" -o /tmp/ast-grep.zip
unzip /tmp/ast-grep.zip -d /tmp/ast-grep
sudo mv /tmp/ast-grep/ast-grep /usr/local/bin/sg
sudo chmod +x /usr/local/bin/sg
# 3. Verify:
sg --version
```
The skill's `install.sh` does steps 1-3 automatically and drops the binary in `<skill_root>/bin/sg` so you can use it without sudo.
---
## Build from source
```bash
git clone https://github.com/ast-grep/ast-grep.git
cd ast-grep
cargo install --path ./crates/cli --locked
```
Requires Rust 1.74+. Slowest path; only useful when you need a specific commit or unreleased fix.
---
## Verifying the install
```bash
ast-grep --version # or `sg --version`
# ast-grep 0.45.0
```
Then sanity-check a real query:
```bash
echo 'console.log("hello")' | sg run -p 'console.log($MSG)' --lang js --stdin
```
Expected: a single match with the `console.log("hello")` call highlighted.
---
## Editor integration
After installing the CLI, set up your editor:
- **VS Code**: install the [`ast-grep`](https://marketplace.visualstudio.com/items?itemName=ast-grep.ast-grep-vscode) extension. Requires `sgconfig.yml` in workspace root for live diagnostics.
- **Neovim**: configure `nvim-lspconfig` with `ast_grep` server, or install [`telescope-ast-grep.nvim`](https://github.com/ray-x/telescope-ast-grep.nvim).
- **Helix**: add `ast-grep lsp` as a language server in `languages.toml`.
- **Emacs**: install [`ast-grep.el`](https://github.com/SunskyXH/ast-grep.el).
See `references/cli.md` for `ast-grep lsp` flags.
---
## Uninstall
| Method | Command |
|---|---|
| brew | `brew uninstall ast-grep` |
| npm | `npm uninstall -g @ast-grep/cli` |
| cargo | `cargo uninstall ast-grep` |
| pip | `pip uninstall ast-grep-cli` |
| pipx | `pipx uninstall ast-grep-cli` |
| scoop | `scoop uninstall ast-grep` |
| winget | `winget uninstall --id ast-grep.ast-grep` |
| choco | `choco uninstall ast-grep` |
| GitHub binary | `rm <skill_root>/bin/sg` |
optional-skills/software-development/ast-grep/references/patterns.md
+147
View File
@@ -0,0 +1,147 @@
# Pattern syntax — meta-variables and how patterns parse
ast-grep is **not regex**. Patterns are written in the **same syntax as the target language** (TypeScript, Python, Go, etc.), and ast-grep matches them against the AST of every file. The wildcards are called **meta-variables**.
This page is the canonical primer. If a pattern fails, 90% of the time it is one of the issues on this page.
---
## The three meta-variables
| Syntax | Matches | Capture |
|---|---|---|
| `$VAR` | exactly **one** AST node | yes, by name |
| `$$$` | **zero or more** AST nodes (a list) | no (anonymous) |
| `$$$VAR` | zero or more AST nodes | yes, by name |
| `$_` | one AST node | no (anonymous) |
A meta-variable always replaces a **whole AST node**, never a substring of a node. `$VAR` cannot match the first three characters of an identifier, only an entire identifier (or expression, or statement, depending on context).
### Naming rules
- Must start with `$`.
- Then uppercase letters `A-Z`, digits, or underscores.
- **Valid**: `$X`, `$VAR`, `$VAR_1`, `$_`, `$_VAR`, `$ARG1`.
- **Invalid**: `$lower`, `$kebab-case`, `$1` (digit first), `$$single` (use `$_` for anonymous).
### Same-name = same content
Two occurrences of the same metavariable in a pattern must capture **identical text**:
```ts
// Pattern
$X === $X
// Matches
a === a
foo.bar === foo.bar
// Does NOT match
a === b
foo === foo.bar
```
Useful for finding redundant comparisons, double assignments, etc.
### `$$$` is greedy
When you write `foo($$$A, b, $$$C)`, the matcher does **not** backtrack or try every possible split. It greedily fills `$$$A` until the pattern can match `b`, then everything left goes into `$$$C`.
```ts
// Pattern
foo($$$A, b, $$$C)
// Input
foo(a, c, b, b, c)
// Capture
$$$A = [a, c]
$$$C = [b, c]
```
If you need a different split, restructure the pattern (e.g. add a constraint).
---
## Patterns must be valid code
The pattern itself must parse with the target language's grammar. ast-grep treats `$VAR` and `$$$` as identifiers/argument lists during parsing, then matches structurally.
### What goes wrong
| Bad pattern | Why it fails | Fix |
|---|---|---|
| `function $NAME` | Function declaration without body — not a valid AST node in JS/TS/Go/Rust. | `function $NAME($$$) { $$$ }` |
| `def $FN($$$):` | Trailing colon. ast-grep parses as a complete function definition; the colon makes it a statement. | `def $FN($$$)` |
| `class Foo:` | Same — Python class without body. | `class Foo($$$)` |
| `fn $NAME` | Rust fn without signature. | `fn $NAME($$$) -> $RET { $$$ }` |
| `if x` | Incomplete `if` — most languages require the body. | `if x { $$$ }` (curly-brace languages) or `if x: $$$` (Python uses `pattern.context`/`selector` instead) |
| `"key": "$VAL"` | JSON pattern — a key/value pair on its own isn't valid JSON. | Use `pattern: { context: '{"key": "$VAL"}', selector: pair }` |
### When a sub-expression isn't valid on its own
Sometimes you want to match an *expression* that the language only allows inside a larger context. Use the `pattern` object form:
```yaml
pattern:
context: 'class A { $FIELD = $INIT }'
selector: field_definition
```
This says: parse `class A { $FIELD = $INIT }` as a whole, then keep only the `field_definition` sub-tree as the actual pattern.
---
## Strictness levels
When CST nodes don't match exactly (extra whitespace, different unnamed punctuation), ast-grep can be more or less forgiving. Pass `--strictness <LEVEL>` on the CLI, or set it in a YAML rule:
| Level | Matches |
|---|---|
| `cst` | Every node, including unnamed (commas, parens, etc.) |
| `smart` (default) | All except unnamed nodes in the **target** that aren't in the pattern |
| `ast` | Only named AST nodes |
| `relaxed` | Named AST nodes, ignoring comments |
| `signature` | Only node kinds — text and unnamed nodes ignored |
`smart` is almost always what you want. Reach for `signature` when you want to match "any function called `foo`" regardless of arguments.
---
## Testing a pattern
Two tools help you confirm a pattern parses the way you expect:
```bash
# Print the AST of the pattern itself
sg run -p 'console.log($MSG)' --lang ts --debug-query=ast
# Print the parsed CST of a file (great for figuring out kind names)
sg run -p '$_' --lang ts --debug-query=cst src/example.ts | head -40
```
`--debug-query=ast` shows the named AST nodes only (cleaner). `--debug-query=cst` shows everything including punctuation. Both go to stderr, so they don't interfere with stdout JSON.
The web playground is also fast: <https://ast-grep.github.io/playground.html>.
---
## When ast-grep is the wrong tool
If your pattern is fundamentally text-shaped, switch to `grep` / `rg`:
- Match across multiple files **for any text** → `rg`
- Cross-language regex with alternation → `rg -e foo -e bar`
- Match comments only → `rg --type ts '^\s*//.*TODO'`
- Match URLs, emails, license headers → `rg`
ast-grep is for **code structure**: function shapes, call patterns, control flow, type annotations, imports, error handling. If your "pattern" only depends on the bytes of the file and not on the syntax, regex is the right tool.
---
## See also
- `references/pitfalls.md` — concrete regex anti-patterns and language-specific traps.
- `references/recipes.md` — copy-paste-ready patterns for TS/JS/Py/Go/Rust.
- `references/yaml-rules.md` — `kind`, `regex`, `inside`, `has`, `all`, `any`, `not`, `matches`.
- Official: <https://ast-grep.github.io/guide/pattern-syntax.html>
optional-skills/software-development/ast-grep/references/pitfalls.md
+303
View File
@@ -0,0 +1,303 @@
# Pitfalls — what breaks patterns and how to fix them
This is the failure-mode field guide. The `scripts/ast_grep_helper.py validate` subcommand mechanically checks for the items in §1 before calling `sg`; the rest are lower-frequency but still common.
---
## 1. Regex syntax does not work
ast-grep does **not** interpret regex inside patterns. The following all fail:
| Bad | Why | Use instead |
|---|---|---|
| `foo\|bar` | `\|` is regex alternation. ast-grep does not alternate. | Two separate calls, OR `any: [pattern: foo, pattern: bar]` in a YAML rule, OR `rg -e foo -e bar`. |
| `foo.*bar` | `.*` is a regex wildcard. | `foo($$$) bar` if the gap is a list of nodes; otherwise switch to `rg`. |
| `\w+`, `\d+`, `\s` | Regex character classes. | `$VAR` to capture any identifier. For digits-only, use `kind: number_literal`. |
| `[a-z]+` | Regex character class. | No AST equivalent — switch to `rg`. |
| `^foo$` | Regex anchors. | Anchor by AST: use `kind: program > expression_statement` or use `inside`/`not has`. |
**Why this happens**: LLMs default to regex thinking. The mental switch is "ast-grep patterns are *code*, not *strings*."
When you genuinely need regex, use the `regex` rule field in YAML (matches node text with Rust regex):
```yaml
rule:
all:
- kind: identifier
- regex: '^[A-Z][a-z]+$' # CamelCase identifiers only
```
Note: `regex` matches the **whole node text** — no partial matches. Combine with `kind` or `pattern` for performance.
---
## 2. Incomplete AST nodes
Patterns must be valid code that the parser accepts as a complete node. Common mistakes:
```text
# JS/TS
function foo ❌ no params, no body
function $NAME($$$) { $$$ } ✅
async function $NAME ❌
async function $NAME($$$) { $$$ } ✅
# Python
def foo: ❌ trailing colon makes it a statement
def $FN($$$) ✅
class Foo: ❌
class $C($$$) ✅
# Go
func foo ❌
func $NAME($$$) { $$$ } ✅
# Rust
fn foo ❌
fn $NAME($$$) -> $RET { $$$ } ✅
fn $NAME($$$) { $$$ } ✅ (-> () inferred)
# Java
public void foo ❌
public void $NAME($$$) { $$$ } ✅
```
If a pattern returns 0 matches and looks correct, run `sg run -p '<pattern>' --lang <lang> --debug-query=ast --stdin <<< 'echo'` and see what the parser thinks the pattern is. If it returns an `ERROR` node, the pattern is malformed.
---
## 3. Pattern parses as the wrong kind
A class field initializer `a = 123` *also* parses as an assignment expression. If you want only field definitions, you must disambiguate:
```yaml
# WRONG — pattern parses as assignment_expression, not field_definition
pattern: a = 123
kind: field_definition
# CORRECT — use pattern object with context + selector
pattern:
context: 'class C { a = 123 }'
selector: field_definition
```
`kind` and `pattern` are **independent constraints**, not modifiers of each other. ast-grep does not change *how* it parses based on `kind`.
---
## 4. The `|` ambiguity
A bare `|` in a pattern is interpreted as bitwise-or in most languages, **not** alternation. So:
```yaml
pattern: foo | bar # parses as: foo bitwise-or'd with bar
```
…matches expressions like `x | y`, not "either foo or bar". To get alternation, use `any`:
```yaml
rule:
any:
- pattern: foo
- pattern: bar
```
In TypeScript union types (`A | B`), `|` is part of the type syntax — `pattern: A | B` correctly parses as a union type and matches that.
---
## 5. Same-name metavars collide
```ts
// Pattern: $X = $X
// Captures only when both sides are TEXTUALLY identical.
// Matches:
a = a
foo.bar = foo.bar
// Does NOT match:
a = b
let x = compute() // because $X needs to bind once and re-use
```
If you actually want two independent captures, name them differently: `$X = $Y`.
---
## 6. `$$$` is greedy then commits
`$$$` does **not** backtrack. It captures as much as possible, then commits. If your pattern needs a non-greedy match, structure it differently:
```ts
// You want "match foo($X), where $X is any single arg"
// BAD: foo($$$X) // matches foo(a), foo(a, b), foo(a, b, c) - too broad
// GOOD: foo($X) // matches only single-arg calls
// You want "match foo() with at least one arg"
// BAD: foo($$$X) // also matches foo()
// GOOD: foo($X, $$$REST) // forces at least one arg
```
---
## 7. `kind` names depend on tree-sitter grammar
`kind: function_declaration` works for JavaScript, but Python uses `function_definition`, Rust uses `function_item`, Go uses `function_declaration` (same as JS by coincidence). To find the right name, parse a known-good file:
```bash
sg run -p '$_' --lang python --debug-query=cst path/to/example.py | grep -i function
```
Or open <https://ast-grep.github.io/playground.html> and click on a node to see its `kind`.
---
## 8. `inside` / `has` defaults to `stopBy: neighbor`
```yaml
inside:
kind: function_declaration # only checks the IMMEDIATE parent
```
If you want "anywhere inside a function (any depth)":
```yaml
inside:
kind: function_declaration
stopBy: end # walks up to the file root
```
Same for `has` (descendants):
```yaml
has:
kind: return_statement
stopBy: end # walks down the whole subtree
```
Without `stopBy: end`, `has` only matches direct children.
---
## 9. CLI silently ignores `--update-all` when `--json` is set
This is the single biggest gotcha when scripting ast-grep. If you run:
```bash
sg run -p 'foo()' -r 'bar()' --json=compact --update-all .
```
…you get the JSON output but **no files are mutated**. ast-grep silently drops `--update-all` when `--json` is on. To both preview and apply, run **two passes**:
```bash
# Pass 1: preview as JSON
sg run -p 'foo()' -r 'bar()' --json=compact .
# Pass 2: actually apply
sg run -p 'foo()' -r 'bar()' --update-all .
```
`scripts/ast_grep_helper.py replace` does this automatically when `--apply` is set.
---
## 10. Composite rules apply to a single node
`all` and `any` evaluate against **one target node** at a time:
```yaml
# WRONG — wants "node has BOTH a number child AND a string child"
has:
all:
- kind: number # impossible: one node cannot be both at once
- kind: string
# CORRECT
all:
- has: { kind: number }
- has: { kind: string }
```
Lift relational rules out of composites when the relation is "the surrounding node has X children matching Y."
---
## 11. Field order is not guaranteed
When a rule object has multiple fields:
```yaml
rule:
pattern: $X = compute()
has: { kind: number }
```
…ast-grep evaluates them as an implicit `all`, but the **order** in which metavariables are captured is not guaranteed. If your `transform` or `fix` depends on capture order, use an explicit `all` array:
```yaml
rule:
all:
- pattern: function $F() { $$$ }
- has: { pattern: $F() } # $F captured by pattern first; here we just check
```
---
## 12. `regex` without `kind` is slow
`regex` alone scans every node text in the file. On large repos this is noticeably slow. Always combine:
```yaml
# Slow
rule:
regex: '^TODO'
# Fast
rule:
all:
- kind: comment
- regex: '^//\s*TODO'
```
---
## 13. No scope / type / data-flow analysis
ast-grep is a **structural** matcher. It does NOT know:
- Whether two `foo` references point to the same variable.
- Whether a variable is shadowed.
- Whether a function is async, throws, returns a Promise.
- Whether a value flows from input to output.
For those questions, use a real type-aware tool: TypeScript LSP, Pyright, Semgrep with type inference, CodeQL, etc.
ast-grep is great when *the syntactic shape* is what you care about: "find every call to `eval(...)`", "find every `as any`", "find every empty catch block." It is weak for "find every variable that's never used."
---
## 14. Pattern testing is the fastest debugger
When a pattern returns 0 matches and you can't see why:
1. Open <https://ast-grep.github.io/playground.html>.
2. Paste your code into the left pane, your pattern into the top-right.
3. The bottom-right shows the parsed AST and which nodes matched (highlighted) or failed.
Or locally:
```bash
sg run -p '<pattern>' --lang <lang> --debug-query=ast --stdin <<< '<sample-code>'
```
stderr shows the parsed pattern; stdout shows the JSON match result. If the pattern shows up as `ERROR (XXX)`, it doesn't parse.
---
## See also
- `references/patterns.md` — meta-variables, strictness, naming rules.
- `references/recipes.md` — known-good patterns by language.
- `references/cli.md` — `--debug-query`, `--strictness`, `--update-all`.
optional-skills/software-development/ast-grep/references/recipes.md
+402
View File
@@ -0,0 +1,402 @@
# Recipes — copy-paste patterns by language
Every pattern in this file has been verified against the canonical syntax. They are starting points; tweak metavariable names and constraints to fit your case.
Use them with the helper:
```bash
ast-grep-helper search '<PATTERN>' --lang <LANG> [path]
ast-grep-helper replace '<PATTERN>' '<REWRITE>' --lang <LANG> [path] # dry-run
ast-grep-helper replace '<PATTERN>' '<REWRITE>' --lang <LANG> [path] --apply
```
Or directly:
```bash
sg run -p '<PATTERN>' --lang <LANG> [path]
sg run -p '<PATTERN>' -r '<REWRITE>' --update-all --lang <LANG> [path]
```
---
## TypeScript / TSX / JavaScript
### Find structural patterns
```typescript
// Every function declaration
function $NAME($$$PARAMS) { $$$BODY }
// Every async function
async function $NAME($$$PARAMS) { $$$BODY }
// Every arrow function (any param shape)
($$$PARAMS) => $$$BODY
// Every method on a class
class $C { $$$ $METHOD($$$P) { $$$B } $$$ }
// Every import statement
import { $$$NAMES } from '$MOD'
import $DEFAULT from '$MOD'
import * as $NS from '$MOD'
// Every console.* call
console.$METHOD($$$ARGS)
// Every JSX element of a given name
<MyComponent $$$PROPS>$$$CHILDREN</MyComponent>
// Every try/catch
try { $$$BODY } catch ($E) { $$$HANDLER }
// Every throw
throw $EXPR
// Every new expression
new $CLASS($$$ARGS)
// Every type assertion to any (anti-pattern!)
$EXPR as any
$EXPR as unknown as $T
```
### Common rewrites
```bash
# console.log -> logger.info
sg run -p 'console.log($$$A)' -r 'logger.info($$$A)' --lang ts --update-all .
# require -> import (one-arg case)
sg run -p 'const $V = require($M)' -r 'import $V from $M' --lang ts --update-all .
# .then(callback) -> await on the same line (use with caution; needs async function context)
sg run -p '$P.then($CB)' -r 'const $TMP = await $P; $CB($TMP)' --lang ts --update-all .
# Strip `as any`
sg run -p '$E as any' -r '$E' --lang ts --update-all .
# Rename a function call site
sg run -p 'oldName($$$A)' -r 'newName($$$A)' --lang ts --update-all .
```
---
## Python
> **Reminder**: never end a Python pattern with `:`. Patterns parse as a complete statement, so `def foo($$$):` is invalid.
```python
# Every function definition
def $FN($$$PARAMS)
# Every class definition
class $C($$$BASES)
# Every decorator usage
@$DEC
def $FN($$$P)
# Every print call (Python 3)
print($$$ARGS)
# Every f-string
f"$STR"
# Every with-statement
with $CTX as $VAR: $$$BODY
# Every try/except
try: $$$BODY
except $EXC: $$$HANDLER
# Every list comprehension
[$EXPR for $VAR in $ITER]
# Every async def
async def $FN($$$PARAMS)
# Type hints — Optional[X]
Optional[$T]
# Type hints — X | None (PEP 604)
$T | None
```
### Common rewrites
```bash
# print(...) -> logger.info(...)
sg run -p 'print($$$A)' -r 'logger.info($$$A)' --lang py --update-all .
# Optional[X] -> X | None
sg run -p 'Optional[$T]' -r '$T | None' --lang py --update-all .
# from typing import List -> remove (built-in list works in 3.9+)
sg run -p 'from typing import List' -r 'from typing import List # TODO: remove, use list' --lang py --update-all .
```
---
## Go
```go
// Every function
func $NAME($$$PARAMS) $$$RET { $$$BODY }
// Every method
func ($RECV $TYPE) $NAME($$$PARAMS) $$$RET { $$$BODY }
// The classic err nil-check
if err != nil { $$$BODY }
// Every fmt.Println / fmt.Printf / fmt.Sprintf
fmt.$METHOD($$$ARGS)
// Every defer
defer $EXPR
// Every goroutine
go $EXPR
// Every channel send/recv
$CH <- $VAL
$VAL := <-$CH
// Every type assertion
$EXPR.($TYPE)
```
### Common rewrites
```bash
# fmt.Println -> log.Println
sg run -p 'fmt.Println($$$A)' -r 'log.Println($$$A)' --lang go --update-all .
# Add error wrapping
sg run -p 'return $ERR' -r 'return fmt.Errorf("operation failed: %w", $ERR)' --lang go --update-all .
```
---
## Rust
```rust
// Every fn
fn $NAME($$$PARAMS) -> $RET { $$$BODY }
fn $NAME($$$PARAMS) { $$$BODY } // no return type
// Every async fn
async fn $NAME($$$PARAMS) -> $RET { $$$BODY }
// Every method on impl
impl $TYPE { fn $METHOD($$$P) -> $R { $$$B } }
// Every trait impl
impl $TRAIT for $TYPE { $$$ITEMS }
// Every match expression
match $EXPR { $$$ARMS }
// Every Result-returning fn that uses ?
fn $N($$$P) -> Result<$T, $E> { $$$ }
// .unwrap() / .expect() (anti-patterns)
$EXPR.unwrap()
$EXPR.expect($MSG)
// Every println!/eprintln!/format!
println!($$$ARGS)
format!($$$ARGS)
```
### Common rewrites
```bash
# unwrap() -> ? in Result-returning fns (caution: needs context)
sg run -p '$E.unwrap()' -r '$E?' --lang rust --update-all .
# eprintln! -> log::error!
sg run -p 'eprintln!($$$A)' -r 'log::error!($$$A)' --lang rust --update-all .
```
---
## Java
```java
// Every public class
public class $NAME { $$$BODY }
// Every method (any modifier)
$$$MOD $RET $NAME($$$P) { $$$BODY }
// Every System.out.println / System.err.println
System.$STREAM.println($$$ARGS)
// Every try-with-resources
try ($$$RES) { $$$BODY } catch ($EXC $E) { $$$HANDLER }
// Every annotation usage
@$ANNOTATION
$DECL
```
---
## C / C++
```cpp
// Every printf-family call
printf($$$ARGS)
sprintf($$$ARGS)
fprintf($$$ARGS)
// Every malloc / free pair (find-only — pairing requires data flow)
malloc($SIZE)
free($PTR)
// Every for-loop
for ($INIT; $COND; $POST) { $$$BODY }
// C++ smart pointer make
std::make_shared<$T>($$$ARGS)
std::make_unique<$T>($$$ARGS)
```
### Rewrites
```bash
# malloc(N * sizeof(T)) -> calloc(N, sizeof(T)) - safer
sg run -p 'malloc($N * sizeof($T))' -r 'calloc($N, sizeof($T))' --lang c --update-all .
```
---
## CSS
```css
/* Every rule with a specific property */
{ $$$ color: $VAL; $$$ }
/* Every @media query */
@media $QUERY { $$$BODY }
/* Every var() reference */
var($NAME)
```
---
## HTML
```html
<!-- Every img without alt -->
<img $$$ />
<!-- Every script tag -->
<script $$$>$$$BODY</script>
<!-- Every link to stylesheet -->
<link rel="stylesheet" href=$URL />
```
---
## Bash / Shell
```bash
# Every for-loop
for $VAR in $$$LIST; do $$$BODY; done
# Every if-statement
if $$$COND; then $$$BODY; fi
# Every function definition
$NAME() { $$$BODY }
# Every subshell call
$( $$$CMD )
```
---
## YAML rule recipes (for `sg scan`)
These are full YAML rules you can drop in `rules/*.yml` and run via `sg scan`. See `references/yaml-rules.md` for the full schema.
### no-console (TypeScript)
```yaml
id: no-console
language: TypeScript
severity: warning
message: "Avoid console.* in production"
rule:
pattern: console.$METHOD($$$ARGS)
fix: logger.$METHOD($$$ARGS)
```
### no-as-any (TypeScript)
```yaml
id: no-as-any
language: TypeScript
severity: error
message: "`as any` defeats type safety. Use a proper type."
rule:
pattern: $EXPR as any
fix: $EXPR
```
### empty-catch (JavaScript)
```yaml
id: empty-catch
language: JavaScript
severity: error
message: "Empty catch swallows errors silently."
rule:
all:
- pattern: try { $$$T } catch ($E) { $$$H }
- has:
kind: catch_clause
has:
kind: statement_block
not:
has:
kind: statement
stopBy: end
```
### print-to-logger (Python)
```yaml
id: print-to-logger
language: Python
severity: hint
message: "Use logger.info instead of print"
rule:
pattern: print($$$ARGS)
fix: logger.info($$$ARGS)
```
### no-unwrap (Rust)
```yaml
id: no-unwrap
language: Rust
severity: warning
message: "Avoid .unwrap() in production code; propagate or handle the error."
rule:
pattern: $EXPR.unwrap()
```
---
## See also
- `references/patterns.md` — meta-variable rules.
- `references/yaml-rules.md` — full YAML rule schema (atomic / relational / composite / transform / fix).
- `references/cli.md` — `sg run`, `sg scan`, `sg test`, `sg new`.
- Official catalog: <https://ast-grep.github.io/catalog/> (community-maintained, browse by language).
optional-skills/software-development/ast-grep/references/sgconfig.md
+248
View File
@@ -0,0 +1,248 @@
# sgconfig.yml — project configuration
`sgconfig.yml` lives at your project root (the same place as `package.json`, `Cargo.toml`, `pyproject.toml`, etc.) and tells `sg scan`/`sg test` where to find rules and tests.
`sg` walks **upward** from the current directory until it finds an `sgconfig.yml`. You can also pass `--config <path>` explicitly.
---
## Minimal project layout
```
my-project/
├── sgconfig.yml
├── rules/
│ ├── no-console.yml
│ └── no-as-any.yml
├── utils/
│ └── is-literal.yml
├── tests/
│ ├── no-console.yml
│ └── __snapshots__/
│ └── no-console-snapshot.yml
└── src/
└── ...
```
```yaml
# sgconfig.yml
ruleDirs:
- rules
testConfigs:
- testDir: tests
snapshotDir: __snapshots__
utilDirs:
- utils
```
That's it. `sg scan src/` will load every `.yml` in `rules/`, find every `.ts`/`.py`/whatever matching the rule's `language`, and report violations.
---
## Full schema
```yaml
# Rule directories — required
ruleDirs:
- rules
- team-rules
- vendor/sg-rules
# Test directories — optional
testConfigs:
- testDir: tests
snapshotDir: __snapshots__
- testDir: integration-tests
# Utility rule directories — optional
# Files here become global utilities accessible via `matches: <id>` from any rule.
utilDirs:
- utils
- team-utils
# Override file-extension -> language mapping — optional
# Useful when your code uses non-standard extensions.
languageGlobs:
html:
- '*.vue'
- '*.svelte'
- '*.astro'
json:
- '.eslintrc'
- '.prettierrc'
cpp:
- '*.c' # treat C as C++
tsx:
- '*.ts' # treat all .ts as TSX (so TSX rules work everywhere)
# Custom tree-sitter languages (experimental) — optional
customLanguages:
mojo:
libraryPath: tree-sitter-mojo.so
extensions: [mojo, '🔥']
expandoChar: _ # Replace $ in patterns when language uses $ syntactically
languageSymbol: tree_sitter_mojo
# Language injection — embedded code in another language (experimental) — optional
# Example: CSS inside styled-components template literals.
languageInjections:
- hostLanguage: js
rule:
pattern: 'styled.$TAG`$CONTENT`'
injected: css
```
---
## Field-by-field
### `ruleDirs` (required)
`Array<string>` — directories containing rule YAML files. Resolved relative to `sgconfig.yml`.
Each `.yml`/`.yaml` file in these directories is loaded as a rule. One file can contain multiple rules separated by `---`.
### `testConfigs`
`Array<TestConfig>` where each entry has:
- `testDir` (required): directory of test YAML files.
- `snapshotDir` (optional, default `__snapshots__`): directory for snapshots.
Each test file looks like:
```yaml
id: no-console
valid:
- 'logger.info("hi")'
invalid:
- 'console.log("hi")'
```
`sg test` runs every test, compares matches against the snapshot, and fails on diff. Snapshots are created on first run with `-U`.
### `utilDirs`
`Array<string>` — directories with global utility rules. Each util file must have `id` and `language`. Utils become referenceable via `matches: <id>` from any rule in the project.
### `languageGlobs`
`HashMap<string, Array<string>>` — override which extensions map to which language. Takes precedence over the built-in defaults.
Useful for:
- Custom file extensions (`.eslintrc` is JSON).
- Force-treating `.ts` files as TSX (so JSX-shaped patterns work).
- Vue/Svelte/Astro files (HTML host language).
### `customLanguages` (experimental)
Register a tree-sitter parser that ast-grep doesn't ship with. Requires:
- `libraryPath`: path to a built `.so` / `.dylib` / `.dll` containing the grammar.
- `extensions`: file extensions to recognize.
- `languageSymbol`: the C symbol exported by the grammar (typically `tree_sitter_<name>`).
- `expandoChar` (optional): character to substitute for `$` in patterns when the host language uses `$` syntactically (PHP, jQuery, etc.).
This is **rarely needed** — ast-grep already supports 25 languages out of the box.
### `languageInjections` (experimental)
Match patterns inside embedded languages. Example: CSS inside JS template literals (styled-components, emotion).
```yaml
languageInjections:
- hostLanguage: js
rule:
pattern: 'styled.$TAG`$CONTENT`'
injected: css
```
After this, a `css` rule with pattern `color: $C` will match `$CONTENT` strings.
---
## Common configurations
### Monorepo with shared rules
```
monorepo/
├── sgconfig.yml # root config — applies to entire monorepo
├── shared-rules/
│ ├── no-todo.yml
│ └── no-as-any.yml
└── packages/
├── frontend/
│ ├── sgconfig.yml # extends root with frontend-specific rules
│ └── rules/
└── backend/
├── sgconfig.yml # extends root with backend-specific rules
└── rules/
```
Each package's `sgconfig.yml` references both the package-local rules and the shared ones:
```yaml
# packages/frontend/sgconfig.yml
ruleDirs:
- rules
- ../../shared-rules
```
### Single rule file (no project)
For one-offs, skip `sgconfig.yml` entirely:
```bash
sg scan -r path/to/single-rule.yml src/
```
### Inline rule (no file)
```bash
sg scan --inline-rules '
id: no-todo
language: TypeScript
severity: warning
rule: { pattern: TODO }' src/
```
Multiple rules separated by `---`:
```bash
sg scan --inline-rules '
id: no-todo
language: TypeScript
rule: { pattern: TODO }
---
id: no-fixme
language: TypeScript
rule: { pattern: FIXME }' src/
```
---
## Editor integration
VS Code / Neovim / Helix detect `sgconfig.yml` automatically and surface diagnostics from every rule. Without `sgconfig.yml`, the LSP runs without any rules loaded.
To enable schema validation in your editor, add a header to each rule file:
```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/ast-grep/ast-grep/main/schemas/rule.json
id: no-console
language: TypeScript
rule:
pattern: console.log($_)
```
---
## See also
- `references/yaml-rules.md` — rule schema (atomic / relational / composite / transform / fix).
- `references/cli.md` — `sg scan`, `sg test`, `sg new project`.
- Official: <https://ast-grep.github.io/reference/sgconfig.html>, <https://ast-grep.github.io/guide/project/project-config.html>
optional-skills/software-development/ast-grep/references/yaml-rules.md
+509
View File
@@ -0,0 +1,509 @@
# YAML rule reference — atomic, relational, composite, transform, fix
Use this when you outgrow inline `sg run -p ...` patterns and need a reusable, testable rule. A YAML rule is the unit of work for `sg scan`. Drop one or more files in `ruleDirs/` (configured via `sgconfig.yml`) and they get loaded automatically.
This page is the practical reference. The full upstream docs live at:
- <https://ast-grep.github.io/reference/yaml.html>
- <https://ast-grep.github.io/reference/rule.html>
- <https://ast-grep.github.io/cheatsheet/rule.html>
---
## Skeleton
A single YAML file can hold multiple rules separated by `---`.
```yaml
id: no-console
language: TypeScript
severity: warning
message: "Avoid console.* in production"
note: |
Use a proper logger so we can route logs to stderr in production
and silence them in tests.
url: https://internal.docs/rules/no-console
rule:
pattern: console.$METHOD($$$ARGS)
fix: logger.$METHOD($$$ARGS)
constraints:
METHOD:
not:
regex: '^(error|warn)$'
files:
- 'src/**/*.ts'
ignores:
- 'src/**/*.test.ts'
metadata:
category: logging
```
---
## Top-level fields
| Field | Required | Description |
|---|---|---|
| `id` | yes | Unique identifier. Use `kebab-case`. |
| `language` | yes | One of: `Bash`, `C`, `Cpp`, `CSharp`, `Css`, `Elixir`, `Go`, `Haskell`, `Html`, `Java`, `JavaScript`, `Json`, `Kotlin`, `Lua`, `Nix`, `Php`, `Python`, `Ruby`, `Rust`, `Scala`, `Solidity`, `Swift`, `TypeScript`, `Tsx`, `Yaml`. **Capitalized PascalCase** is canonical, but lowercase often works. |
| `rule` | yes | The matching logic. Object containing one or more atomic / relational / composite rules. |
| `constraints` | no | Filter on captured single-metavariables (`$VAR`, not `$$$`). |
| `utils` | no | Local utility rules referenced by `matches:` in this file. |
| `transform` | no | Manipulate metavariable strings before `fix`. |
| `fix` | no | String or `FixConfig` for auto-rewrite. |
| `rewriters` | no | Rewriter rules for the `rewrite` transform. |
| `severity` | no | `hint` \| `info` \| `warning` \| `error` \| `off` (default: `hint`). |
| `message` | no | Concise lint message. May reference `$VAR` capture text. |
| `note` | no | Detailed markdown explanation (no `$VAR` interpolation). |
| `labels` | no | Custom diagnostic highlighting per-metavariable. |
| `files` | no | Glob include list. |
| `ignores` | no | Glob exclude list. |
| `url` | no | Doc link shown in editor diagnostics. |
| `metadata` | no | Free-form data ignored by `sg`, useful for external tooling. |
---
## Atomic rules — match a single node
### `pattern`
Match by structural pattern. The most common rule.
```yaml
# String form
rule:
pattern: console.log($MSG)
# Object form (when context is needed)
rule:
pattern:
context: 'class C { $FIELD = $INIT }'
selector: field_definition
strictness: relaxed # optional, default: smart
```
### `kind`
Match by AST node type name. Tree-sitter grammar-specific.
```yaml
rule:
kind: call_expression
```
ast-grep 0.39+ supports limited ESQuery selectors:
```yaml
rule:
kind: call_expression > identifier # direct child
kind: call_expression + identifier # next sibling
kind: call_expression ~ identifier # following sibling
kind: call_expression identifier # descendant
```
To find the right `kind`, parse a known-good file:
```bash
sg run -p '$_' --lang ts --debug-query=cst src/foo.ts | head -40
```
### `regex`
Match node text against a Rust regex. Whole-text match (no partial). Always combine with `kind` or `pattern` for performance.
```yaml
rule:
all:
- kind: identifier
- regex: '^[A-Z][a-z]+$' # PascalCase
```
Inline flags work: `(?i)apple`, `(?m)^foo`. No look-around, no backreferences.
### `nthChild`
Match by 1-based index among **named** siblings. Inspired by CSS `:nth-child`.
```yaml
rule:
nthChild: 1 # first sibling
# Functional form
rule:
nthChild: 2n+1 # odd siblings
# With reverse and ofRule
rule:
nthChild:
position: 1
reverse: true # last
ofRule:
kind: function_declaration
```
### `range`
Match by character range. Useful for tooling that pinpoints a known location.
```yaml
rule:
range:
start: { line: 0, column: 0 }
end: { line: 0, column: 11 }
```
---
## Relational rules — match by relation to other nodes
All four take a sub-rule object plus optional `stopBy` and (for `inside`/`has`) `field`.
### `inside` — target is inside parent/ancestor matching sub-rule
```yaml
rule:
pattern: this.$PROP
inside:
kind: class_body
stopBy: end # walk up to file root, default: neighbor
```
### `has` — target has child/descendant matching sub-rule
```yaml
rule:
kind: function_declaration
has:
kind: throw_statement
stopBy: end
```
### `precedes` — target appears before sibling matching sub-rule
```yaml
rule:
kind: import_statement
precedes:
kind: function_declaration
```
### `follows` — target appears after sibling matching sub-rule
```yaml
rule:
pattern: super($$$)
follows:
pattern: $X = $Y
```
### `stopBy`
| Value | Behavior |
|---|---|
| `"neighbor"` (default) | Stop at immediate parent/child/sibling. |
| `"end"` | Walk all the way to root / leaf / sequence boundary. |
| Rule object | Stop when sub-rule matches (inclusive). |
### `field`
Specify the semantic role of the target inside its parent (e.g. `name`, `body`, `value`, `key`).
```yaml
rule:
kind: pair
has:
field: key
regex: '^password$'
```
---
## Composite rules — combine sub-rules
| Rule | Meaning |
|---|---|
| `all` | All sub-rules must match the same target node. Metavariables from all sub-rules merge. |
| `any` | At least one sub-rule must match. Only metavars from the matched branch survive. |
| `not` | Inverse: target must NOT match the sub-rule. |
| `matches` | Reference a utility rule by id. |
```yaml
rule:
all:
- kind: call_expression
- pattern: $FN($$$ARGS)
- inside:
kind: function_declaration
stopBy: end
rule:
any:
- pattern: console.log($X)
- pattern: console.warn($X)
- pattern: console.error($X)
rule:
all:
- pattern: $E.unwrap()
- not:
inside:
kind: function_item
has:
kind: result_type
stopBy: end
rule:
matches: is-react-component
```
> Composites apply to a **single** target. To express "node X has BOTH a number child AND a string child," use two relational rules at the top level, not `all` inside `has`. See `references/pitfalls.md` §10.
---
## Implicit `all` — multiple rule fields
A rule object with multiple fields is treated as an implicit `all`:
```yaml
# These two are equivalent
rule:
pattern: this.$PROP
inside: { kind: class_body }
rule:
all:
- pattern: this.$PROP
- inside: { kind: class_body }
```
Use the explicit `all` array when capture order matters (rare, but possible with downstream `transform`).
---
## `constraints` — post-match metavariable filtering
After the main `rule` matches, additional checks on captured single metavariables:
```yaml
rule:
pattern: function $NAME($$$P) { $$$B }
constraints:
NAME:
regex: '^[a-z][a-zA-Z0-9]*$' # camelCase only
not:
regex: '^_' # not starting with _
```
Constraints **only apply to single metavars** (`$VAR`), not multi (`$$$VAR`).
---
## `utils` — local reusable sub-rules
```yaml
utils:
is-literal:
any:
- kind: number
- kind: string
- kind: 'true'
- kind: 'false'
rule:
all:
- pattern: $X = $Y
- has:
matches: is-literal # references utils.is-literal
```
For utils accessible across multiple rule files, use `utilDirs` in `sgconfig.yml` and put each util in its own YAML file with `id` and `language`.
---
## `transform` — manipulate captures before `fix`
Operations: `replace`, `substring`, `convert`, `rewrite`.
### `replace` — regex search/replace on a captured string
```yaml
rule:
pattern: $OLD_FN($$$A)
constraints:
OLD_FN:
regex: '^debug_'
transform:
NEW_FN:
replace:
source: $OLD_FN
replace: '^debug_'
by: 'release_'
fix: $NEW_FN($$$A)
```
### `substring` — character slicing (negative indices supported)
```yaml
transform:
INNER:
substring:
source: $WRAPPED
startChar: 1
endChar: -1
```
### `convert` — case conversion
```yaml
transform:
KEBAB:
convert:
source: $CAMEL
toCase: kebabCase # camelCase | snakeCase | kebabCase | pascalCase | upperCase | lowerCase | capitalize
separatedBy: [underscore] # optional: dash | dot | space | slash | underscore | caseChange
```
### `rewrite` — apply other rewriter rules (experimental)
```yaml
rewriters:
- id: stringify
rule: { pattern: "'' + $A" }
fix: "String($A)"
rule:
pattern: stringify-all($EXPR)
transform:
REWRITTEN:
rewrite:
source: $EXPR
rewriters: [stringify]
joinBy: "\n"
fix: $REWRITTEN
```
### Transforms can chain
Later transforms can reference variables produced by earlier ones:
```yaml
transform:
KEBABED:
convert: { source: $X, toCase: kebabCase }
PREFIXED:
replace:
source: $KEBABED
replace: '^'
by: 'css-'
fix: $PREFIXED
```
---
## `fix` — auto-rewrite
### String form
```yaml
fix: logger.log($$$ARGS)
# Empty string deletes the match
fix: ""
```
### FixConfig form (for list-item deletion that needs to expand the range)
When deleting one item from a comma-separated list, you also need to remove the trailing comma. Use `expandEnd`:
```yaml
rule:
kind: pair
has:
field: key
regex: '^password$'
fix:
template: ''
expandEnd:
regex: ','
```
`expandStart` and `expandEnd` accept `regex` matching characters that should be absorbed into the rewrite range.
---
## `rewriters` — sub-rule library for `rewrite` transform
Top-level field defining one or more named rewriters:
```yaml
rewriters:
- id: nullable-to-optional
rule: { pattern: $X | null }
fix: '$X | undefined'
- id: stringify
rule: { pattern: "'' + $A" }
fix: 'String($A)'
```
Used inside `transform` via the `rewrite` operation (see above).
---
## `labels` — custom diagnostic highlighting
```yaml
rule:
pattern: $FN($$$ARGS)
labels:
FN:
style: primary
message: "this function shouldn't be called"
ARGS:
style: secondary
message: "with these arguments"
```
Editor extensions render the diagnostic with these labels. Defaults are usually fine.
---
## `files` and `ignores` — file selection per-rule
```yaml
files:
- 'src/**/*.ts'
- 'lib/**/*.ts'
ignores:
- 'src/**/*.test.ts'
- '**/__generated__/**'
```
If omitted, the rule runs on every file matching its `language`. These globs override `sgconfig.yml`-level globs for this rule only.
Object form (rare):
```yaml
files:
- pattern: 'src/**/*.ts'
case_sensitive: true
```
---
## See also
- `references/recipes.md` — copy-paste rules by language.
- `references/cli.md` — `sg scan`, `sg test`.
- `references/sgconfig.md` — project-level configuration.
- Official rule reference: <https://ast-grep.github.io/reference/rule.html>
- Cheat sheets: <https://ast-grep.github.io/cheatsheet/rule.html>, <https://ast-grep.github.io/cheatsheet/yaml.html>
optional-skills/software-development/ast-grep/scripts/ast_grep_helper.py
Executable
+761
View File
@@ -0,0 +1,761 @@
#!/usr/bin/env python3
"""ast-grep-helper: a thin LLM-friendly wrapper around `sg` (ast-grep).
Single-file Python 3 stdlib. No deps. Works on macOS, Linux, Windows, WSL.
WHAT IT ADDS over plain `sg`:
1. Binary auto-resolution: cached -> @ast-grep/cli -> PATH -> Homebrew -> error with install hint
2. Pattern hint validation: detects regex misuse (\\w, .*, |, [a-z]) and language-specific
mistakes (Python trailing colon, JS/Go/Rust missing function body) BEFORE calling sg
3. Two-pass replace: ast-grep silently ignores --update-all when --json is set, so we run
a JSON pass to collect matches, then a separate --update-all pass to mutate files
4. Stable JSON output: parses sg --json=compact, salvages truncated output, normalizes shape
5. Cross-OS path handling: works the same on POSIX and Windows (uses pathlib + shutil)
USAGE
ast_grep_helper.py search PATTERN [PATH...] [--lang LANG] [--globs GLOB ...] [-C N]
ast_grep_helper.py replace PATTERN REWRITE [PATH...] [--lang LANG] [--apply] [--globs GLOB ...]
ast_grep_helper.py scan RULE_FILE [PATH...] [--apply] [--report-style STYLE]
ast_grep_helper.py test [-c CONFIG] [-t TEST_DIR] [-U]
ast_grep_helper.py new {project,rule,test,util} [NAME] [--lang LANG]
ast_grep_helper.py langs # list 25 supported languages
ast_grep_helper.py doctor # check binary availability + version
ast_grep_helper.py install # delegate to ../install.sh / install.ps1
ast_grep_helper.py validate PATTERN [--lang LANG] # offline pattern hint check only
ast_grep_helper.py --version
ast_grep_helper.py --help
EXAMPLES
# Find all console.log calls in TypeScript
ast_grep_helper.py search 'console.log($MSG)' --lang ts src/
# Migrate console.log -> logger.info (dry-run preview)
ast_grep_helper.py replace 'console.log($MSG)' 'logger.info($MSG)' --lang ts src/
# Apply the same replacement
ast_grep_helper.py replace 'console.log($MSG)' 'logger.info($MSG)' --lang ts src/ --apply
# Validate a pattern offline (no sg call, no filesystem access)
ast_grep_helper.py validate '\\w+' --lang ts
# -> exit 2, hint: "regex \\w not supported. Use $VAR for identifiers."
EXIT CODES
0 Success (matches found OR replacement applied OR validation passed)
1 Argument error
2 Pattern hint failure (regex misuse, missing body, etc.) - call would have failed
3 ast-grep binary not found and auto-install declined
4 ast-grep call failed (returned non-zero, with stderr forwarded)
5 Timeout (5 minutes per call by default)
"""
from __future__ import annotations
import argparse
import json
import os
import platform
import re
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Optional
VERSION = "0.1.0"
# 25 CLI languages supported by ast-grep, with their aliases (mirrors official docs)
LANGUAGES: dict[str, list[str]] = {
"bash": [".bash", ".sh", ".zsh"],
"c": [".c", ".h"],
"cpp": [".cc", ".cpp", ".cxx", ".hpp", ".hxx"],
"csharp": [".cs"],
"css": [".css"],
"elixir": [".ex", ".exs"],
"go": [".go"],
"haskell": [".hs"],
"html": [".html", ".htm"],
"java": [".java"],
"javascript": [".js", ".jsx", ".cjs", ".mjs"],
"json": [".json"],
"kotlin": [".kt", ".kts"],
"lua": [".lua"],
"nix": [".nix"],
"php": [".php"],
"python": [".py", ".pyi"],
"ruby": [".rb"],
"rust": [".rs"],
"scala": [".scala"],
"solidity": [".sol"],
"swift": [".swift"],
"typescript": [".ts", ".cts", ".mts"],
"tsx": [".tsx"],
"yaml": [".yml", ".yaml"],
}
# Aliases that ast-grep CLI accepts; we normalize to the canonical name.
LANG_ALIASES: dict[str, str] = {
"js": "javascript", "jsx": "javascript",
"ts": "typescript",
"py": "python", "py3": "python",
"rb": "ruby",
"rs": "rust",
"kt": "kotlin",
"ex": "elixir",
"hs": "haskell",
"sh": "bash", "zsh": "bash",
"cc": "cpp", "c++": "cpp", "cxx": "cpp",
"cs": "csharp",
"yml": "yaml",
"sol": "solidity",
"golang": "go",
}
# Default search timeout (5 min). ast-grep calls can be slow on huge repos.
DEFAULT_TIMEOUT_S = 300
# ---------- logging ----------
def trace(msg: str) -> None:
"""Print a trace line to stderr (suppressible via --quiet, default off)."""
if not _QUIET:
print(f"[ast-grep-helper] {msg}", file=sys.stderr, flush=True)
def err(msg: str) -> None:
"""Print an error line to stderr (always shown)."""
print(f"[ast-grep-helper] error: {msg}", file=sys.stderr, flush=True)
_QUIET = False
# ---------- binary resolution ----------
def script_dir() -> Path:
return Path(__file__).resolve().parent
def skill_root() -> Path:
return script_dir().parent
def cached_binary() -> Optional[Path]:
"""Look in <skill_root>/bin/ for a previously downloaded binary."""
binname = "sg.exe" if os.name == "nt" else "sg"
altname = "ast-grep.exe" if os.name == "nt" else "ast-grep"
for name in (binname, altname):
p = skill_root() / "bin" / name
if p.is_file() and os.access(p, os.X_OK):
return p
return None
def npm_binary() -> Optional[Path]:
"""If @ast-grep/cli is installed globally via npm, find its binary."""
# `sg` shipped by @ast-grep/cli is on PATH when npm prefix bin is on PATH.
# We rely on shutil.which for that case.
return None # handled by which_binary
def which_binary() -> Optional[Path]:
"""Use shutil.which to find sg or ast-grep on PATH.
On Linux, plain `sg` collides with the setgroups command from util-linux
(sometimes called via /usr/bin/sg) which has flag --version that returns
non-zero, so we prefer `ast-grep` when both are on PATH and the `sg` we find
is the wrong one.
"""
for name in ("ast-grep", "sg"):
found = shutil.which(name)
if found:
p = Path(found)
# On Linux, double-check by trying --version. The util-linux `sg`
# rejects --version, while ast-grep prints "ast-grep <version>".
if name == "sg" and platform.system() == "Linux":
try:
out = subprocess.run(
[str(p), "--version"],
capture_output=True,
text=True,
timeout=5,
)
if out.returncode != 0 or "ast-grep" not in (out.stdout + out.stderr).lower():
continue
except Exception:
continue
return p
return None
def homebrew_binary() -> Optional[Path]:
"""Common Homebrew install paths."""
candidates = [
Path("/opt/homebrew/bin/ast-grep"),
Path("/opt/homebrew/bin/sg"),
Path("/usr/local/bin/ast-grep"),
Path("/usr/local/bin/sg"),
]
for p in candidates:
if p.is_file() and os.access(p, os.X_OK):
return p
return None
# --- OMO runtime resolution (vendored patch) ---
def omo_env_binary() -> Optional[Path]:
raw_path = os.environ.get("OMO_AST_GREP_SG_PATH")
if not raw_path:
return None
path = Path(raw_path).expanduser()
if path.is_file() and os.access(path, os.X_OK):
return path
return None
def omo_runtime_slug() -> str:
if sys.platform.startswith("win"):
os_slug = "win32"
elif sys.platform == "darwin":
os_slug = "darwin"
else:
os_slug = "linux"
machine = platform.machine().lower()
arch_slug = "arm64" if machine in {"arm64", "aarch64"} else "x64"
return f"{os_slug}-{arch_slug}"
def omo_runtime_binary() -> Optional[Path]:
binary_name = "sg.exe" if sys.platform.startswith("win") else "sg"
slug = omo_runtime_slug()
candidates: list[Path] = []
codex_home = os.environ.get("CODEX_HOME")
if codex_home:
candidates.append(Path(codex_home) / "runtime" / "ast-grep" / slug / binary_name)
candidates.append(Path.home() / ".omo" / "runtime" / "ast-grep" / slug / binary_name)
for path in candidates:
if path.is_file() and os.access(path, os.X_OK):
return path
return None
def resolve_binary() -> Optional[Path]:
"""Resolve the ast-grep binary in priority order.
1. OMO_AST_GREP_SG_PATH override
2. OMO runtime dirs
3. Cached binary in <skill>/bin/
4. PATH (via shutil.which)
5. Homebrew default paths
"""
for fn in (omo_env_binary, omo_runtime_binary, cached_binary, which_binary, homebrew_binary):
result = fn()
if result:
return result
return None
def require_binary() -> Path:
"""Resolve binary, or print an actionable install hint and exit 3."""
p = resolve_binary()
if p:
return p
err("ast-grep binary not found.")
err("")
err("Install via one of:")
err(f" bash {skill_root()}/install.sh # POSIX (auto-detects best method)")
err(f" pwsh {skill_root()}/install.ps1 # Windows")
err("")
err("Or manually:")
err(" brew install ast-grep # macOS / linuxbrew")
err(" npm install -g @ast-grep/cli # any OS with Node")
err(" cargo install ast-grep --locked # any OS with Rust")
err(" pip install ast-grep-cli # any OS with Python")
err(" scoop install main/ast-grep # Windows / Scoop")
err("")
err("See references/install.md for the full table.")
sys.exit(3)
# ---------- pattern hint validation ----------
# Regex anti-patterns that ast-grep does NOT support but LLMs frequently emit.
# Each tuple: (regex_to_detect, hint_message)
REGEX_ANTIPATTERNS: list[tuple[re.Pattern[str], str]] = [
(re.compile(r"\\w|\\d|\\s|\\b"),
"Backslash escapes (\\w, \\d, \\s, \\b) are regex syntax, not ast-grep. "
"Use $VAR to capture any identifier, or switch to grep for text patterns."),
(re.compile(r"(?<!\$)\.\*|(?<!\$)\.\+"),
"'.*' and '.+' are regex wildcards, not ast-grep. "
"Use $$$ between AST fragments to match many nodes, or $VAR for one node."),
(re.compile(r"\[[a-zA-Z0-9-]+\]"),
"Character classes like '[a-z]' are regex syntax. "
"ast-grep has no AST equivalent - use grep for character-level patterns."),
]
def find_alternation(pattern: str) -> bool:
"""Detect a literal '|' that is not inside a string/template literal.
Heuristic - mark as alternation if `|` appears outside obvious string contexts.
"""
# Strip simple string contents to reduce false positives in patterns like
# `'a|b'` or `"x|y"`. This is a heuristic, not a parser.
stripped = re.sub(r"'[^']*'|\"[^\"]*\"|`[^`]*`", "", pattern)
# Require word chars on both sides to avoid catching bitwise or ||
return bool(re.search(r"\w\s*\|\s*\w", stripped)) and "||" not in stripped
def lang_specific_hints(pattern: str, lang: Optional[str]) -> list[str]:
"""Return a list of hints for language-specific common mistakes."""
if not lang:
return []
canonical = LANG_ALIASES.get(lang.lower(), lang.lower())
hints: list[str] = []
if canonical == "python":
# def foo($$$): <-- trailing colon breaks the parse
if re.search(r"^\s*(def|class)\s+\$?\w+[^:]*:\s*$", pattern, re.MULTILINE):
hints.append(
"Python pattern has trailing ':'. ast-grep parses pattern as a complete "
"definition - drop the trailing colon. Try: 'def $FUNC($$$)' or 'class $C($$$)'."
)
if canonical in ("javascript", "typescript", "tsx"):
if re.search(r"^\s*(async\s+)?function\s+\$?\w+\s*$", pattern):
hints.append(
"JS/TS function pattern is incomplete. Add params and body: "
"'function $NAME($$$) { $$$ }'."
)
if canonical == "go":
if re.search(r"^\s*func\s+\$?\w+\s*$", pattern):
hints.append(
"Go function pattern is incomplete. Add params and body: "
"'func $NAME($$$) { $$$ }'."
)
if canonical == "rust":
if re.search(r"^\s*fn\s+\$?\w+\s*$", pattern):
hints.append(
"Rust fn pattern is incomplete. Add params, return type, and body: "
"'fn $NAME($$$) -> $RET { $$$ }' (or '-> ()' if returning unit)."
)
return hints
def validate_pattern(pattern: str, lang: Optional[str]) -> list[str]:
"""Return a list of hints. Empty list = pattern looks plausible."""
hints: list[str] = []
for rx, msg in REGEX_ANTIPATTERNS:
if rx.search(pattern):
hints.append(msg)
if find_alternation(pattern):
hints.append(
"Literal '|' alternation is regex syntax, not ast-grep. "
"Run two separate ast-grep calls (one per alternative), or switch to grep."
)
hints.extend(lang_specific_hints(pattern, lang))
return hints
def normalize_lang(lang: Optional[str]) -> Optional[str]:
if not lang:
return None
canonical = LANG_ALIASES.get(lang.lower(), lang.lower())
if canonical not in LANGUAGES:
err(f"unknown language '{lang}'. Run 'ast_grep_helper.py langs' for the full list.")
sys.exit(1)
return canonical
# ---------- subprocess helpers ----------
def run_sg(
binary: Path,
args: list[str],
*,
timeout: int = DEFAULT_TIMEOUT_S,
capture: bool = True,
) -> subprocess.CompletedProcess[str]:
"""Spawn `sg <args>` with a hard timeout. Capture stdout/stderr by default."""
cmd = [str(binary), *args]
trace(f"exec: {' '.join(cmd)}")
try:
return subprocess.run(
cmd,
capture_output=capture,
text=True,
timeout=timeout,
)
except subprocess.TimeoutExpired:
err(f"ast-grep call timed out after {timeout}s")
sys.exit(5)
# ---------- subcommands ----------
def cmd_search(args: argparse.Namespace) -> int:
pattern: str = args.pattern
lang = normalize_lang(args.lang)
hints = validate_pattern(pattern, lang)
if hints:
err("pattern looks invalid for ast-grep:")
for h in hints:
err(f" - {h}")
if not args.force:
err("(pass --force to call ast-grep anyway)")
return 2
binary = require_binary()
sg_args = ["run", "-p", pattern, "--json=compact"]
if lang:
sg_args.extend(["--lang", lang])
if args.context:
sg_args.extend(["-C", str(args.context)])
for g in args.globs or []:
sg_args.extend(["--globs", g])
sg_args.extend(args.paths or ["."])
proc = run_sg(binary, sg_args)
if proc.returncode not in (0, 1): # 0=match, 1=no match - both fine
sys.stderr.write(proc.stderr or "")
return 4
matches = parse_compact_json(proc.stdout)
if args.json_out:
json.dump(matches, sys.stdout, indent=2)
print()
else:
format_matches(matches)
if not matches:
# Re-run pattern hints in case empty result was caused by something subtle.
# Already done above; here we just give a generic suggestion.
trace("no matches. If you expected matches, double-check --lang and the pattern shape.")
return 0
def cmd_replace(args: argparse.Namespace) -> int:
pattern: str = args.pattern
rewrite: str = args.rewrite
lang = normalize_lang(args.lang)
pattern_hints = validate_pattern(pattern, lang)
rewrite_hints = validate_pattern(rewrite, lang)
all_hints = []
if pattern_hints:
all_hints.append("pattern issues:")
all_hints.extend(f" - {h}" for h in pattern_hints)
if rewrite_hints:
all_hints.append("rewrite issues:")
all_hints.extend(f" - {h}" for h in rewrite_hints)
if all_hints:
err("input looks invalid for ast-grep:")
for line in all_hints:
err(line)
if not args.force:
err("(pass --force to call ast-grep anyway)")
return 2
binary = require_binary()
# Pass 1: dry-run via JSON to collect what would change.
sg_args1 = ["run", "-p", pattern, "-r", rewrite, "--json=compact"]
if lang:
sg_args1.extend(["--lang", lang])
for g in args.globs or []:
sg_args1.extend(["--globs", g])
sg_args1.extend(args.paths or ["."])
proc1 = run_sg(binary, sg_args1)
if proc1.returncode not in (0, 1):
sys.stderr.write(proc1.stderr or "")
return 4
matches = parse_compact_json(proc1.stdout)
if not matches:
trace("no matches; nothing to replace.")
return 0
if not args.apply:
# Show the dry-run preview and exit.
print(f"DRY-RUN: would rewrite {len(matches)} match(es) across "
f"{len({m['file'] for m in matches})} file(s):")
format_matches(matches, show_replacement=True)
print()
print("Re-run with --apply to mutate files.")
return 0
# Pass 2: apply with --update-all (no --json; sg silently ignores --update-all
# when --json is present, so we MUST run a second invocation).
sg_args2 = ["run", "-p", pattern, "-r", rewrite, "--update-all"]
if lang:
sg_args2.extend(["--lang", lang])
for g in args.globs or []:
sg_args2.extend(["--globs", g])
sg_args2.extend(args.paths or ["."])
proc2 = run_sg(binary, sg_args2)
if proc2.returncode not in (0, 1):
sys.stderr.write(proc2.stderr or "")
return 4
print(f"APPLIED: rewrote {len(matches)} match(es) across "
f"{len({m['file'] for m in matches})} file(s).")
return 0
def cmd_scan(args: argparse.Namespace) -> int:
binary = require_binary()
sg_args = ["scan"]
if args.config:
sg_args.extend(["-c", args.config])
if args.rule:
sg_args.extend(["-r", args.rule])
if args.inline_rules:
sg_args.extend(["--inline-rules", args.inline_rules])
if args.report_style:
sg_args.extend(["--report-style", args.report_style])
if args.apply:
sg_args.append("-U")
sg_args.extend(args.paths or [])
proc = run_sg(binary, sg_args, capture=False)
return proc.returncode
def cmd_test(args: argparse.Namespace) -> int:
binary = require_binary()
sg_args = ["test"]
if args.config:
sg_args.extend(["-c", args.config])
if args.test_dir:
sg_args.extend(["-t", args.test_dir])
if args.update:
sg_args.append("-U")
proc = run_sg(binary, sg_args, capture=False)
return proc.returncode
def cmd_new(args: argparse.Namespace) -> int:
binary = require_binary()
sg_args = ["new", args.what]
if args.name:
sg_args.append(args.name)
if args.lang:
sg_args.extend(["--lang", args.lang])
if args.yes:
sg_args.append("--yes")
proc = run_sg(binary, sg_args, capture=False)
return proc.returncode
def cmd_langs(_args: argparse.Namespace) -> int:
print("ast-grep supported languages (25):")
for lang, exts in sorted(LANGUAGES.items()):
print(f" {lang:<12} {' '.join(exts)}")
print()
print("Aliases accepted by --lang:")
for alias, canonical in sorted(LANG_ALIASES.items()):
print(f" {alias:<8} -> {canonical}")
return 0
def cmd_doctor(_args: argparse.Namespace) -> int:
print(f"ast-grep-helper v{VERSION}")
print(f"Python: {sys.version.split()[0]}")
print(f"Platform: {platform.system()} {platform.release()} ({platform.machine()})")
print(f"Skill: {skill_root()}")
print()
binary = resolve_binary()
if not binary:
print("ast-grep binary: NOT FOUND")
print(" -> run: bash install.sh (POSIX) or pwsh install.ps1 (Windows)")
return 1
print(f"ast-grep binary: {binary}")
proc = run_sg(binary, ["--version"], timeout=5)
if proc.returncode == 0:
print(f" version: {proc.stdout.strip()}")
else:
print(f" --version returned exit {proc.returncode}")
print(f" stderr: {proc.stderr.strip()}")
return 1
return 0
def cmd_install(_args: argparse.Namespace) -> int:
"""Delegate to install.sh / install.ps1 in the skill root."""
if os.name == "nt":
installer = skill_root() / "install.ps1"
cmd = ["pwsh", "-File", str(installer)]
else:
installer = skill_root() / "install.sh"
cmd = ["bash", str(installer)]
if not installer.is_file():
err(f"installer not found: {installer}")
return 1
trace(f"running installer: {' '.join(cmd)}")
return subprocess.run(cmd).returncode
def cmd_validate(args: argparse.Namespace) -> int:
"""Offline pattern validation. No sg call. Useful for CI / quick checks."""
lang = normalize_lang(args.lang) if args.lang else None
hints = validate_pattern(args.pattern, lang)
if hints:
for h in hints:
print(f"hint: {h}")
return 2
print("pattern looks plausible for ast-grep.")
return 0
# ---------- output formatting ----------
def parse_compact_json(text: str) -> list[dict]:
"""Parse `sg --json=compact` output. Salvages partial output when truncated."""
if not text.strip():
return []
try:
data = json.loads(text)
if isinstance(data, list):
return data
return []
except json.JSONDecodeError:
# Try line-by-line salvage for truncated output.
results = []
for line in text.splitlines():
line = line.strip().rstrip(",")
if not line.startswith("{"):
continue
try:
obj = json.loads(line)
if isinstance(obj, dict):
results.append(obj)
except json.JSONDecodeError:
continue
return results
def format_matches(matches: list[dict], *, show_replacement: bool = False) -> None:
if not matches:
print("(no matches)")
return
by_file: dict[str, list[dict]] = {}
for m in matches:
by_file.setdefault(m.get("file", "?"), []).append(m)
for path, items in sorted(by_file.items()):
print(f"{path} ({len(items)} match{'es' if len(items) != 1 else ''})")
for m in items:
r = m.get("range", {})
start = r.get("start", {})
line = start.get("line", "?")
col = start.get("column", "?")
text = (m.get("text") or "").splitlines()
preview = text[0] if text else ""
print(f" {path}:{line}:{col} {preview}")
if show_replacement and "replacement" in m:
rep = (m.get("replacement") or "").splitlines()
rep_preview = rep[0] if rep else ""
print(f" -> {rep_preview}")
# ---------- argparse ----------
def build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(
prog="ast-grep-helper",
description="LLM-friendly wrapper around ast-grep (sg).",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
p.add_argument("--version", action="version", version=f"ast-grep-helper {VERSION}")
p.add_argument("--quiet", "-q", action="store_true", help="Suppress trace lines on stderr.")
sub = p.add_subparsers(dest="cmd", required=True, metavar="COMMAND")
s = sub.add_parser("search", help="Search code by AST pattern.")
s.add_argument("pattern", help="AST pattern, e.g. 'console.log($MSG)'")
s.add_argument("paths", nargs="*", help="Paths to search (default: '.')")
s.add_argument("--lang", "-l", help="Language (e.g. ts, py, go, rust). See: langs subcommand.")
s.add_argument("--globs", action="append", help="Include/exclude glob (repeat; prefix '!' to exclude).")
s.add_argument("--context", "-C", type=int, help="Lines of context around each match.")
s.add_argument("--json-out", action="store_true", help="Emit raw JSON instead of human format.")
s.add_argument("--force", action="store_true", help="Skip pattern hint validation.")
s.set_defaults(func=cmd_search)
r = sub.add_parser("replace", help="Rewrite code by AST pattern (dry-run by default).")
r.add_argument("pattern", help="AST pattern.")
r.add_argument("rewrite", help="Replacement pattern (can reuse $VAR from pattern).")
r.add_argument("paths", nargs="*", help="Paths (default: '.')")
r.add_argument("--lang", "-l", help="Language.")
r.add_argument("--globs", action="append", help="Include/exclude glob.")
r.add_argument("--apply", action="store_true", help="Mutate files (default: dry-run preview).")
r.add_argument("--force", action="store_true", help="Skip pattern hint validation.")
r.set_defaults(func=cmd_replace)
sc = sub.add_parser("scan", help="Run YAML-rule-based scan.")
sc.add_argument("paths", nargs="*", help="Paths to scan.")
sc.add_argument("--config", "-c", help="Path to sgconfig.yml.")
sc.add_argument("--rule", "-r", help="Single rule file.")
sc.add_argument("--inline-rules", help="Inline YAML rule string.")
sc.add_argument("--report-style", choices=["rich", "medium", "short"], help="Report style.")
sc.add_argument("--apply", "-U", action="store_true", help="Apply fixes (default: report only).")
sc.set_defaults(func=cmd_scan)
t = sub.add_parser("test", help="Run ast-grep snapshot tests.")
t.add_argument("--config", "-c", help="Path to sgconfig.yml.")
t.add_argument("--test-dir", "-t", help="Test directory.")
t.add_argument("--update", "-U", action="store_true", help="Update snapshots.")
t.set_defaults(func=cmd_test)
n = sub.add_parser("new", help="Scaffold a new project / rule / test / util.")
n.add_argument("what", choices=["project", "rule", "test", "util"], help="What to create.")
n.add_argument("name", nargs="?", help="Name of the artifact.")
n.add_argument("--lang", "-l", help="Language.")
n.add_argument("--yes", "-y", action="store_true", help="Accept defaults.")
n.set_defaults(func=cmd_new)
sub.add_parser("langs", help="List supported languages.").set_defaults(func=cmd_langs)
sub.add_parser("doctor", help="Check ast-grep binary availability.").set_defaults(func=cmd_doctor)
sub.add_parser("install", help="Run the install script for this OS.").set_defaults(func=cmd_install)
v = sub.add_parser("validate", help="Validate a pattern offline (pattern hint check only).")
v.add_argument("pattern", help="AST pattern.")
v.add_argument("--lang", "-l", help="Language for language-specific hints.")
v.set_defaults(func=cmd_validate)
return p
def main(argv: Optional[list[str]] = None) -> int:
global _QUIET
parser = build_parser()
# Accept `search PATTERN --lang js .` — plain parse_args greedily
# finalizes the nargs='*' paths list before the optional, then errors
# "unrecognized arguments: ." on the trailing path.
# parse_intermixed_args cannot be used with subparsers, so collect the
# leftover non-flag tokens and fold them into `paths` ourselves.
args, extras = parser.parse_known_args(argv)
bad = [tok for tok in extras if tok.startswith("-")]
if bad:
parser.error(f"unrecognized arguments: {' '.join(bad)}")
if extras:
if hasattr(args, "paths"):
args.paths = list(getattr(args, "paths") or []) + extras
else:
parser.error(f"unrecognized arguments: {' '.join(extras)}")
_QUIET = bool(getattr(args, "quiet", False))
return args.func(args)
if __name__ == "__main__":
sys.exit(main())
optional-skills/software-development/ast-grep/tests/smoke.ps1
+123
View File
@@ -0,0 +1,123 @@
#Requires -Version 5.1
# Smoke test for the ast-grep skill on Windows (PowerShell 5.1+).
$ErrorActionPreference = 'Stop'
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$SkillDir = Split-Path -Parent $ScriptDir
$Helper = Join-Path $SkillDir 'scripts/ast_grep_helper.py'
$Python = if (Get-Command py -ErrorAction SilentlyContinue) { 'py' } else { 'python' }
$Output = Join-Path $env:TEMP ("ast-grep-skill-smoke-" + [guid]::NewGuid().ToString('N').Substring(0,8))
New-Item -ItemType Directory -Path $Output -Force | Out-Null
function Pass([string]$msg) { Write-Host "PASS: $msg" }
function Fail([string]$msg) { Write-Host "FAIL: $msg" -ForegroundColor Red; Remove-Item -Recurse -Force $Output -ErrorAction SilentlyContinue; exit 1 }
function Run([string[]]$Args) {
$stdoutFile = Join-Path $Output ("out-" + [guid]::NewGuid().ToString('N').Substring(0,8) + ".txt")
$proc = Start-Process -FilePath $Python -ArgumentList (@($Helper) + $Args) -NoNewWindow -PassThru -Wait -RedirectStandardOutput $stdoutFile -RedirectStandardError "$stdoutFile.err"
$stdout = if (Test-Path $stdoutFile) { Get-Content $stdoutFile -Raw } else { '' }
$stderr = if (Test-Path "$stdoutFile.err") { Get-Content "$stdoutFile.err" -Raw } else { '' }
return [pscustomobject]@{
ExitCode = $proc.ExitCode
Stdout = $stdout
Stderr = $stderr
Combined = "$stdout`n$stderr"
}
}
try {
# 1. --version
$r = Run @('--version')
if ($r.Combined -notmatch 'ast-grep-helper') { Fail '--version output missing' }
Pass '--version'
# 2. langs (must list >=25)
$r = Run @('langs')
$langCount = ($r.Stdout -split "`n" | Where-Object { $_ -match '^ [a-z]' }).Count
if ($langCount -lt 25) { Fail "langs listed only $langCount (expected >=25)" }
Pass 'langs lists at least 25 languages'
# 3. regex misuse: \w+
$r = Run @('validate', '\w+', '--lang', 'ts')
if ($r.ExitCode -ne 2) { Fail "validate '\w+' should exit 2, got $($r.ExitCode)" }
if ($r.Combined -notmatch 'regex') { Fail "validate '\w+' should mention regex" }
Pass 'validate detects \w regex misuse'
# 4. valid pattern
$r = Run @('validate', 'console.log($MSG)', '--lang', 'ts')
if ($r.ExitCode -ne 0) { Fail "validate 'console.log(`$MSG)' should exit 0, got $($r.ExitCode)" }
Pass 'validate accepts plausible pattern'
# 5. Python trailing colon
$r = Run @('validate', 'def $F($$$):', '--lang', 'py')
if ($r.ExitCode -ne 2) { Fail "validate 'def `$F(`$`$`$):' should exit 2, got $($r.ExitCode)" }
if ($r.Combined -notmatch 'colon|trailing') { Fail 'validate should mention trailing colon' }
Pass 'validate detects Python trailing colon'
# 6. Incomplete TS function
$r = Run @('validate', 'function $N', '--lang', 'ts')
if ($r.ExitCode -ne 2) { Fail "validate 'function `$N' should exit 2, got $($r.ExitCode)" }
if ($r.Combined -notmatch 'incomplete|params|body') { Fail 'validate should hint about params/body' }
Pass 'validate detects incomplete TS function'
# 7. Alternation pipe
$r = Run @('validate', 'foo|bar', '--lang', 'ts')
if ($r.ExitCode -ne 2) { Fail "validate 'foo|bar' should exit 2, got $($r.ExitCode)" }
if ($r.Combined -notmatch 'alternation|regex') { Fail 'validate should mention alternation' }
Pass 'validate detects literal | alternation'
# 8. doctor
$r = Run @('doctor')
if ($r.Combined -notmatch 'ast-grep-helper') { Fail 'doctor missing helper version line' }
Pass 'doctor produces output'
# 9. search w/o binary
$r = Run @('-q', 'search', 'foo()', '--lang', 'ts', 'C:/nonexistent-path-xyzzy')
switch ($r.ExitCode) {
{ $_ -in 0,1,4 } { Pass "search runs (rc=$($r.ExitCode), ast-grep available)" }
3 {
if ($r.Combined -notmatch 'install') { Fail 'search rc=3 should print install hint' }
Pass 'search without binary prints install hint'
}
default { Fail "search returned unexpected rc=$($r.ExitCode): $($r.Combined)" }
}
# 10. install.ps1 syntax (parse-check via PowerShell tokenizer)
$tokens = $null
$errors = $null
[System.Management.Automation.Language.Parser]::ParseFile(
(Join-Path $SkillDir 'install.ps1'), [ref]$tokens, [ref]$errors) | Out-Null
if ($errors -and $errors.Count -gt 0) { Fail "install.ps1 has parse errors: $($errors -join '; ')" }
Pass 'install.ps1 parses cleanly'
# 11. SKILL.md frontmatter
$skill = Get-Content (Join-Path $SkillDir 'SKILL.md') -Raw
if (-not $skill.StartsWith("---`n") -and -not $skill.StartsWith("---`r`n")) {
Fail 'SKILL.md must start with YAML frontmatter'
}
$endIdx = $skill.IndexOf("`n---`n", 4)
if ($endIdx -lt 0) { $endIdx = $skill.IndexOf("`r`n---`r`n", 4) }
if ($endIdx -lt 0) { Fail 'SKILL.md missing closing ---' }
$fm = $skill.Substring(4, $endIdx - 4)
if ($fm -notmatch '(?m)^name:\s*ast-grep\s*$') { Fail 'frontmatter missing name: ast-grep' }
if ($fm -notmatch '(?m)^description:') { Fail 'frontmatter missing description' }
Pass 'SKILL.md frontmatter shape'
# 12. All required reference files exist
$required = @(
'references/install.md', 'references/patterns.md', 'references/pitfalls.md',
'references/recipes.md', 'references/cli.md', 'references/yaml-rules.md',
'references/sgconfig.md'
)
foreach ($f in $required) {
if (-not (Test-Path (Join-Path $SkillDir $f))) { Fail "missing $f" }
}
Pass 'all references present'
Write-Host ''
Write-Host 'all smoke tests passed'
}
finally {
Remove-Item -Recurse -Force $Output -ErrorAction SilentlyContinue
}
optional-skills/software-development/ast-grep/tests/smoke.sh
Executable
+212
View File
@@ -0,0 +1,212 @@
#!/usr/bin/env bash
# Smoke test for the ast-grep skill on POSIX (macOS / Linux / WSL / Git Bash).
#
# Tests:
# 1. helper --version
# 2. helper langs (lists 25 languages)
# 3. helper validate '\w+' --lang ts (must exit 2 with hint)
# 4. helper validate 'console.log($MSG)' (must exit 0 plausible)
# 5. helper validate 'def $F($$$):' --lang py (must exit 2 - trailing colon)
# 6. helper validate 'function $N' --lang ts (must exit 2 - incomplete)
# 7. helper validate 'foo|bar' --lang ts (must exit 2 - alternation)
# 8. helper doctor (informational; tolerates no-binary)
# 9. helper search w/o binary => exit 3 with install hint
# 10. install.sh --help (parses)
# 11. SKILL.md frontmatter shape (canonical)
#
# Runs against the helper using ONLY stdlib python3 - no ast-grep needed.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
SKILL_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
HELPER="python3 $SKILL_DIR/scripts/ast_grep_helper.py"
OUTPUT_DIR="$(mktemp -d -t ast-grep-skill-smoke-XXXXXX)"
trap 'rm -rf "$OUTPUT_DIR"' EXIT
fail() { echo "FAIL: $*" >&2; exit 1; }
pass() { echo "PASS: $*"; }
omo_runtime_slug() {
case "$(uname -s)" in
Darwin) local os_slug="darwin" ;;
MINGW*|MSYS*|CYGWIN*) local os_slug="win32" ;;
*) local os_slug="linux" ;;
esac
case "$(uname -m | tr '[:upper:]' '[:lower:]')" in
arm64|aarch64) local arch_slug="arm64" ;;
*) local arch_slug="x64" ;;
esac
printf '%s-%s' "$os_slug" "$arch_slug"
}
fake_sg() {
local target="$1"
mkdir -p "$(dirname "$target")"
cat > "$target" <<'SH'
#!/usr/bin/env bash
if [ "${1:-}" = "--version" ]; then
printf 'ast-grep 0.45.0 fake\n'
else
printf 'fake ast-grep\n'
fi
SH
chmod +x "$target"
}
# 1. --version
$HELPER --version | grep -q "ast-grep-helper" || fail "--version output missing"
pass "--version"
# 2. langs (must list 25)
LANG_LINES=$($HELPER langs | grep -E '^ [a-z]' | wc -l | tr -d ' ')
[ "$LANG_LINES" -ge 25 ] || fail "langs listed only $LANG_LINES (expected >=25)"
pass "langs lists at least 25 languages"
# 3. regex misuse: \w+
set +e
$HELPER validate '\w+' --lang ts > "$OUTPUT_DIR/v1.out" 2>&1
RC=$?
set -e
[ $RC -eq 2 ] || fail "validate '\\w+' should exit 2, got $RC (output: $(cat $OUTPUT_DIR/v1.out))"
grep -qi 'regex' "$OUTPUT_DIR/v1.out" || fail "validate '\\w+' should mention regex (output: $(cat $OUTPUT_DIR/v1.out))"
pass "validate detects \\w regex misuse"
# 4. valid pattern
set +e
$HELPER validate 'console.log($MSG)' --lang ts > "$OUTPUT_DIR/v2.out" 2>&1
RC=$?
set -e
[ $RC -eq 0 ] || fail "validate 'console.log(\$MSG)' should exit 0, got $RC (output: $(cat $OUTPUT_DIR/v2.out))"
pass "validate accepts plausible pattern"
# 5. Python trailing colon
set +e
$HELPER validate 'def $F($$$):' --lang py > "$OUTPUT_DIR/v3.out" 2>&1
RC=$?
set -e
[ $RC -eq 2 ] || fail "validate 'def \$F(\$\$\$):' should exit 2, got $RC"
grep -qi 'colon\|trailing' "$OUTPUT_DIR/v3.out" || fail "validate should mention trailing colon"
pass "validate detects Python trailing colon"
# 6. Incomplete TS function
set +e
$HELPER validate 'function $N' --lang ts > "$OUTPUT_DIR/v4.out" 2>&1
RC=$?
set -e
[ $RC -eq 2 ] || fail "validate 'function \$N' should exit 2, got $RC"
grep -qi 'incomplete\|params\|body' "$OUTPUT_DIR/v4.out" || fail "validate should hint about params/body"
pass "validate detects incomplete TS function"
# 7. Alternation pipe
set +e
$HELPER validate 'foo|bar' --lang ts > "$OUTPUT_DIR/v5.out" 2>&1
RC=$?
set -e
[ $RC -eq 2 ] || fail "validate 'foo|bar' should exit 2, got $RC"
grep -qi 'alternation\|regex' "$OUTPUT_DIR/v5.out" || fail "validate should mention alternation"
pass "validate detects literal | alternation"
# 8. doctor (informational)
$HELPER doctor > "$OUTPUT_DIR/doc.out" 2>&1 || true
grep -q "ast-grep-helper" "$OUTPUT_DIR/doc.out" || fail "doctor missing helper version line"
pass "doctor produces output"
# Given OMO_AST_GREP_SG_PATH points at a fake executable.
# When doctor resolves ast-grep, then it reports that exact path.
FAKE_ENV_SG="$OUTPUT_DIR/fake-env/sg"
fake_sg "$FAKE_ENV_SG"
OMO_AST_GREP_SG_PATH="$FAKE_ENV_SG" $HELPER doctor > "$OUTPUT_DIR/omo-env.out" 2>&1
grep -Fq "ast-grep binary: $FAKE_ENV_SG" "$OUTPUT_DIR/omo-env.out" || fail "OMO_AST_GREP_SG_PATH was not preferred: $(cat "$OUTPUT_DIR/omo-env.out")"
pass "OMO_AST_GREP_SG_PATH resolves first"
# Given HOME has an OMO runtime sg executable.
# When doctor resolves ast-grep, then it reports the HOME runtime path.
RUNTIME_HOME="$OUTPUT_DIR/home"
RUNTIME_SLUG="$(omo_runtime_slug)"
RUNTIME_BIN="sg"
case "$RUNTIME_SLUG" in
win32-*) RUNTIME_BIN="sg.exe" ;;
esac
FAKE_RUNTIME_SG="$RUNTIME_HOME/.omo/runtime/ast-grep/$RUNTIME_SLUG/$RUNTIME_BIN"
fake_sg "$FAKE_RUNTIME_SG"
HOME="$RUNTIME_HOME" CODEX_HOME= OMO_AST_GREP_SG_PATH= $HELPER doctor > "$OUTPUT_DIR/omo-runtime.out" 2>&1
grep -Fq "ast-grep binary: $FAKE_RUNTIME_SG" "$OUTPUT_DIR/omo-runtime.out" || fail "OMO HOME runtime was not resolved: $(cat "$OUTPUT_DIR/omo-runtime.out")"
pass "OMO HOME runtime resolves before standalone fallback"
# 9. search w/o binary => either runs successfully (binary found) OR exits 3 with hint
# The CI may or may not have ast-grep installed; both are valid.
set +e
$HELPER -q search 'foo()' --lang ts /nonexistent-path-xyzzy > "$OUTPUT_DIR/s1.out" 2>&1
RC=$?
set -e
case "$RC" in
0|1|4)
# Binary present but no matches OR sg returned non-fatal error - both fine
pass "search runs (rc=$RC, ast-grep available)"
;;
3)
grep -qi 'install' "$OUTPUT_DIR/s1.out" || fail "search rc=3 should print install hint"
pass "search without binary prints install hint"
;;
*)
fail "search returned unexpected rc=$RC: $(cat $OUTPUT_DIR/s1.out)"
;;
esac
# 10. install.sh syntax check + --help
bash -n "$SKILL_DIR/install.sh" || fail "install.sh has syntax errors"
$SKILL_DIR/install.sh --help > "$OUTPUT_DIR/inst.out" 2>&1 || true
grep -qi 'install' "$OUTPUT_DIR/inst.out" || fail "install.sh --help missing keyword 'install'"
pass "install.sh syntax + --help"
# 11. SKILL.md frontmatter
python3 - <<PY
import re, sys
src = open("$SKILL_DIR/SKILL.md").read()
assert src.startswith("---\n"), "SKILL.md must start with YAML frontmatter"
end = src.find("\n---\n", 4)
assert end > 0, "SKILL.md missing closing ---"
fm = src[4:end]
assert re.search(r"^name:\s*ast-grep\s*$", fm, re.M), "frontmatter missing name: ast-grep"
assert re.search(r"^description:", fm, re.M), "frontmatter missing description"
PY
pass "SKILL.md frontmatter shape"
# 12. All required reference files exist
for f in references/install.md references/patterns.md references/pitfalls.md \
references/recipes.md references/cli.md references/yaml-rules.md \
references/sgconfig.md; do
test -f "$SKILL_DIR/$f" || fail "missing $f"
done
pass "all references present"
python3 - "$SKILL_DIR" <<'PY' || fail "Korean characters found in skill content"
import pathlib, re, sys
root = pathlib.Path(sys.argv[1])
hangul = re.compile(r"[\uac00-\ud7a3]")
targets = [root / "SKILL.md", root / "README.md"]
for d in ("references", "scripts", "tests", ".github"):
targets.extend((root / d).rglob("*"))
targets.extend([root / "install.sh", root / "install.ps1"])
hits = 0
for p in targets:
if not p.is_file():
continue
try:
text = p.read_text(encoding="utf-8")
except (UnicodeDecodeError, OSError):
continue
for n, line in enumerate(text.splitlines(), 1):
if hangul.search(line):
print(f"{p}:{n}: {line.rstrip()}")
hits += 1
sys.exit(1 if hits else 0)
PY
pass "no Korean in skill content"
echo ""
echo "all smoke tests passed"
optional-skills/software-development/code-wiki/SKILL.md
+445
View File
@@ -0,0 +1,445 @@
---
name: code-wiki
description: "Generate wiki docs + Mermaid diagrams for any codebase."
version: 0.1.0
author: Teknium (teknium1), Hermes Agent
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [Documentation, Mermaid, Architecture, Diagrams, Wiki, Code-Analysis]
related_skills: [codebase-inspection, github]
---
# Code Wiki Skill
Generate a full wiki for any codebase — overview, architecture, per-module deep-dives, Mermaid class and sequence diagrams. Inspired by Google CodeWiki, but works on local repos, private repos, and any language. Uses only existing Hermes tools (`terminal`, `read_file`, `search_files`, `write_file`); no Docker, no external services, no extra dependencies.
This skill produces **reference documentation** (what/how). It does not produce strategic narrative (why — that's a different skill).
## When to Use
- User says "document this codebase", "generate a wiki", "make architecture diagrams"
- Onboarding to an unfamiliar repo and wants a structured reference
- User points at a GitHub URL and asks for documentation
- Need a stable artifact (markdown + Mermaid) that renders on GitHub
Do NOT use this for:
- Single-file or single-function documentation — just answer directly
- API reference for one specific endpoint — use `read_file` and answer inline
- Strategic "why does this exist" narrative — different skill, different purpose
- Codebases the user is actively developing in this session — just answer questions as they come
## Prerequisites
- No env vars required.
- `git` on PATH for repo SHA tracking and remote clones.
- Optional: `pygount` for language-breakdown stats (see the `codebase-inspection` skill).
## How to Run
Invoke through the `terminal` tool from the target repo's root, then use `read_file` / `search_files` / `write_file` to produce the wiki. Default output location is `~/.hermes/wikis/<repo-name>/`. Only write into the repo (`docs/wiki/`) when the user explicitly requests it.
## Quick Reference
| Step | Action |
|---|---|
| 1 | Resolve target — local cwd, given path, or `git clone --depth 50 <url>` to a temp dir |
| 2 | Scan structure — `ls`, `find -maxdepth 3`, manifest files, README |
| 3 | Pick 8–10 modules to document |
| 4 | Write `README.md` (overview + module map) |
| 5 | Write `architecture.md` with Mermaid flowchart |
| 6 | Write per-module docs in `modules/` |
| 7 | Write `diagrams/class-diagram.md` (Mermaid classDiagram) |
| 8 | Write `diagrams/sequences.md` (Mermaid sequenceDiagram, 2–4 workflows) |
| 9 | Write `getting-started.md` |
| 10 | Write `api.md` if applicable, else skip |
| 11 | Write `.codewiki-state.json` |
| 12 | Report paths to user |
## Procedure
### 1. Resolve the target
For a GitHub URL:
```bash
WIKI_TMP=$(mktemp -d)
git clone --depth 50 <url> "$WIKI_TMP/repo"
cd "$WIKI_TMP/repo"
REPO_SHA=$(git rev-parse HEAD)
REPO_NAME=$(basename <url> .git)
```
For a local path (or cwd if none given):
```bash
cd <path>
REPO_SHA=$(git rev-parse HEAD 2>/dev/null || echo "uncommitted")
REPO_NAME=$(basename "$PWD")
```
Then set the output dir:
```bash
OUTPUT_DIR="$HOME/.hermes/wikis/$REPO_NAME"
mkdir -p "$OUTPUT_DIR/modules" "$OUTPUT_DIR/diagrams"
```
### 2. Scan repo structure
Use the `terminal` tool for the shell work, `read_file` for manifests:
```bash
# Shallow tree first
ls -la
# Deeper tree, noise filtered
find . -type d \
-not -path '*/\.*' \
-not -path '*/node_modules*' \
-not -path '*/venv*' \
-not -path '*/__pycache__*' \
-not -path '*/dist*' \
-not -path '*/build*' \
-not -path '*/target*' \
-maxdepth 3 | sort
# Language breakdown (skip if pygount unavailable)
pygount --format=summary \
--folders-to-skip=".git,node_modules,venv,.venv,__pycache__,.cache,dist,build,target" \
. 2>/dev/null || true
```
Then `read_file` the relevant manifests (`package.json`, `pyproject.toml`, `setup.py`, `Cargo.toml`, `go.mod`, `pom.xml`, `build.gradle`) and the project README. Use `search_files target='files'` to find them rather than guessing names.
### 3. Pick modules to document
Cap initial pass at **8–10 modules**. Heuristics by language:
- Python: top-level packages (dirs with `__init__.py`), plus subsystem dirs
- JS/TS: `src/<subdir>`, top-level workspace dirs
- Rust: each crate in a workspace, or top-level `src/<module>` dirs
- Go: each top-level package directory
- Mixed/unfamiliar: top-level directories that contain source code (not config, not tests)
For very large repos, prioritize by:
1. Imported-from count (a module imported by many is core)
2. LOC (bigger modules usually warrant their own doc)
3. Mentions in README / top-level docs
State the module list to the user before generating per-module docs on big repos — gives them a chance to redirect.
### 4. Write `README.md`
`read_file` the actual project README plus the top 2–3 entry-point files. Then `write_file`:
````markdown
# <Project Name>
<One paragraph: what it is and what it's for. Self-contained — don't assume the
reader has the source README.>
## Key Concepts
- **<Concept 1>** — <one line>
- **<Concept 2>** — <one line>
## Entry Points
- [`path/to/main.py`](<link>) — <what runs when you start it>
- [`path/to/cli.py`](<link>) — <CLI surface>
## High-Level Architecture
<2-3 sentences. Detail goes in architecture.md.>
See [architecture.md](architecture.md).
## Module Map
| Module | Purpose |
|---|---|
| [`<module>`](modules/<module>.md) | <one-line purpose> |
## Getting Started
See [getting-started.md](getting-started.md).
````
For link targets in local mode use relative paths. For cloned repos use `https://github.com/<owner>/<repo>/blob/<sha>/<path>` so links survive future commits.
### 5. Write `architecture.md`
````markdown
# Architecture
<2-3 paragraphs: shape of the system. What talks to what. Where data enters,
where it exits, where state lives.>
## Components
- **<Component>** — <1-2 sentences>. See [`modules/<module>.md`](modules/<module>.md).
## System Diagram
```mermaid
flowchart TD
User([User]) --> Entry[Entry Point]
Entry --> Core[Core Engine]
Core --> StorageA[(Database)]
Core --> ExternalAPI{{External API}}
```
## Data Flow
1. **<Step>** — [`<file>`](<link>)
2. **<Step>** — [`<file>`](<link>)
## Key Design Decisions
- <Anything load-bearing the reader should know>
````
**Mermaid shape semantics:**
- `[]` = component
- `[()]` = database / storage
- `{{}}` = external service
- `(())` = entry point or terminal
- `-->` = sync call, `-.->` = async/event
Cap at ~20 nodes per diagram. Split into sub-diagrams if larger.
### 6. Write per-module docs in `modules/`
For each selected module, inspect its layout with `ls`, identify 3–5 most important files (by size, by being named `core.py` / `main.py` / `__init__.py`, by being imported a lot), then `read_file` those files (use `offset` / `limit` to read only what you need; prefer `search_files` for specific symbols).
````markdown
# Module: `<module>`
<1-2 sentence purpose.>
## Responsibilities
- <bullet>
- <bullet>
## Key Files
- [`<module>/<file>`](<link>) — <what it does>
## Public API
<Functions/classes/constants other code uses. Group related items. Show
signatures, not full implementations.>
## Internal Structure
<How the module is organized internally. State management.>
## Dependencies
- **Used by:** <other modules>
- **Uses:** <other modules + external libs>
## Notable Patterns / Gotchas
- <Anything non-obvious>
````
### 7. Write `diagrams/class-diagram.md`
Pick the 5–10 most important classes/types. `read_file` them, then write:
````markdown
# Class Diagram
## Core Types
```mermaid
classDiagram
class Agent {
+string name
+list~Tool~ tools
+chat(message) string
}
class Tool {
<<interface>>
+name string
+execute(args) any
}
Agent --> Tool : uses
Tool <|-- TerminalTool
Tool <|-- WebTool
```
## Notes
<Anything the diagram can't express — lifecycle, threading, etc.>
````
For languages without classes (Go, C, Rust): use the diagram for struct relationships, or skip class-diagram.md and explain it in prose in architecture.md. Don't force-fit.
### 8. Write `diagrams/sequences.md`
Pick 2–4 of the most important workflows. Trace each call path through the code (read entry point, follow function calls), then:
````markdown
# Sequence Diagrams
## Workflow: <Name>
<1 sentence describing what this does and when it runs.>
```mermaid
sequenceDiagram
participant User
participant CLI
participant Agent
participant LLM
User->>CLI: types message
CLI->>Agent: chat(message)
Agent->>LLM: API call
LLM-->>Agent: response + tool_calls
Agent->>Agent: execute tools
Agent-->>CLI: final response
```
### Walkthrough
1. **User input** — [`cli.py:HermesCLI.run_session`](<link>)
2. **Message dispatch** — [`run_agent.py:AIAgent.chat`](<link>)
````
Don't invent participants. Every box must correspond to a real component the reader can find in the code.
### 9. Write `getting-started.md`
````markdown
# Getting Started
## Prerequisites
<From manifest files + README. Be specific — versions if pinned.>
## Installation
```bash
<exact commands>
```
## First Run
```bash
<minimum command to see the system do something useful>
```
## Common Workflows
### <Workflow 1>
<commands>
## Configuration
- `<config-file>` — <what it controls>
- Env var `<VAR>` — <what it controls>
## Where to Go Next
- Architecture: [architecture.md](architecture.md)
- Module reference: [README.md#module-map](README.md#module-map)
````
### 10. Write `api.md` (skip if not applicable)
Only write this if the project is a library or API server. If it is:
- Find the public API surface (`__init__.py` exports, OpenAPI specs, route handlers, exported types)
- Document each public entry with signature, parameters, return type, one-line description
- Group by category
### 11. Write the state file
```bash
cat > "$OUTPUT_DIR/.codewiki-state.json" <<EOF
{
"repo_name": "$REPO_NAME",
"source_path": "$PWD",
"source_sha": "$REPO_SHA",
"generated_at": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
"generator": "hermes-agent code-wiki skill v0.1.0",
"modules_documented": []
}
EOF
```
### 12. Report to user
State exactly what was generated and where:
```
Generated wiki at ~/.hermes/wikis/<repo-name>/:
README.md project overview, module map
architecture.md system architecture + flowchart
getting-started.md setup, first run, workflows
modules/<N files> per-module deep-dives
diagrams/architecture.md Mermaid flowchart
diagrams/class-diagram.md Mermaid class diagram
diagrams/sequences.md Mermaid sequence diagrams
```
If you cloned to a temp dir, remind the user it can be removed (`rm -rf "$WIKI_TMP"`) after they've reviewed the wiki.
## Scope Control
Generating a full wiki for a 500K-LOC monorepo is wildly token-expensive. Default to bounded scope:
- Initial scan: max depth 3 directories
- Per-module docs: cap at 10 modules unless user expands scope
- Per-file reads: prefer `search_files` for symbols + `read_file` with `offset`/`limit` over full reads
- Skip vendored code (`vendor/`, `third_party/`, generated code, `_pb2.py`, `.min.js`)
If the user says "do the whole thing exhaustively", believe them — but ballpark the cost first: "this repo has ~340 source files, comprehensive coverage will be expensive — confirm?"
## Re-Run / Update
If `.codewiki-state.json` already exists at the target path:
- Read it for previous SHA and module list
- If source SHA matches: ask user if they want to regenerate or skip
- If SHA differs: offer to regenerate only modules with changed files (`git diff --name-only <old-sha> HEAD`)
Full incremental-regeneration is a future enhancement — for now, regenerating the whole thing is acceptable.
## Pitfalls
- **Fabricating components.** Every diagram node and claimed function call must be in the source. `read_file` before writing. The single biggest failure mode for auto-generated docs is plausible-sounding fabrication.
- **Generic AI prose.** "This module is responsible for..." is content-free. Say what the module actually does in domain-specific terms.
- **Restating code as prose.** A module doc that says "the `process` function processes things by calling `process_item` on each item" is worse than just linking to the function.
- **Mermaid > 50 nodes.** They don't render legibly. Split them.
- **Documenting tests, generated code, or vendored deps as if they were product code.** Skip them.
- **In-repo output without asking.** Default is `~/.hermes/wikis/`. Only write into the repo when the user explicitly requests it.
- **Mermaid special chars need quotes:** `A["Tool / Agent"]` not `A[Tool / Agent]`. `<br>` for line breaks inside a node.
- **Nested code fences in SKILL.md.** When writing a markdown example that contains a Mermaid block, use 4-backtick outer fences so the 3-backtick inner ` ```mermaid ` doesn't close the outer. (This SKILL.md does it.)
- **classDiagram generics** render as `~T~` (e.g. `List~Tool~`), not `<T>`.
- **GitHub Mermaid theme is fixed** — don't include `%%{init: ...}%%` blocks; they're stripped on render.
## Verification
After writing, verify:
1. **Mermaid blocks balance** — opens equal closes per file:
```bash
for f in "$OUTPUT_DIR"/diagrams/*.md "$OUTPUT_DIR"/architecture.md; do
opens=$(grep -c '^```mermaid' "$f")
total=$(grep -c '^```' "$f")
echo "$f: $opens mermaid blocks, $total total fences (expect total = opens*2)"
done
```
2. **All expected files exist** —
```bash
ls "$OUTPUT_DIR"/{README.md,architecture.md,getting-started.md,.codewiki-state.json} \
"$OUTPUT_DIR"/modules/ "$OUTPUT_DIR"/diagrams/
```
3. **Module count matches what you intended** — `ls "$OUTPUT_DIR/modules" | wc -l` should equal the number of modules you committed to in Step 3.
4. **No fabricated paths** — sanity-check 2–3 source links resolve to real files.
optional-skills/software-development/code-wiki/templates/README.md
+31
View File
@@ -0,0 +1,31 @@
# {{PROJECT_NAME}}
{{ONE_PARAGRAPH_DESCRIPTION}}
## Key Concepts
- **{{CONCEPT_1}}** — {{ONE_LINE}}
- **{{CONCEPT_2}}** — {{ONE_LINE}}
- **{{CONCEPT_3}}** — {{ONE_LINE}}
## Entry Points
- [`{{PATH_1}}`]({{LINK_1}}) — {{WHAT_IT_DOES}}
- [`{{PATH_2}}`]({{LINK_2}}) — {{WHAT_IT_DOES}}
## High-Level Architecture
{{TWO_TO_THREE_SENTENCES}}
See [architecture.md](architecture.md) for the full picture.
## Module Map
| Module | Purpose |
|---|---|
| [`{{MODULE_1}}`](modules/{{MODULE_1}}.md) | {{ONE_LINE_PURPOSE}} |
| [`{{MODULE_2}}`](modules/{{MODULE_2}}.md) | {{ONE_LINE_PURPOSE}} |
## Getting Started
See [getting-started.md](getting-started.md).
optional-skills/software-development/code-wiki/templates/architecture.md
+30
View File
@@ -0,0 +1,30 @@
# Architecture
{{TWO_TO_THREE_PARAGRAPHS_SHAPE_OF_SYSTEM}}
## Components
- **{{COMPONENT_1}}** — {{ONE_TO_TWO_SENTENCES}} See [`modules/{{MODULE}}.md`](modules/{{MODULE}}.md).
- **{{COMPONENT_2}}** — {{ONE_TO_TWO_SENTENCES}}
## System Diagram
```mermaid
flowchart TD
User([User]) --> Entry[Entry Point]
Entry --> Core[Core Engine]
Core --> StorageA[(Database)]
Core --> ExternalAPI{{External API}}
```
## Data Flow
1. **{{STEP_1}}** — [`{{FILE}}`]({{LINK}})
2. **{{STEP_2}}** — [`{{FILE}}`]({{LINK}})
3. **{{STEP_3}}** — [`{{FILE}}`]({{LINK}})
## Key Design Decisions
- {{DECISION_1}}
- {{DECISION_2}}
- {{DECISION_3}}
optional-skills/software-development/code-wiki/templates/getting-started.md
+47
View File
@@ -0,0 +1,47 @@
# Getting Started
## Prerequisites
- {{LANGUAGE_RUNTIME_VERSION}}
- {{DEPENDENCY}}
## Installation
```bash
{{INSTALL_COMMANDS}}
```
## First Run
```bash
{{FIRST_RUN_COMMAND}}
```
You should see {{EXPECTED_OUTPUT}}.
## Common Workflows
### {{WORKFLOW_1}}
```bash
{{COMMANDS}}
```
### {{WORKFLOW_2}}
```bash
{{COMMANDS}}
```
## Configuration
Key config files and settings:
- `{{CONFIG_FILE}}` — {{WHAT_IT_CONTROLS}}
- Env var `{{VAR}}` — {{WHAT_IT_CONTROLS}}
## Where to Go Next
- Architecture overview: [architecture.md](architecture.md)
- Module reference: [README.md#module-map](README.md#module-map)
- Diagrams: [diagrams/](diagrams/)
optional-skills/software-development/code-wiki/templates/module.md
+38
View File
@@ -0,0 +1,38 @@
# Module: `{{MODULE_NAME}}`
{{ONE_TO_TWO_SENTENCE_PURPOSE}}
## Responsibilities
- {{BULLET_1}}
- {{BULLET_2}}
- {{BULLET_3}}
## Key Files
- [`{{PATH_1}}`]({{LINK_1}}) — {{WHAT_IT_DOES}}
- [`{{PATH_2}}`]({{LINK_2}}) — {{WHAT_IT_DOES}}
## Public API
### `{{FUNCTION_NAME}}({{SIGNATURE}})`
{{ONE_LINE_DESCRIPTION}}
**Parameters:**
- `{{PARAM}}` ({{TYPE}}) — {{DESCRIPTION}}
**Returns:** {{TYPE}} — {{DESCRIPTION}}
## Internal Structure
{{HOW_THE_MODULE_IS_ORGANIZED}}
## Dependencies
- **Used by:** {{OTHER_MODULES}}
- **Uses:** {{OTHER_MODULES_AND_LIBS}}
## Notable Patterns / Gotchas
- {{ANYTHING_NON_OBVIOUS}}
optional-skills/software-development/grill-me/SKILL.md
+117
View File
@@ -0,0 +1,117 @@
---
name: grill-me
description: "Adversarial plan interview before implementation."
version: 2.0.0
author: "Rafael Zendron (rafaumeu) + Matt Pocock (mattpocock/skills, grilling) + Hermes Agent"
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [planning, adversarial, interview, decision-tree, pre-implementation, review, alignment]
related_skills: [requesting-code-review, subagent-driven-development, test-driven-development]
---
# Grill Me
Stress-tests a plan through structured adversarial questioning before any
code is written. Models the plan as a **design tree** — every decision
branches into the decisions that hang off it — and interviews the user in
rounds until every branch is resolved and nothing is silently assumed.
Combines the phase discipline of the original with the frontier-rounds
mechanic from mattpocock/skills' `grilling`.
## When to Use
- User says "grill me", "interview my plan", "stress test this idea"
- Before complex work: auth flows, schema changes, migrations, payments
- A plan has unresolved decisions or seems vague
- Before `subagent-driven-development` decomposition
Do NOT use for existing code (use `requesting-code-review`) or simple one-off
tasks.
## Prerequisites
None. The skill works on any plan or raw idea.
## Core Mechanic: Frontier Rounds
Map the plan as a design tree. The **frontier** is every decision whose
prerequisites are already settled — the questions you can ask NOW without
guessing at answers you haven't heard yet.
Work in **rounds**: ask the whole current frontier in one message, numbered,
each question carrying your recommended answer. Then wait. A question whose
answer depends on another question still open in this round belongs to a
LATER round, not this one.
Format each round like so:
```
❓ Q1 — <question title>: <question body, options if relevant>
➡️ Recommendation: <your recommended answer + one-line why>
❓ Q2 — <question title>: <question body>
➡️ Recommendation: <...>
```
Each answer reshapes the tree: settled decisions push the frontier outward
and unblock dependent questions. Recompute the frontier and ask the next
round.
**Facts are your job; decisions are the user's.** When a frontier question
needs a fact from the environment (codebase, filesystem, config, docs), find
it yourself with `search_files` / `read_file` / `terminal` — or dispatch a
subagent via `delegate_task` for a heavy exploration. Never ask the user for
anything you could look up. Don't block on an exploration: only the questions
downstream of it wait; ask the rest of the frontier now.
## Question Coverage (work these branches into the tree)
**Understanding** — the real goal and boundaries:
- What is the ACTUAL objective? What is explicitly IN and OUT of scope?
- What are the constraints (time, tech, team, budget)? Who are the users?
**Technical decisions** — for each architectural choice:
- "Why this approach and not X?" / "What happens if Y fails?"
- "What's the worst case?" / "How would you roll back?"
- Cross-reference the existing codebase; if the project already has a
pattern for this, call it out.
**Edge cases:**
- "What happens if the user does Z?" / "What if dependency X goes down?"
- "What if volume is 100x expected?" / "What are the security implications?"
## Synthesis (when the frontier is empty)
1. Summarize ALL decisions in bullet points
2. List anything left open, and what is explicitly OUT of scope
3. Ask: "Aligned? Should I start implementing, or adjust anything?"
Do not act on the plan until the user confirms shared understanding.
## Pitfalls
1. **Asking questions out of dependency order.** A question that depends on
an unanswered question is a guess wearing a question mark. Keep it for a
later round.
2. **Skipping the codebase.** Find facts in code with Hermes tools instead of
asking the user.
3. **Accepting "I don't know" as final.** Suggest options, explain
trade-offs, make a recommendation.
4. **Writing code during the interrogation.** Alignment only — code after the
explicit green light.
5. **Being too agreeable.** Your job is to find problems. If everything looks
fine, look harder.
6. **Not adapting to the user's language.** Interview in whatever language
the user speaks.
## Verification
- [ ] Every question in a round had all its prerequisites already settled
- [ ] Provided a recommendation with each question
- [ ] Explored the codebase for facts instead of asking the user
- [ ] Frontier empty (no branch silently assumed) before synthesizing
- [ ] Produced a clear summary of all decisions and open items
- [ ] Confirmed user alignment before stopping
optional-skills/software-development/rest-graphql-debug/SKILL.md
+515
View File
@@ -0,0 +1,515 @@
---
name: rest-graphql-debug
description: "Debug REST/GraphQL APIs: status codes, auth, schemas, repro."
version: 1.2.0
author: eren-karakus0
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [api, rest, graphql, http, debugging, testing, curl, integration]
category: software-development
related_skills: [systematic-debugging, test-driven-development]
---
# API Testing & Debugging
Drive REST and GraphQL diagnosis through Hermes tools — `terminal` for `curl`, `execute_code` for Python `requests`, `web_extract` for vendor docs. Isolate the failing layer before guessing at the fix.
## When to Use
- API returns unexpected status or body
- Auth fails (401/403 after token refresh, OAuth, API key)
- Works in Postman but fails in code
- Webhook / callback integration debugging
- Building or reviewing API integration tests
- Rate limiting or pagination issues
Skip for UI rendering, DB query tuning, or DNS/firewall infra (escalate).
## Core Principle
**Isolate the layer, then fix.** A 200 OK can hide broken data. A 500 can mask a one-character auth typo. Walk the chain in order; never skip a step.
```
1. Connectivity → can we reach the host at all?
1.5 Timeouts → connect-slow vs read-slow?
2. TLS/SSL → cert valid and trusted?
3. Auth → credentials correct and unexpired?
4. Request format → payload shape match server expectations?
5. Response parse → does our code accept what came back?
6. Semantics → does the data mean what we assume?
```
## 5-Minute Quickstart
### REST via terminal
```python
# Verbose request/response exchange
terminal('curl -v https://api.example.com/users/1')
# POST with JSON
terminal("""curl -X POST https://api.example.com/users \\
-H 'Content-Type: application/json' \\
-H "Authorization: Bearer $TOKEN" \\
-d '{"name":"test","email":"test@example.com"}'""")
# Headers only
terminal('curl -sI https://api.example.com/health')
# Pretty-print JSON
terminal('curl -s https://api.example.com/users | python -m json.tool')
```
### GraphQL via terminal
```python
terminal("""curl -X POST https://api.example.com/graphql \\
-H 'Content-Type: application/json' \\
-H "Authorization: Bearer $TOKEN" \\
-d '{"query":"{ user(id: 1) { name email } }"}'""")
```
**GraphQL gotcha:** servers often return HTTP 200 even when the query failed. Always inspect the `errors` field regardless of status code:
```python
execute_code('''
import os, requests
resp = requests.post(
"https://api.example.com/graphql",
json={"query": "{ user(id: 1) { name email } }"},
headers={"Authorization": f"Bearer {os.environ['TOKEN']}"},
timeout=10,
)
data = resp.json()
if data.get("errors"):
for err in data["errors"]:
print(f"GraphQL error: {err['message']} (path: {err.get('path')})")
print(data.get("data"))
''')
```
### Python (requests) via execute_code
```python
execute_code('''
import requests
resp = requests.get(
"https://api.example.com/users/1",
headers={"Authorization": "Bearer <TOKEN>"},
timeout=(3.05, 30), # (connect, read)
)
print(resp.status_code, dict(resp.headers))
print(resp.text[:500])
''')
```
## Layered Debug Flow
### Step 1 — Connectivity
```python
terminal('nslookup api.example.com')
terminal('curl -v --connect-timeout 5 https://api.example.com/health')
```
Failures: DNS not resolving, firewall, VPN required, proxy missing.
### Step 1.5 — Timeouts
Distinguish *can't reach* from *reaches but slow*:
```python
terminal('''curl -w "dns:%{time_namelookup}s connect:%{time_connect}s tls:%{time_appconnect}s ttfb:%{time_starttransfer}s total:%{time_total}s\\n" \\
-o /dev/null -s https://api.example.com/endpoint''')
```
In Python, always pass a tuple timeout — `requests` has no default and will hang forever:
```python
execute_code('''
import requests
from requests.exceptions import ConnectTimeout, ReadTimeout
try:
requests.get(url, timeout=(3.05, 30))
except ConnectTimeout:
print("Cannot reach host — DNS, firewall, VPN")
except ReadTimeout:
print("Connected but server is slow")
''')
```
Diagnosis: high `time_connect` is network/firewall; high `time_starttransfer` with low `time_connect` is a slow server.
### Step 2 — TLS/SSL
```python
terminal('curl -vI https://api.example.com 2>&1 | grep -E "SSL|subject|expire|issuer"')
```
Failures: expired cert, self-signed, hostname mismatch, missing CA bundle. Use `-k` only for ad-hoc debug, never in code.
### Step 3 — Authentication
```python
# Token validity check
terminal('curl -s -o /dev/null -w "%{http_code}\\n" -H "Authorization: Bearer $TOKEN" https://api.example.com/me')
# Decode JWT exp claim — handles base64url padding correctly
execute_code('''
import json, base64, os
tok = os.environ["TOKEN"]
payload = tok.split(".")[1]
payload += "=" * (-len(payload) % 4)
print(json.dumps(json.loads(base64.urlsafe_b64decode(payload)), indent=2))
''')
```
Checklist:
- Token expired? (`exp` claim in JWT)
- Right scheme? Bearer vs Basic vs Token vs `X-Api-Key`
- Right environment? Staging key on prod is a classic
- API key in header vs query param (`?api_key=…`)?
### Step 4 — Request Format
```python
terminal("""curl -v -X POST https://api.example.com/endpoint \\
-H 'Content-Type: application/json' \\
-d '{"key":"value"}' 2>&1""")
```
**Content-Type / body mismatch — the silent 415/400:**
```python
# WRONG — data= sends form-encoded, header lies
requests.post(url, data='{"k":"v"}', headers={"Content-Type": "application/json"})
# RIGHT — json= auto-sets header AND serializes
requests.post(url, json={"k": "v"})
# WRONG — Accept says XML, code calls .json()
requests.get(url, headers={"Accept": "text/xml"})
# RIGHT — let requests build multipart with boundary
requests.post(url, files={"file": open("doc.pdf", "rb")})
```
Common: form-encoded vs JSON, missing required fields, wrong HTTP method, unencoded query params.
### Step 5 — Response Parsing
Always inspect content-type before calling `.json()`:
```python
execute_code('''
import requests
resp = requests.post(url, json=payload, timeout=10)
print(f"status={resp.status_code}")
print(f"headers={dict(resp.headers)}")
ct = resp.headers.get("Content-Type", "")
if "application/json" in ct:
print(resp.json())
else:
print(f"unexpected content-type {ct!r}, body={resp.text[:500]!r}")
''')
```
Failures: HTML error page where JSON expected, empty body, wrong charset.
### Step 6 — Semantic Validation
Parsed cleanly — but is the data *correct*?
- Does `"status": "active"` mean what your code thinks?
- ID in response matches the one requested?
- Timestamps in expected timezone?
- Pagination returning all results, or just page 1?
## HTTP Status Playbook
### 401 Unauthorized — credentials missing or invalid
1. `Authorization` header actually present? (`curl -v` to confirm)
2. Token correct and unexpired?
3. Right auth scheme? (`Bearer` vs `Basic` vs `Token`)
4. Some APIs use query param (`?api_key=…`) instead of header.
### 403 Forbidden — authenticated but not authorized
1. Token has the required scopes/permissions?
2. Resource owned by a different account?
3. IP allowlist blocking you?
4. CORS in browser? (check `Access-Control-Allow-Origin`)
### 404 Not Found — resource doesn't exist or URL is wrong
1. Path correct? (trailing slash, typo, version prefix)
2. Resource ID exists?
3. Right API version (`/v1/` vs `/v2/`)?
4. Right base URL (staging vs prod)?
### 409 Conflict — state collision
1. Resource already exists (duplicate create)?
2. Stale `ETag` / `If-Match`?
3. Concurrent modification by another process?
### 422 Unprocessable Entity — valid JSON, invalid data
The error body usually names the bad fields. Check:
- Field types (string vs int, date format)
- Required vs optional
- Enum values inside the allowed set
### 429 Too Many Requests — rate limited
Check `Retry-After` and `X-RateLimit-*` headers. Exponential backoff:
```python
execute_code('''
import time, requests
def with_backoff(method, url, **kwargs):
for attempt in range(5):
resp = requests.request(method, url, **kwargs)
if resp.status_code != 429:
return resp
wait = int(resp.headers.get("Retry-After", 2 ** attempt))
time.sleep(wait)
return resp
''')
```
### 5xx — server-side, usually not your fault
- **500** — server bug. Capture correlation ID, file with provider.
- **502** — upstream down. Backoff + retry.
- **503** — overloaded / maintenance. Check status page.
- **504** — upstream timeout. Reduce payload or raise timeout.
For all 5xx: backoff with jitter, alert on persistence.
## Pagination & Idempotency
**Pagination.** Verify you're getting *all* results. Look for `next_cursor`, `next_page`, `total_count`. Two patterns:
- Offset (`?limit=100&offset=200`) — simple, can skip items if data shifts.
- Cursor (`?cursor=abc123`) — preferred for live or large datasets.
**Idempotency.** For non-idempotent operations (POST), send `Idempotency-Key: <uuid>` so retries don't double-charge / double-create. Mandatory for payments and orders.
## Contract Validation
Catch schema drift before it hits production:
```python
execute_code('''
import requests
def validate_user(data: dict) -> list[str]:
errors = []
required = {"id": int, "email": str, "created_at": str}
for field, expected in required.items():
if field not in data:
errors.append(f"missing field: {field}")
elif not isinstance(data[field], expected):
errors.append(f"{field}: want {expected.__name__}, got {type(data[field]).__name__}")
return errors
resp = requests.get(f"{BASE}/users/1", headers=HEADERS, timeout=10)
issues = validate_user(resp.json())
if issues:
print(f"contract violations: {issues}")
''')
```
Run after API upgrades, when integrating new third parties, or in CI smoke tests.
## Correlation IDs
Always capture the provider's request ID — fastest path to vendor support:
```python
execute_code('''
import requests
resp = requests.post(url, json=payload, headers=headers, timeout=10)
request_id = (
resp.headers.get("X-Request-Id")
or resp.headers.get("X-Trace-Id")
or resp.headers.get("CF-Ray") # Cloudflare
)
if resp.status_code >= 400:
print(f"failed status={resp.status_code} req_id={request_id} ts={resp.headers.get('Date')}")
''')
```
**Vendor bug-report template:**
```
Endpoint: POST /api/v1/orders
Request ID: req_abc123xyz
Timestamp: 2026-03-17T14:30:00Z
Status: 500
Expected: 201 with order object
Actual: 500 {"error":"internal server error"}
Repro: curl -X POST … (auth: <REDACTED>)
```
## Regression Test Template
Drop this into `tests/` and run via `terminal('pytest tests/test_api_smoke.py -v')`:
```python
import os, requests, pytest
BASE_URL = os.environ.get("API_BASE_URL", "https://api.example.com")
TOKEN = os.environ.get("API_TOKEN", "")
HEADERS = {"Authorization": f"Bearer {TOKEN}"}
class TestAPISmoke:
def test_health(self):
resp = requests.get(f"{BASE_URL}/health", timeout=5)
assert resp.status_code == 200
def test_list_users_returns_array(self):
resp = requests.get(f"{BASE_URL}/users", headers=HEADERS, timeout=10)
assert resp.status_code == 200
data = resp.json()
assert isinstance(data.get("data", data), list)
def test_get_user_required_fields(self):
resp = requests.get(f"{BASE_URL}/users/1", headers=HEADERS, timeout=10)
assert resp.status_code in (200, 404)
if resp.status_code == 200:
user = resp.json()
assert "id" in user and "email" in user
def test_invalid_auth_returns_401(self):
resp = requests.get(
f"{BASE_URL}/users",
headers={"Authorization": "Bearer invalid-token"},
timeout=10,
)
assert resp.status_code == 401
```
## Security
### Token handling
- Never log full tokens. Redact: `Bearer <REDACTED>`.
- Never hardcode tokens in scripts. Read from env (`os.environ["API_TOKEN"]`) or `${HERMES_HOME:-~/.hermes}/.env`.
- Rotate immediately if a token surfaces in logs, error messages, or git history.
### Safe logging
```python
def redact_auth(headers: dict) -> dict:
sensitive = {"authorization", "x-api-key", "cookie", "set-cookie"}
return {k: ("<REDACTED>" if k.lower() in sensitive else v) for k, v in headers.items()}
```
### Leak checklist
- [ ] **Credentials in URLs.** API keys in query strings end up in server logs, browser history, referrer headers — use headers.
- [ ] **PII in error responses.** `404 on /users/123` shouldn't reveal whether the user exists (enumeration).
- [ ] **Stack traces in prod.** 500s shouldn't leak file paths, framework versions.
- [ ] **Internal hostnames/IPs.** `10.x.x.x`, `internal-api.corp.local` in error bodies.
- [ ] **Tokens echoed back.** Some APIs include the auth token in error details. Verify they don't.
- [ ] **Verbose `Server` / `X-Powered-By`.** Stack-info leaks. Note for security review.
## Hermes Tool Patterns
### terminal — for curl, dig, openssl
```python
terminal('curl -sI https://api.example.com')
terminal('openssl s_client -connect api.example.com:443 -servername api.example.com </dev/null 2>/dev/null | openssl x509 -noout -dates')
```
### execute_code — for multi-step Python flows
When debugging spans auth → fetch → paginate → validate, use `execute_code`. Variables persist for the script, results print to stdout, no risk of token spam in your context:
```python
execute_code('''
import os, requests
token = os.environ["API_TOKEN"]
base = "https://api.example.com"
H = {"Authorization": f"Bearer {token}"}
# 1. auth
me = requests.get(f"{base}/me", headers=H, timeout=10)
print(f"auth {me.status_code}")
# 2. paginate
all_users, cursor = [], None
while True:
params = {"cursor": cursor} if cursor else {}
r = requests.get(f"{base}/users", headers=H, params=params, timeout=10)
body = r.json()
all_users.extend(body["data"])
cursor = body.get("next_cursor")
if not cursor:
break
print(f"users={len(all_users)}")
''')
```
### web_extract — for vendor API docs
Pull the spec for the endpoint you're debugging instead of guessing:
```python
web_extract(urls=["https://docs.example.com/api/v1/users"])
```
### delegate_task — for full CRUD test sweeps
```python
delegate_task(
goal="Test all CRUD endpoints for /api/v1/users",
context="""
Follow the rest-graphql-debug skill (optional-skills/software-development/rest-graphql-debug).
Base URL: https://api.example.com
Auth: Bearer token from API_TOKEN env var.
For each verb (POST, GET, PATCH, DELETE):
- happy path: assert status + response schema
- error cases: 400, 404, 422
- log a repro curl for any failure (redact tokens)
Output: pass/fail per endpoint + correlation IDs for failures.
""",
toolsets=["terminal", "file"],
)
```
## Output Format
When reporting findings:
```
## Finding
Endpoint: POST /api/v1/users
Status: 422 Unprocessable Entity
Req ID: req_abc123xyz
## Repro
curl -X POST https://api.example.com/api/v1/users \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <REDACTED>' \
-d '{"name":"test"}'
## Root Cause
Missing required field `email`. Server validation rejects before processing.
## Fix
-d '{"name":"test","email":"test@example.com"}'
```
## Related
- `systematic-debugging` — once the failing API layer is isolated, root-cause your code
- `test-driven-development` — write the regression test before shipping the fix
optional-skills/software-development/subagent-driven-development/SKILL.md
+352
View File
@@ -0,0 +1,352 @@
---
name: subagent-driven-development
description: "Execute plans via delegate_task subagents (2-stage review)."
version: 1.1.0
author: Hermes Agent (adapted from obra/superpowers)
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [delegation, subagent, implementation, workflow, parallel]
related_skills: [requesting-code-review, test-driven-development]
---
# Subagent-Driven Development
## Overview
Execute implementation plans by dispatching fresh subagents per task with systematic two-stage review.
**Core principle:** Fresh subagent per task + two-stage review (spec then quality) = high quality, fast iteration.
## When to Use
Use this skill when:
- You have an implementation plan (from the `plan` skill or user requirements)
- Tasks are mostly independent
- Quality and spec compliance are important
- You want automated review between tasks
**vs. manual execution:**
- Fresh context per task (no confusion from accumulated state)
- Automated review process catches issues early
- Consistent quality checks across all tasks
- Subagents can ask questions before starting work
## The Process
### 1. Read and Parse Plan
Read the plan file. Extract ALL tasks with their full text and context upfront. Create a todo list:
```python
# Read the plan
read_file("docs/plans/feature-plan.md")
# Create todo list with all tasks
todo([
{"id": "task-1", "content": "Create User model with email field", "status": "pending"},
{"id": "task-2", "content": "Add password hashing utility", "status": "pending"},
{"id": "task-3", "content": "Create login endpoint", "status": "pending"},
])
```
**Key:** Read the plan ONCE. Extract everything. Don't make subagents read the plan file — provide the full task text directly in context.
### 2. Per-Task Workflow
For EACH task in the plan:
#### Step 1: Dispatch Implementer Subagent
Use `delegate_task` with complete context:
```python
delegate_task(
goal="Implement Task 1: Create User model with email and password_hash fields",
context="""
TASK FROM PLAN:
- Create: src/models/user.py
- Add User class with email (str) and password_hash (str) fields
- Use bcrypt for password hashing
- Include __repr__ for debugging
FOLLOW TDD:
1. Write failing test in tests/models/test_user.py
2. Run: pytest tests/models/test_user.py -v (verify FAIL)
3. Write minimal implementation
4. Run: pytest tests/models/test_user.py -v (verify PASS)
5. Run: pytest tests/ -q (verify no regressions)
6. Commit: git add -A && git commit -m "feat: add User model with password hashing"
PROJECT CONTEXT:
- Python 3.11, Flask app in src/app.py
- Existing models in src/models/
- Tests use pytest, run from project root
- bcrypt already in requirements.txt
""",
toolsets=['terminal', 'file']
)
```
#### Step 2: Dispatch Spec Compliance Reviewer
After the implementer completes, verify against the original spec:
```python
delegate_task(
goal="Review if implementation matches the spec from the plan",
context="""
ORIGINAL TASK SPEC:
- Create src/models/user.py with User class
- Fields: email (str), password_hash (str)
- Use bcrypt for password hashing
- Include __repr__
CHECK:
- [ ] All requirements from spec implemented?
- [ ] File paths match spec?
- [ ] Function signatures match spec?
- [ ] Behavior matches expected?
- [ ] Nothing extra added (no scope creep)?
OUTPUT: PASS or list of specific spec gaps to fix.
""",
toolsets=['file']
)
```
**If spec issues found:** Fix gaps, then re-run spec review. Continue only when spec-compliant.
#### Step 3: Dispatch Code Quality Reviewer
After spec compliance passes:
```python
delegate_task(
goal="Review code quality for Task 1 implementation",
context="""
FILES TO REVIEW:
- src/models/user.py
- tests/models/test_user.py
CHECK:
- [ ] Follows project conventions and style?
- [ ] Proper error handling?
- [ ] Clear variable/function names?
- [ ] Adequate test coverage?
- [ ] No obvious bugs or missed edge cases?
- [ ] No security issues?
OUTPUT FORMAT:
- Critical Issues: [must fix before proceeding]
- Important Issues: [should fix]
- Minor Issues: [optional]
- Verdict: APPROVED or REQUEST_CHANGES
""",
toolsets=['file']
)
```
**If quality issues found:** Fix issues, re-review. Continue only when approved.
#### Step 4: Mark Complete
```python
todo([{"id": "task-1", "content": "Create User model with email field", "status": "completed"}], merge=True)
```
### 3. Final Review
After ALL tasks are complete, dispatch a final integration reviewer:
```python
delegate_task(
goal="Review the entire implementation for consistency and integration issues",
context="""
All tasks from the plan are complete. Review the full implementation:
- Do all components work together?
- Any inconsistencies between tasks?
- All tests passing?
- Ready for merge?
""",
toolsets=['terminal', 'file']
)
```
### 4. Verify and Commit
```bash
# Run full test suite
pytest tests/ -q
# Review all changes
git diff --stat
# Final commit if needed
git add -A && git commit -m "feat: complete [feature name] implementation"
```
## Task Granularity
**Each task = 2-5 minutes of focused work.**
**Too big:**
- "Implement user authentication system"
**Right size:**
- "Create User model with email and password fields"
- "Add password hashing function"
- "Create login endpoint"
- "Add JWT token generation"
- "Create registration endpoint"
## Red Flags — Never Do These
- Start implementation without a plan
- Skip reviews (spec compliance OR code quality)
- Proceed with unfixed critical/important issues
- Dispatch multiple implementation subagents for tasks that touch the same files
- Make subagent read the plan file (provide full text in context instead)
- Skip scene-setting context (subagent needs to understand where the task fits)
- Ignore subagent questions (answer before letting them proceed)
- Accept "close enough" on spec compliance
- Skip review loops (reviewer found issues → implementer fixes → review again)
- Let implementer self-review replace actual review (both are needed)
- **Start code quality review before spec compliance is PASS** (wrong order)
- Move to next task while either review has open issues
## Handling Issues
### If Subagent Asks Questions
- Answer clearly and completely
- Provide additional context if needed
- Don't rush them into implementation
### If Reviewer Finds Issues
- Implementer subagent (or a new one) fixes them
- Reviewer reviews again
- Repeat until approved
- Don't skip the re-review
### If Subagent Fails a Task
- Dispatch a new fix subagent with specific instructions about what went wrong
- Don't try to fix manually in the controller session (context pollution)
## Efficiency Notes
**Why fresh subagent per task:**
- Prevents context pollution from accumulated state
- Each subagent gets clean, focused context
- No confusion from prior tasks' code or reasoning
**Why two-stage review:**
- Spec review catches under/over-building early
- Quality review ensures the implementation is well-built
- Catches issues before they compound across tasks
**Cost trade-off:**
- More subagent invocations (implementer + 2 reviewers per task)
- But catches issues early (cheaper than debugging compounded problems later)
## Integration with Other Skills
### With plan
This skill EXECUTES plans created by the `plan` skill:
1. User requirements → plan → implementation plan
2. Implementation plan → subagent-driven-development → working code
### With test-driven-development
Implementer subagents should follow TDD:
1. Write failing test first
2. Implement minimal code
3. Verify test passes
4. Commit
Include TDD instructions in every implementer context.
### With requesting-code-review
The two-stage review process IS the code review. For final integration review, use the requesting-code-review skill's review dimensions.
### With systematic-debugging
If a subagent encounters bugs during implementation:
1. Follow systematic-debugging process
2. Find root cause before fixing
3. Write regression test
4. Resume implementation
## Example Workflow
```
[Read plan: docs/plans/auth-feature.md]
[Create todo list with 5 tasks]
--- Task 1: Create User model ---
[Dispatch implementer subagent]
Implementer: "Should email be unique?"
You: "Yes, email must be unique"
Implementer: Implemented, 3/3 tests passing, committed.
[Dispatch spec reviewer]
Spec reviewer: ✅ PASS — all requirements met
[Dispatch quality reviewer]
Quality reviewer: ✅ APPROVED — clean code, good tests
[Mark Task 1 complete]
--- Task 2: Password hashing ---
[Dispatch implementer subagent]
Implementer: No questions, implemented, 5/5 tests passing.
[Dispatch spec reviewer]
Spec reviewer: ❌ Missing: password strength validation (spec says "min 8 chars")
[Implementer fixes]
Implementer: Added validation, 7/7 tests passing.
[Dispatch spec reviewer again]
Spec reviewer: ✅ PASS
[Dispatch quality reviewer]
Quality reviewer: Important: Magic number 8, extract to constant
Implementer: Extracted MIN_PASSWORD_LENGTH constant
Quality reviewer: ✅ APPROVED
[Mark Task 2 complete]
... (continue for all tasks)
[After all tasks: dispatch final integration reviewer]
[Run full test suite: all passing]
[Done!]
```
## Remember
```
Fresh subagent per task
Two-stage review every time
Spec compliance FIRST
Code quality SECOND
Never skip reviews
Catch issues early
```
**Quality is not an accident. It's the result of systematic process.**
## Further reading (load when relevant)
When the orchestration involves significant context usage, long review loops, or complex validation checkpoints, load these references for the specific discipline:
- **`references/context-budget-discipline.md`** — Four-tier context degradation model (PEAK / GOOD / DEGRADING / POOR), read-depth rules that scale with context window size, and early warning signs of silent degradation. Load when a run will clearly consume significant context (multi-phase plans, many subagents, large artifacts).
- **`references/gates-taxonomy.md`** — The four canonical gate types (Pre-flight, Revision, Escalation, Abort) with behavior, recovery, and examples. Load when designing or reviewing any workflow that has validation checkpoints — use the vocabulary explicitly so each gate has defined entry, failure behavior, and resumption rules.
Both references adapted from gsd-build/get-shit-done (MIT © 2025 Lex Christopherson).
optional-skills/software-development/subagent-driven-development/references/context-budget-discipline.md
+53
View File
@@ -0,0 +1,53 @@
# Context Budget Discipline
Practical rules for keeping orchestrator context lean when spawning subagents or reading large artifacts. Use these whenever you're running a multi-step agent loop that will consume significant context — plan execution, subagent orchestration, review pipelines, multi-file refactors.
Adapted from the GSD (Get Shit Done) project's context-budget reference — MIT © 2025 Lex Christopherson ([gsd-build/get-shit-done](https://github.com/gsd-build/get-shit-done)).
## Universal rules
Every workflow that spawns agents or reads significant content must follow these:
1. **Never read agent definition files.** `delegate_task` auto-loads them — you reading them too just doubles the cost.
2. **Never inline large files into subagent prompts.** Tell the agent to read the file from disk with `read_file` instead. The subagent gets full content; your context stays lean.
3. **Read depth scales with context window.** See the table below.
4. **Delegate heavy work to subagents.** The orchestrator routes; it doesn't execute.
5. **Proactively warn** the user when you've consumed significant context ("Context is getting heavy — consider checkpointing progress before we continue").
## Read depth by context window
Check the model's actual context window (not "it's Claude so 200K"). Some Sonnet deployments are 1M, some are 200K. If you don't know, assume the smaller one — err toward leanness.
| Context window | Subagent output reading | Summary files | Verification files | Plans for other phases |
|----------------|-------------------------|---------------|--------------------|-----------------------|
| < 500k (e.g. 200k) | Frontmatter only | Frontmatter only | Frontmatter only | Current phase only |
| >= 500k (1M models) | Full body permitted | Full body permitted | Full body permitted | Current phase only |
"Frontmatter only" means: read enough to see the final status/verdict/conclusion. If the subagent wrote a 3000-line debug log, read the summary section it produced, not the log.
## Four-tier degradation model
Monitor your context usage and shift behavior as you climb the tiers. The point is to notice *before* you hit the wall, not when responses start truncating.
| Tier | Usage | Behavior |
|------|-------|----------|
| **PEAK** | 0 – 30% | Full operations. Read bodies, spawn multiple agents in parallel, inline results freely. |
| **GOOD** | 30 – 50% | Normal operations. Prefer frontmatter reads. Delegate aggressively. |
| **DEGRADING** | 50 – 70% | Economize. Frontmatter-only reads, minimal inlining, **warn the user** about budget. |
| **POOR** | 70%+ | Emergency mode. **Checkpoint progress immediately.** No new reads unless critical. Finish the current task and stop cleanly. |
## Early warning signs (before panic thresholds fire)
Quality degrades *gradually* before hard limits hit. Watch for these:
- **Silent partial completion.** Subagent claims done but implementation is incomplete. Self-checks catch file existence, not semantic completeness. Always verify subagent output against the plan's must-haves, not just "did a file appear?"
- **Increasing vagueness.** Agent starts using phrases like "appropriate handling" or "standard patterns" instead of specific code. This is context pressure showing up before budget warnings fire.
- **Skipped protocol steps.** Agent omits steps it would normally follow. If success criteria has 8 items and the report covers 5, suspect context pressure, not "the agent decided 5 was enough."
When these signs appear, checkpoint the work and either reset context or hand off to a fresh subagent.
## Fundamental limitation
When you orchestrate, you cannot verify semantic correctness of subagent output — only structural completeness ("did the file appear?", "does the test pass?"). Semantic verification requires either running the code yourself or delegating a review pass to another fresh subagent.
**Mitigation:** in every task you delegate, include explicit "must-have" truths the subagent must confirm in its response (e.g., "confirm your test actually tests X, not just that X was imported"). The subagent re-asserting concrete facts is evidence; vague summaries are not.
optional-skills/software-development/subagent-driven-development/references/gates-taxonomy.md
+93
View File
@@ -0,0 +1,93 @@
# Gates Taxonomy
Canonical gate types for validation checkpoints across any workflow that spawns subagents, runs review loops, or has human-approval pauses. Every validation checkpoint maps to one of these four types — naming them explicitly makes the workflow legible and prevents "what happens when this check fails?" confusion.
Adapted from the GSD (Get Shit Done) project's gates reference — MIT © 2025 Lex Christopherson ([gsd-build/get-shit-done](https://github.com/gsd-build/get-shit-done)).
## The four gate types
### 1. Pre-flight gate
**Purpose:** Validates preconditions before starting an operation.
**Behavior:** Blocks entry if conditions unmet. No partial work created — bail before anything changes.
**Recovery:** Fix the missing precondition, then retry.
**Examples:**
- Implementation phase checks that the plan file exists before it starts writing code.
- Delegated subagent checks that required env vars are set before making API calls.
- Commit checks that tests passed before pushing.
### 2. Revision gate
**Purpose:** Evaluates output quality and routes to revision if insufficient.
**Behavior:** Loops back to the producer with specific feedback. Bounded by an iteration cap (typically 3).
**Recovery:** Producer addresses feedback; checker re-evaluates. The loop escalates early if issue count does not decrease between consecutive iterations (stall detection). After max iterations, escalates to the user unconditionally — never loop forever.
**Examples:**
- Plan reviewer reads a draft plan, returns specific issues, planner revises, reviewer re-reads (max 3 cycles).
- Code reviewer checks subagent-produced code against must-haves; dispatches fixes back to the implementer if any must-have failed.
- Test coverage checker validates new tests exercise the new paths; if not, sends back to author.
### 3. Escalation gate
**Purpose:** Surfaces unresolvable issues to the human for a decision.
**Behavior:** Pauses workflow, presents options, waits for human input. Never guesses, never picks a default.
**Recovery:** Human chooses action; workflow resumes on the selected path.
**Examples:**
- Revision loop exhausted after 3 iterations.
- Merge conflict during automated worktree cleanup.
- Ambiguous requirement — two reasonable interpretations and the choice changes the approach.
- Subagent reports "the plan says X but the codebase actually does Y" — human decides which is right.
### 4. Abort gate
**Purpose:** Terminates the operation to prevent damage or waste.
**Behavior:** Stops immediately, preserves state (checkpoint current progress), reports the specific reason.
**Recovery:** Human investigates root cause, fixes, restarts from checkpoint.
**Examples:**
- Context window critically low during execution (POOR tier, >70%) — abort cleanly rather than produce truncated output.
- Critical dependency unavailable mid-run (network down, API key revoked).
- Unrecoverable filesystem state (disk full, permissions lost).
- Safety invariant violated (agent attempted an irreversible destructive action outside approved scope).
## How to use this in a skill
When you write an orchestration skill that has validation checkpoints, **name each checkpoint by its gate type explicitly** and answer three questions:
1. **What condition triggers this gate?** (e.g., "plan file missing", "issue count didn't decrease", "context >70%")
2. **What happens when it fails?** (block / loop back / ask human / abort)
3. **Who resumes, and from where?** (fix precondition + retry, revise + re-check, human decision, restart from checkpoint)
Answering these three up front means your skill never hits "what do we do now?" at runtime.
## Example — a review loop with all four gate types
```
[Pre-flight] plan.md exists and is non-empty? → no: bail, ask user to write a plan first
↓ yes
[Execute] subagent implements task
↓
[Revision] reviewer checks against must-haves → fail: loop back to subagent (max 3)
↓ pass
[Pre-flight] tests pass? → no: bail, report failing tests
↓ yes
[Commit]
↓
(on revision loop exhaustion)
[Escalation] "3 review cycles failed to converge on issue X — pick: force-merge, rewrite task, abandon"
↓ user picks
(on any tier-POOR context pressure during loop)
[Abort] "context at 73%, checkpointing and stopping"
```
The vocabulary is small on purpose. Every gate in every workflow should fit one of these four. If you find yourself inventing a fifth, it's probably a revision gate with extra branching, or an escalation gate in disguise.