Steering the Vibe: Verify

This is the second post in a series about tightening the loop and improving the output when programming using ai code assist. In the previous post I introduced assist, the CLI I’m building to enforce determinism on the LLM’s output. In this post, we’ll look at another command in the tool: assist verify. The purpose of assist verify is to run a series of deterministic checks against the codebase to help steer the LLM towards more maintainable code.

If you are an experienced developer you’ll notice this approach is similar to the checks we might execute during CI. The key change here is running those checks in parallel and at the end of every change the code-assist tool makes.

Configuration#

While it’s possible to put this in a Stop hook, my preference is a slash command that I can also call manually as needed, even directly from the terminal via claude /verify.

        
markdown
        

            
        
     
# CLAUDE.md

When finishing code changes, you should run /verify

        
markdown
        

            
        
     
# .claude/commands/verify.md

---

## description: Run all verification commands

Run `assist verify 2>&1` (not npm run, not npx - just `assist verify 2>&1` directly). If it fails, fix all errors and run again until it passes.

In my package.json I can then define the scripts that assist verify will run, which will be everything prefixed with verify:. Here’s an example of this configuration:

        
json
        

            
        
     
"scripts": {
  "verify:knip": "knip --no-progress",
  "verify:build": "tsc -b && vite build --logLevel error",
  "verify:lint": "biome check --write .",
  "verify:test": "vitest run --silent",
  "verify:hardcoded-colors": "tsx scripts/find-hardcoded-colors.ts",
  "verify:duplicate-code": "jscpd --format 'typescript,tsx' --exitCode 1 --ignore '**/*.test.*' src"
}

The slash command definition is global (assist sync keeps it up to date) but the scripts are project specific, allowing each project to define its own verification commands.

Reducing token usage#

While it’s possible to have these checks defined in CLAUDE.md and direct Claude Code to use sub-agents to run them, this is very expensive in terms of token usage. You could also use npm-run-all and define the parallelism in package.json, but this leads to Claude Code sometimes trying to run individual scripts rather than the aggregate command, so having a single assist verify command reduces confusion and token usage.

The scripts above have options to reduce output where possible. We only care about failures and so we don’t write to the context window anything other than errors. Running them all in parallel means the checks complete quickly, and even with multiple failures Claude can decipher and fix the errors that are presented.

Leveraging existing tools#

Build, lint and test commands are familiar scripts that might already exist in your project, but there is more we can do to steer the LLM to more maintainable code. Often, and especially when iterating on a new implementation, AI code-assist tools will introduce orphan code that is never used. To help catch this I use knip which finds unused exports and packages, reducing the slow build up of dead code.

With no memory outside of what is in the current context window, code-assist can often result in the same functionality being reimplemented across multiple files. To steer away from this I use jscpd which finds duplicate code blocks across the codebase. Notice the --ignore '**/*.test.*' flag to allow test code to have some duplication without causing failures.

Implementing custom checks#

As we observe the LLM’s emergent behaviours we can implement custom checks to enforce correctness. Again, where we could put these suggestions in CLAUDE.md and hope the LLM follows them, having them as deterministic scripts means they are always applied.

In a game project I’m vibe coding I’m using open-color to provide a consistent color palette. However, I’ve noticed that hardcoded colors often creep into the codebase and to steer away from this one of the verify commands calls tsx scripts/find-hardcoded-colors.ts to enforce the usage of the color library:

        
typescript
        

            
        
     
// scripts/find-hardcoded-colors.ts
import { execSync } from "node:child_process";

const pattern = "0x[0-9a-fA-F]{6}|#[0-9a-fA-F]{3,6}";

try {
  const output = execSync(`grep -rEnH '${pattern}' src/`, {
    encoding: "utf-8",
  });

  const lines = output.trim().split("\n");

  console.log("Hardcoded colors found:\n");

  for (const line of lines) {
    const match = line.match(/^(.+):(\d+):(.+)$/);
    if (match) {
      const [, file, lineNum, content] = match;
      const colorMatch = content.match(/0x[0-9a-fA-F]{6}|#[0-9a-fA-F]{3,6}/);
      const color = colorMatch?.[0] ?? "unknown";
      console.log(`${file}:${lineNum} → ${color}`);
    }
  }

  console.log(`\nTotal: ${lines.length} hardcoded color(s)`);
  console.log("\nUse colors from the 'open-color' (oc) library instead.");
  process.exit(1);
} catch {
  console.log("No hardcoded colors found.");
  process.exit(0);
}

Even stricter linting#

Where I can usually trust humans to follow existing conventions, again, the context window limitation might not lead to the LLM choosing the best option. Biome already has a number of defaults but to further steer the vibe I configure some additional rules:

        
json
        

            
        
     
// biome.json
"linter": {
  "enabled": true,
  "rules": {
    "recommended": true,
    "style": {
      "useFilenamingConvention": {
        "level": "error",
        "options": {
          "filenameCases": ["export"],
          "requireAscii": true
        }
      },
      "useConsistentTypeDefinitions": {
        "level": "error",
        "options": {
          "style": "type"
        }
      },
      "useConsistentMemberAccessibility": {
        "level": "error",
        "options": {
          "accessibility": "noPublic"
        }
      }
    }
  }
},

Even so, there are rules that biome does not support, like enforcing no require() or import(). We can of course write our own custom scripts (like the hardcoded colors example above) to enforce these rules as needed. However, as I feel their absence in other projects I’ll hoist them into assist lint so they are available across all projects that use assist, and add a "verify:lint": "assist lint" to my package.json.

Workflow loop#

The result is a workflow loop where at the end of every set of changes Claude will run /verify and immediately fix any of the issues that are found. This steers the output towards more maintainable code with every change. What this does not solve is the LLM’s tendency to continually make files larger and larger as changes are made. In my next post, I’ll share how I’m using assist refactor to solve for this problem.