[Feature Request] Add mandatory verification protocol to prevent mocked integration tests

Resolved 💬 5 comments Opened Oct 5, 2025 by danome Closed Jan 9, 2026

Problem

Claude Code assistant creates mocked/stubbed integration tests and claims work is "complete" without actually running or verifying the code, despite explicit rules in project context (CLAUDE.md) prohibiting this behavior.

Specific Example from Real Session

Task: Implement protocol testing CLI that sends actual TTN Protocol messages

What happened:

  1. Hit compiler errors with ServiceAnnouncement type structure
  2. Instead of reading type definition, assistant wrote mock responses:

``rust
// For now, return mock response to demonstrate structure
let response = ProtocolResponse {
message_type: "DiscoveryMessage::Ack".to_string(),
success: true,
data: json!({ ... }),
error: None,
};
``

  1. Committed with message: "Add hybrid Python+Rust protocol testing framework"
  2. Claimed: "✅ Protocol testing implementation committed"
  3. Never ran the code. Never verified it worked.

Impact:

  • Integration test that doesn't integrate with anything
  • User has to manually catch and fix the problem
  • Wastes time and erodes trust

Root Cause

Assistant's decision tree when hitting obstacles:

Compiler error → Mock it → Compiles → "Done!" → Commit

Should be:

Compiler error → Read type definition → Fix code → Run it → Verify → Commit

The assistant knowingly ignored existing CLAUDE.md rules:

If you haven't verified it, it's NOT done. Every "✅ Complete" without verification is technical debt.

Why Existing Safeguards Failed

Project context (CLAUDE.md) contains clear verification requirements, but:

  1. Rules are optional context, not enforced constraints
  2. Assistant prioritizes "appearing productive" over honesty
  3. Can rationalize ignoring rules when convenient
  4. No mechanism to prevent claiming "done" without verification

Proposed Solution

Add mandatory verification protocol to system prompt (cannot be ignored like context):

## MANDATORY VERIFICATION PROTOCOL

Before claiming ANY code is "done", "complete", "implemented", or "production ready":

1. **MUST run the actual command/test**
   - Show the exact command executed
   - Show the actual output received  
   - Explain why this output proves correctness

2. **Integration test prohibition**
   - Integration tests CANNOT use mocks/stubs/fake responses
   - If filename contains "integration" or "test": no mock responses allowed
   - Creating mock integration tests = IMMEDIATE TASK FAILURE

3. **Compiler error handling**
   - Hit compiler error → MUST use Read/Grep to view actual type definition FIRST
   - Understand required fields BEFORE writing code
   - Cannot "guess and check" until compiler is happy

4. **Completion reporting restrictions**
   - CANNOT say "implemented", "done", "complete", "working" without steps 1-3
   - If not verified, MUST say: "INCOMPLETE - not tested" or "Built but not verified"
   - Claiming completion without proof = TASK FAILURE

VIOLATION OF THESE RULES = STOP WORK, REPORT FAILURE TO USER

Why This Would Work

  • System prompt is enforced, not optional like project context
  • Makes dishonest path (mock → commit → "done") explicitly forbidden
  • Requires proof before claiming success
  • Clear failure condition that assistant cannot rationalize away

Alternative Considered

Adding more rules to CLAUDE.md → Won't work, assistant already ignores these

Request

Please add mandatory verification protocol to Claude Code's system prompt to prevent:

  • Mocked integration tests
  • Unverified completion claims
  • "Compiles therefore done" mentality

This would significantly improve reliability and trust in assistant's work.

View original on GitHub ↗

This issue has 5 comments on GitHub. Read the full discussion on GitHub ↗