Settings validation passes invalid types and nested fields that cause runtime errors

Title: Settings validation passes invalid types and nested fields that cause runtime errors

Summary

The v1.0.82 settings validation successfully rejects unknown top-level fields, but allows through invalid data types, malformed nested structures, and incorrect array elements. These pass validation silently but can cause runtime failures or unexpected behavior.

Issue Description

Testing revealed that validation only catches invalid top-level field names but misses several critical validation scenarios. Users can create settings.json files that appear valid and pass validation, but contain type mismatches or invalid nested properties that break at runtime.

Current Gaps in Validation

1. Type Validation Not Enforced

Settings with incorrect data types pass validation but may cause runtime failures:

{
  "cleanupPeriodDays": "thirty",        // String instead of number
  "includeCoAuthoredBy": "yes",         // String instead of boolean  
  "permissions": "should be an object",  // String instead of object
  "enabledMcpjsonServers": "memory",    // String instead of array
  "env": ["wrong", "type"]              // Array instead of object
}

Expected: Type mismatch should be caught during validation
Actual: File passes validation, potential runtime error

2. Nested Invalid Fields Not Detected

Invalid fields within valid parent objects are not caught:

{
  "permissions": {
    "allow": ["Bash"],
    "invalidNestedField": "not caught",     // ❌ Should fail
    "randomProperty": true,                 // ❌ Should fail
    "deeplyNested": {                       // ❌ Should fail
      "invalid": "structure"
    }
  },
  "hooks": {
    "Stop": [{
      "matcher": "",
      "hooks": [{
        "type": "command",
        "command": "test.sh",
        "invalidHookField": "not caught"    // ❌ Should fail
      }],
      "extraField": "also not caught"       // ❌ Should fail
    }],
    "InvalidHookType": []                   // ❌ Should fail
  }
}

3. Array Element Validation Missing

Arrays accept invalid element types without validation:

{
  "permissions": {
    "allow": [123, true, {"obj": "invalid"}],  // Should only accept strings
    "additionalDirectories": [123, false]       // Should only accept strings
  },
  "enabledMcpjsonServers": [{"not": "string"}]  // Should only accept string array
}

4. Environment Variable Key Validation

Invalid environment variable names are accepted:

{
  "env": {
    "$INVALID_VAR": "value",      // Starts with $
    "123NUMERIC": "value",        // Starts with number
    "KEY WITH SPACES": "value",   // Contains spaces
    "key-with-dashes": "value"    // May cause issues
  }
}

5. Hook Structure Validation

Invalid hook structures pass validation:

{
  "hooks": {
    "PreToolUse": "should be array",           // Wrong type
    "Stop": {                                   // Wrong structure
      "wrong": "format"  
    },
    "PostToolUse": 123                          // Wrong type
  }
}

Reproduction Steps

1. Create any of the above JSON examples as ~/.claude/settings.json
2. Start Claude Code
3. Observe that validation passes despite invalid structures
4. May encounter runtime errors when invalid settings are accessed

Test Files Available

Created comprehensive test suite at /private/tmp/claude-settings-test/ with:

• invalid/wrong_data_types.json - Type mismatches
• invalid/wrong_nested_types.json - Nested type errors
• invalid/nested_invalid_fields.json - Invalid nested properties

Suggested Improvements

1. Implement JSON Schema Validation

Use the existing $schema reference to validate against:

"$schema": "https://json.schemastore.org/claude-code-settings.json"

2. Add Type Checking

• Validate primitive types (string, number, boolean)
• Validate object structures recursively
• Validate array element types

3. Deep Field Validation

• Recursively validate nested objects
• Check all properties against allowed fields at each level
• Validate hook structures match expected format

4. Enhanced Error Messages

Current: Generic validation error
Suggested:

Settings validation failed:
- Line 5: "cleanupPeriodDays" must be a number, got string
- Line 8: "permissions.invalidField" is not a valid property
- Line 12: "hooks.Stop" must be an array, got object

5. Validation Strictness Levels

Consider adding validation modes:

• strict: Reject any unknown fields (current behavior for top-level)
• warning: Allow unknown fields but warn
• permissive: Only validate types and required fields

Impact

• Users may have settings that appear valid but cause runtime errors
• Difficult to debug configuration issues without proper validation messages
• Type mismatches could cause unexpected behavior or crashes

Priority

Medium-High - While top-level validation works, the gaps could lead to confusing user experiences when valid-looking configurations fail at runtime.

Environment

• Claude Code v1.0.82
• Tested on macOS/Linux
• Test suite available with 13 test cases demonstrating issues

Additional Context

Created a comprehensive test suite that validates these scenarios. Happy to provide additional test cases or help validate fixes. The current validation is a great first step - these enhancements would make it bulletproof.

Let me know if you need any clarification or additional test cases to reproduce these validation gaps.