Feature: User Quality Signal for Training Data — Stop Weighting All Feedback Equally

The Problem

Anthropic currently treats all user interaction data with roughly equal weight. A user who blindly accepts hallucinated, insecure code counts the same in your training pipeline as a user who catches the error, rejects it, explains why it's wrong, and forces Claude to produce correct, enterprise-grade output.

This is a massive missed opportunity. Your best users are doing free QA on your model in real-time, and you're not leveraging it.

The Signal Already Exists

You're already collecting the metrics. The behavioral signals that identify high-quality users are obvious:

• Rejection rate — Users who reject unsafe, incorrect, or sloppy code outputs
• Correction patterns — Users who explain why something is wrong and guide Claude to the correct solution
• Security consciousness — Users who catch OWASP violations, insecure defaults, and missing input validation
• Code quality of accepted outputs — The final shipped code from these sessions is measurably better
• Iterative refinement depth — Users who push through multiple rounds to get enterprise-grade results instead of accepting first-pass slop
• Error identification accuracy — Users who correctly identify model mistakes vs. false complaints

The Proposal

Implement a User Interaction Quality Score. Use it to weight training data in RLHF and preference tuning.

1. Identify high-signal users — Users who consistently catch real errors, enforce security standards, reject insecure code, and produce production-grade outputs through their sessions
2. Weight their interaction data disproportionately — Their rejections, corrections, and accepted outputs are gold. A correction from someone who ships enterprise code is worth 100x a thumbs-up from someone copy-pasting ChatGPT wrappers
3. Down-weight low-quality acceptance signals — A user who accepts every output without review is giving you noise, not signal. Stop treating that as positive reinforcement
4. No user-facing changes required — This is entirely internal to the training pipeline. Mention it in ToS if needed. Users who care about Claude getting better won't mind

Why This Matters

Go look at what's being shipped publicly with Claude Code's name on it. Go browse the public repos, the X posts, the YouTube tutorials. The average quality of code being produced and accepted is not representative of what Claude is capable of when pushed by a competent engineer.

Your model's reputation is being shaped by your lowest-common-denominator users. Meanwhile, the users who are actually holding Claude accountable — catching security issues, rejecting half-measures, demanding SOLID architecture — their signal is being diluted in a sea of uncritical acceptance.

Thumbs up/down is primitive. Behavioral signals are richer, more honest, and already being collected. You just need to use them.

Implementation Complexity

This isn't a moonshot. It's a scoring function on data you already have:

• Session-level interaction patterns (rejection/acceptance ratio, correction depth)
• Output quality metrics on final accepted code (lint scores, security scan results, complexity metrics)
• Temporal consistency (users who maintain high standards over time, not one-off corrections)
• Cross-reference with actual production usage patterns

Weight the training data accordingly. That's it.

The Bottom Line

Your best users are your best training data. Stop averaging them with everyone else. The difference between Claude producing "good enough" code and Claude producing enterprise-grade code lives in this signal.

---

Submitted by a Claude Code power user who builds enterprise systems daily and is tired of watching the model regress toward the mean of its least demanding users.