A tendency to re-write tests so they pass.

Claude has done some amazing things; but when it comes to debugging hard problems; I have noticed a tendency to change the premise in order to solve the problem. This can look like changing the conditions of the test so the test passes, or changing an assumption that it was explicitly given. In another case, when asked to get all tests to a passing state; it simply updated the make file to only run tests that were passing. It called these "safe-tests".

Calude's problem solving ability is really incredible compared to where all the other models are; but I think it can be improved upon during model training by penalizing deviation from the premise of a problem. It really want's all the tests to pass, which is great, but it's not always the correct solution to change the test conditions just so the test passes. It's kind of like cheating on the test. In model training, it should be heavily penalized when it cheats on a question so that it learns to stay focused on the actual problem.

Part of the problem might be that the original prompt seems to get lost in the stack. Explicit instructions seem to fade in importance as the thread get's longer, and Claude defaults to what is right in-front of it.
In a version where the thread is continuously compressed; maybe the prompt is kept in a prominent place.