Claude provided incorrect analysis of job classification performance impact

Resolved 💬 2 comments Opened Apr 9, 2026 by hey-ronja-hieke Closed Apr 12, 2026

Bug Description
I used claude to find out if jobs that have a wrong clasification have performance issues, claude provided wrong outputs see: Investigation Summary — Jobtype "other" Misclassification

Background

Investigated whether jobs sent to Neuvoo and other affiliate partners have correct company_info_name. During that check, discovered that some jobs are classified as jobtype other despite having valid
KLDB predictions. Investigated whether this has a performance impact.

Findings

553 Reach jobs and 96 Hire jobs in DE are classified as jobtype other despite the KLDB classifier correctly identifying them. Example: Job 37442680 "Pflegefachkraft (m/w/d) bis zu 4.200€ Gehalt +
3.000€ Willkommensprämie" — KLDB says Geriatric care (code 8210), jobtype says other.

Root cause hypothesis: the jobtype classifier fails on titles with salary/bonus text (e.g. "4.200€ Gehalt + 3.000€ Willkommensprämie"). The KLDB classifier handles these correctly.

KLDB breakdown of misclassified jobs (Hire + Reach combined):

  • Medical and health care: 142 jobs
  • Non-medical healthcare / geriatric care: 116 jobs
  • Cleaning services: 85 jobs
  • Traffic and logistics: 66 jobs
  • Machine building / automotive: 28 jobs
  • Other categories: ~212 jobs

Where I got it wrong

  1. I incorrectly claimed CPC tiers are based on complexity/jobtype. In reality, Hire CPC tiers (0-5) are based on budget pacing — how much a job has spent vs its target budget. Every job starts at

tier 3 and adjusts based on spend performance. Complexity is not an input to the tier system.

  1. I incorrectly claimed Reach CPCs are affected. Reach CPCs are set at contract group level, not job level. The misclassification has zero pricing impact for Reach.
  2. I created Jira ticket IPA-143 with incorrect severity, claiming CPCs are "roughly half what they should be." This is not accurate.

Actual impact

  • No impact on CPC pricing — tiers are pacing-based, not complexity-based
  • No impact on budget allocation — budget comes from purchase amount at company level
  • Reporting/analytics only — complexity breakdowns in dashboards are wrong for these jobs
  • Possibly suboptimal channel steering — if the ROAS-based steering POC (MAD-454) is in production, it uses complexity to recommend Facebook vs MSG vs eBay budget splits. Misclassified jobs would get

steering recommendations for "easy" jobs when they should get "hard" recommendations.

Environment Info

  • Platform: darwin
  • Terminal: Apple_Terminal
  • Version: 2.1.77
  • Feedback ID: 4adfb095-3261-46b7-b451-2f654e0d49ad

Errors

[{"error":"Error: Failed to fetch version from https://storage.googleapis.com/claude-code-dist-86c565f3-f756-42ad-8dfa-d59b1c096819/claude-code-releases/latest: ECONNREFUSED\n    at z3K (/$bunfs/root/src/entrypoints/cli.js:2646:2074)\n    at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:33:29.348Z"},{"error":"Error: Failed to fetch version from https://storage.googleapis.com/claude-code-dist-86c565f3-f756-42ad-8dfa-d59b1c096819/claude-code-releases/latest: ECONNREFUSED\n    at z3K (/$bunfs/root/src/entrypoints/cli.js:2646:2074)\n    at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:33:29.348Z"},{"error":"Error: 1P event logging: 3 events failed to export (code=ECONNREFUSED, ECONNREFUSED)\n    at queueFailedEvents (/$bunfs/root/src/entrypoints/cli.js:7036:2694)\n    at async doExport (/$bunfs/root/src/entrypoints/cli.js:7036:1563)\n    at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:33:29.349Z"},{"error":"Error: {\"message\":\"Failed to export 3 events (code=ECONNREFUSED, ECONNREFUSED)\",\"originalLine\":\"7036\",\"originalColumn\":\"1715\",\"line\":\"7036\",\"column\":\"1715\",\"sourceURL\":\"/$bunfs/root/src/entrypoints/cli.js\",\"stack\":\"Error: Failed to export 3 events (code=ECONNREFUSED, ECONNREFUSED)\\n    at doExport (/$bunfs/root/src/entrypoints/cli.js:7036:1715)\\n    at processTicksAndRejections (native:7:39)\",\"name\":\"Error\"}\n    at error (/$bunfs/root/src/entrypoints/cli.js:2594:27423)\n    at <anonymous> (/$bunfs/root/src/entrypoints/cli.js:104:81721)\n    at Z$4 (/$bunfs/root/src/entrypoints/cli.js:104:82265)\n    at <anonymous> (/$bunfs/root/src/entrypoints/cli.js:105:17366)\n    at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:33:29.349Z"},{"error":"Error: 1P event logging: 3 events failed to export (code=ECONNREFUSED, ECONNREFUSED)\n    at queueFailedEvents (/$bunfs/root/src/entrypoints/cli.js:7036:2694)\n    at async doExport (/$bunfs/root/src/entrypoints/cli.js:7036:1563)\n    at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:49:13.322Z"},{"error":"Error: {\"message\":\"Failed to export 3 events (code=ECONNREFUSED, ECONNREFUSED)\",\"originalLine…

Note: Content was truncated.

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗