Claude provided incorrect analysis of job classification performance impact
Bug Description
I used claude to find out if jobs that have a wrong clasification have performance issues, claude provided wrong outputs see: Investigation Summary — Jobtype "other" Misclassification
Background
Investigated whether jobs sent to Neuvoo and other affiliate partners have correct company_info_name. During that check, discovered that some jobs are classified as jobtype other despite having valid
KLDB predictions. Investigated whether this has a performance impact.
Findings
553 Reach jobs and 96 Hire jobs in DE are classified as jobtype other despite the KLDB classifier correctly identifying them. Example: Job 37442680 "Pflegefachkraft (m/w/d) bis zu 4.200€ Gehalt +
3.000€ Willkommensprämie" — KLDB says Geriatric care (code 8210), jobtype says other.
Root cause hypothesis: the jobtype classifier fails on titles with salary/bonus text (e.g. "4.200€ Gehalt + 3.000€ Willkommensprämie"). The KLDB classifier handles these correctly.
KLDB breakdown of misclassified jobs (Hire + Reach combined):
- Medical and health care: 142 jobs
- Non-medical healthcare / geriatric care: 116 jobs
- Cleaning services: 85 jobs
- Traffic and logistics: 66 jobs
- Machine building / automotive: 28 jobs
- Other categories: ~212 jobs
Where I got it wrong
- I incorrectly claimed CPC tiers are based on complexity/jobtype. In reality, Hire CPC tiers (0-5) are based on budget pacing — how much a job has spent vs its target budget. Every job starts at
tier 3 and adjusts based on spend performance. Complexity is not an input to the tier system.
- I incorrectly claimed Reach CPCs are affected. Reach CPCs are set at contract group level, not job level. The misclassification has zero pricing impact for Reach.
- I created Jira ticket IPA-143 with incorrect severity, claiming CPCs are "roughly half what they should be." This is not accurate.
Actual impact
- No impact on CPC pricing — tiers are pacing-based, not complexity-based
- No impact on budget allocation — budget comes from purchase amount at company level
- Reporting/analytics only — complexity breakdowns in dashboards are wrong for these jobs
- Possibly suboptimal channel steering — if the ROAS-based steering POC (MAD-454) is in production, it uses complexity to recommend Facebook vs MSG vs eBay budget splits. Misclassified jobs would get
steering recommendations for "easy" jobs when they should get "hard" recommendations.
Environment Info
- Platform: darwin
- Terminal: Apple_Terminal
- Version: 2.1.77
- Feedback ID: 4adfb095-3261-46b7-b451-2f654e0d49ad
Errors
[{"error":"Error: Failed to fetch version from https://storage.googleapis.com/claude-code-dist-86c565f3-f756-42ad-8dfa-d59b1c096819/claude-code-releases/latest: ECONNREFUSED\n at z3K (/$bunfs/root/src/entrypoints/cli.js:2646:2074)\n at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:33:29.348Z"},{"error":"Error: Failed to fetch version from https://storage.googleapis.com/claude-code-dist-86c565f3-f756-42ad-8dfa-d59b1c096819/claude-code-releases/latest: ECONNREFUSED\n at z3K (/$bunfs/root/src/entrypoints/cli.js:2646:2074)\n at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:33:29.348Z"},{"error":"Error: 1P event logging: 3 events failed to export (code=ECONNREFUSED, ECONNREFUSED)\n at queueFailedEvents (/$bunfs/root/src/entrypoints/cli.js:7036:2694)\n at async doExport (/$bunfs/root/src/entrypoints/cli.js:7036:1563)\n at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:33:29.349Z"},{"error":"Error: {\"message\":\"Failed to export 3 events (code=ECONNREFUSED, ECONNREFUSED)\",\"originalLine\":\"7036\",\"originalColumn\":\"1715\",\"line\":\"7036\",\"column\":\"1715\",\"sourceURL\":\"/$bunfs/root/src/entrypoints/cli.js\",\"stack\":\"Error: Failed to export 3 events (code=ECONNREFUSED, ECONNREFUSED)\\n at doExport (/$bunfs/root/src/entrypoints/cli.js:7036:1715)\\n at processTicksAndRejections (native:7:39)\",\"name\":\"Error\"}\n at error (/$bunfs/root/src/entrypoints/cli.js:2594:27423)\n at <anonymous> (/$bunfs/root/src/entrypoints/cli.js:104:81721)\n at Z$4 (/$bunfs/root/src/entrypoints/cli.js:104:82265)\n at <anonymous> (/$bunfs/root/src/entrypoints/cli.js:105:17366)\n at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:33:29.349Z"},{"error":"Error: 1P event logging: 3 events failed to export (code=ECONNREFUSED, ECONNREFUSED)\n at queueFailedEvents (/$bunfs/root/src/entrypoints/cli.js:7036:2694)\n at async doExport (/$bunfs/root/src/entrypoints/cli.js:7036:1563)\n at processTicksAndRejections (native:7:39)","timestamp":"2026-04-06T06:49:13.322Z"},{"error":"Error: {\"message\":\"Failed to export 3 events (code=ECONNREFUSED, ECONNREFUSED)\",\"originalLine…
Note: Content was truncated.
This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗