[BUG] Product model names not trimmed before save, causing duplicate products

Summary

GPT completions occasionally return model names with leading/trailing whitespace (e.g., " coco handle" instead of "coco handle"). Since the model name is not trimmed before saving, and find_product uses exact match, duplicate products are created.

Impact

Current state in production:

• 71 duplicate product groups across all brands
• 144 affected products (73 will need to be merged/deleted)
• 21,309 affected variants
• 58,357 affected listings

Problems caused:

1. Deduplication/Reassignment failures - apply_db_ops.py loads products into a dictionary keyed by normalized model. When duplicates exist, the wrong product gets selected, causing operations to fail silently
2. Fragmented data - Listings and variants split across duplicate products
3. Incorrect product statistics - Listing counts, price ranges fragmented
4. Affects any code that looks up products by normalized model name

Root Cause

In listing_processor_service.py line 199-203:

product_model = Product(
    brand=brand,
    model=completion.model,  # <-- No .strip() here!
    category=category,
)

Then find_product does exact match:

return await Product.find_one({
    "brand": product.brand,
    "category": product.category,
    "model": product.model,  # " coco handle" != "coco handle"
})

Proposed Fix

Option 1 (Recommended): Add .strip() in listing_processor_service.py:

product_model = Product(
    brand=brand,
    model=completion.model.strip(),  # Trim whitespace
    category=category,
)

Option 2: Add Pydantic validator in Product model:

from pydantic import field_validator

class Product(Document):
    brand: BrandEnum
    model: str
    category: CategoryEnum

    @field_validator('model')
    @classmethod
    def strip_model(cls, v: str) -> str:
        return v.strip() if v else v

Option 3: Normalize in find_product query (less ideal - doesn't prevent bad data)

Related

• GitHub Issue #1532 - Documents the deduplication failure caused by this bug
• cleanup_duplicate_products.py - Script to fix existing duplicates in MongoDB
• report_duplicate_products.py - Script to generate report of all duplicates

Checklist

• [ ] Fix the code to trim model names before saving
• [ ] Run cleanup_duplicate_products.py --execute --brands all to fix existing data
• [ ] Re-run apply_db_ops.py for affected brands after cleanup

🤖 Generated with Claude Code