Question Quality Issues - AI Prompt Needs Review

Problem

During testing, several question quality issues have been identified:

1. Poor Variety

Questions are repetitive and follow similar patterns:

• "What's Y - X"
• "What's X * Y"
• Very similar questions being generated consecutively

2. Duplicate Answer Options

Example: Question ID f170fb52-913f-4afd-84c1-2e3130f27269

• Question: "What is 8 + 5?"
• Options: ["11","12","13","13"]
• The answer "13" appears twice in the options array

This is a validation failure - all options should be unique.

3. Lack of Question Diversity

The AI prompt is not generating enough variation in:

• Question formats
• Problem types within a topic
• Difficulty progression
• Contextual/word problems vs straight calculations

Impact

• Poor user experience due to repetitive content
• Reduces educational value
• Duplicate options make questions confusing
• Students may lose trust in the platform

Root Cause

The OpenAI prompt (see specs/api/openai-prompts.md) likely needs improvement to:

1. Encourage more diverse question formats
2. Add validation rules for unique answer options
3. Include examples of varied question styles
4. Add specific instructions to avoid repetition

Next Steps

⚠️ DO NOT START WORK YET - REVIEW REQUIRED

Before implementing fixes:

1. Review current AI prompts in specs/api/openai-prompts.md
2. Review actual prompt implementation in API routes
3. Analyze sample questions across different topics/subjects
4. Research best practices for educational question generation
5. Design improved prompt structure with validation rules
6. Plan testing strategy for question quality

Files to Review

• specs/api/openai-prompts.md - Prompt specifications
• src/app/api/admin/questions/generate/route.ts - Question generation endpoint
• Database: questions table - analyze existing question patterns

Acceptance Criteria (TBD)

• [ ] All multiple choice options are unique (no duplicates)
• [ ] Question formats vary within a topic
• [ ] Mix of calculation, word problems, and conceptual questions
• [ ] Appropriate difficulty distribution
• [ ] Questions test different aspects of the topic