Bug: Bedrock stateful conversation corrupted when LLM returns only reasoning content (no text output)

Bug Summary

When using Bedrock's Converse API in stateful mode, if the LLM returns a response containing only reasoning/thinking content with no text output, an empty content field is added to the conversation history. This corrupts the stateful session, causing all subsequent API calls to fail with a ValidationException.

Environment

• Component: app/services/llm/bedrock_llm_service.py (respondent agent simulation)
• LLM Provider: AWS Bedrock
• Model: openai.gpt-oss-120b-1:0 (via Bedrock)
• Feature: Stateful conversation (multi-turn simulation)

Error Message

botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the Converse operation: The content field in the Message object at messages.46 is empty. Add a ContentBlock object to the content field and try again.

Root Cause

The logs show this sequence:

1. Warning logged (first indication of the problem):

``
[Northsea API] 2025-12-11 17:16:53,951 - app.services.llm.bedrock_llm_service - WARNING - Model returned only reasoning content with no text output. Reasoning preview: We need to infer persona facts from prior answers...
``

1. Empty response recorded:

``
[SIMULATION] Respondent Agent generated answer for type 'multi-choice': 'Structured choice: {"type": "simulated_choice", "value": "", "question_id": "1b89238a-30f4-47b5-9a1c-aa53a93a0d55"}'
``

1. Subsequent calls fail because messages.46 (the assistant's response from step 2) has an empty content field.

Impact

• Session corruption: Once an empty content message is added to the stateful conversation, all subsequent calls fail
• No recovery: The session is locked to bedrock due to session affinity, and retries continue to fail
• Circuit breaker triggered: After 3 consecutive failures, bedrock is marked unhealthy (though this doesn't help the corrupted session)

Expected Behavior

When the LLM returns only reasoning content with no text output:

1. The system should not add an empty content message to the conversation history
2. Either retry the request, or use a fallback/placeholder response
3. The stateful session should remain valid for subsequent calls

Reproduction Steps

1. Run a simulation with a persona that causes extended reasoning
2. Wait for a multi-choice question where the LLM "thinks" extensively but produces no text output
3. Observe the empty response being recorded
4. Next turn will fail with the ValidationException

Suggested Fix

In bedrock_llm_service.py, when handling responses in stateful mode:

1. Detect when only reasoning content is returned (the warning already does this)
2. Prevent adding empty content to the conversation history
3. Options:

• Retry the request with a modified prompt encouraging text output
• Use a placeholder response like "[No response generated]"
• Return an error before corrupting the conversation state

Relevant Logs

[Northsea API] 2025-12-11 17:16:53,951 - app.services.llm.bedrock_llm_service - WARNING - Model returned only reasoning content with no text output. Reasoning preview: We need to infer persona facts from prior answers. The persona: 30-year-old Dutch person, living in Netherlands, rent private, household 2 people, no children, postcode gibberish, recently changed bro...

[Northsea API] 2025-12-11 17:16:53,952 - app.agent.respondent_agent - INFO - [SIMULATION] Respondent Agent generated answer for type 'multi-choice': 'Structured choice: {"type": "simulated_choice", "value": "", "question_id": "1b89238a-30f4-47b5-9a1c-aa53a93a0d55"}'

[Northsea API] 2025-12-11 17:16:59,258 - app.services.llm.bedrock_llm_service - ERROR - [respondent-session-c1e21dab-cefc-428b-98d1-9433d755f405] Bedrock validation error - invalid request format. Error: The content field in the Message object at messages.46 is empty. Add a ContentBlock object to the content field and try again.

Labels

• bug
• bedrock
• stateful-conversation
• simulation

🤖 Generated with Claude Code