Test Framework: E2E tests require hardcoded API keys
Issue Description
The end-to-end tests in tests/v2/test_progress_modal_ui.py and tests/v2/test_latency_optimizations.py require API keys to be hardcoded directly in the test files, which is not ideal for automated testing or CI/CD pipelines.
Current Behavior
Tests are currently marked with @pytest.mark.skip and include placeholder values:
configure_api_keys(
popup_page,
gemini_key='YOUR_GEMINI_KEY', # Replace with real key
splitwise_key='YOUR_SPLITWISE_KEY', # Replace with real key
group_id='YOUR_GROUP_ID' # Replace with real group ID
)
Problems
- Security Risk: API keys would be committed to source control if hardcoded
- Not Automated: Tests cannot run in CI/CD without manual key insertion
- No Test Isolation: Using production group can pollute real data
- Manual Setup: Each developer needs to manually update test files
Affected Tests
Progress Modal Tests (4 tests):
test_progress_modal_appears_on_button_clicktest_progress_modal_step_transitionstest_progress_modal_shows_error_statetest_progress_modal_persistence_during_workflow
Latency Optimization Tests (3 tests):
test_llm_cache_hit_on_duplicate_ordertest_async_comment_does_not_block_success_messagetest_end_to_end_latency_improvement
Total: 7 tests currently skipped
Proposed Solution
Option 1: Environment Variables
import os
GEMINI_KEY = os.getenv('TEST_GEMINI_API_KEY')
SPLITWISE_KEY = os.getenv('TEST_SPLITWISE_API_KEY')
GROUP_ID = os.getenv('TEST_SPLITWISE_GROUP_ID')
# Skip if not configured
pytestmark = pytest.mark.skipif(
not all([GEMINI_KEY, SPLITWISE_KEY, GROUP_ID]),
reason="API keys not configured in environment"
)
Option 2: Test Configuration File
# tests/v2/test_config.json (gitignored)
{
"gemini_api_key": "...",
"splitwise_api_key": "...",
"test_group_id": "..."
}
Option 3: pytest-env Plugin
# pytest.ini
[pytest]
env =
TEST_GEMINI_API_KEY=file:.env:GEMINI_API_KEY
TEST_SPLITWISE_API_KEY=file:.env:SPLITWISE_API_KEY
TEST_GROUP_ID=file:.env:SPLITWISE_GROUP_ID
Recommendation
Option 1 (Environment Variables) is recommended because:
- ✅ Standard practice for CI/CD
- ✅ No additional dependencies
- ✅ Works with GitHub Actions secrets
- ✅ Easy to configure locally (
.envfile)
Related Issue
See also: "Create dedicated test Splitwise group to avoid polluting production data" (separate issue)
Impact
Current: 7/12 tests (58%) are skipped
After Fix: All tests can run in CI/CD with proper secrets configuration
Files to Modify
tests/v2/test_progress_modal_ui.pytests/v2/test_latency_optimizations.pytests/v2/conftest.py(add fixture for API keys).env.example(template for developers).gitignore(ensure.envis ignored)README.mdortests/v2/README_LATENCY_TESTS.md(document setup)
Priority
Medium - Tests work with manual setup, but automation would improve developer experience and enable CI/CD.
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗