socialgpt: Onboarding - Stream ad-hoc video analysis with real-time Content DNA reveal

Problem

Currently, first-time users in /onboarding/2 get inferior content personalization because:

• Video analyses don't exist yet (scraper runs hourly, not immediately)
• System falls back to lightweight API metadata (descriptions/captions only)
• No transcriptions, visual analysis, or executive summaries
• Results in generic, less accurate content pillar and account type suggestions

Impact: Users need to manually edit inferred topics more often, reducing onboarding quality and conversion.

Priority: 🔴 CRITICAL - This directly impacts onboarding conversion and should be prioritized.

Proposed Solution

Create a premium streaming onboarding experience where we:

1. Analyze user's top 5 videos on-demand during onboarding step 2
2. Stream progress and insights in real-time via Server-Sent Events
3. Display video-by-video analysis results as they complete
4. Generate and present a rich Content DNA summary at the end
5. Make it feel dynamic, personalized, and impressive (maximum wow factor)

Refined Requirements

Core Decisions

• ✅ Video limit: Fixed at 5 videos (~2-3 minute experience)
• ✅ Fallback: If analysis fails, use current API metadata approach
• ✅ Polish level: Maximum wow factor with advanced animations, haptics, particles, branded experience
• ✅ Priority: Critical - blocking other onboarding work

User Experience Goals

• Engaging: Users should feel like they're watching their content come to life
• Impressive: "Wow, this app really understands me" moment
• Informative: Clear insights into what makes their content unique
• Fast: 2-3 minutes feels quick with engaging visuals
• Reliable: Graceful degradation if some videos fail

Architecture Proposal

Backend Changes

1. New Streaming Endpoint

POST /api/v1/onboarding/analyze-content (SSE)

Request:

{
  "platforms": ["tiktok", "instagram"],
  "video_limit": 5  // Fixed, not user-configurable
}

Response: Server-Sent Events stream

Event Types:

// Phase 1: Fetching videos
{
  event: "fetching_videos",
  data: {
    platform: "tiktok",
    status: "fetching"
  }
}

// Phase 2: Analyzing individual videos (parallel)
{
  event: "analyzing_video",
  data: {
    platform: "tiktok",
    video_index: 1,
    total_videos: 5,
    video_title: "5 minute HIIT workout",
    thumbnail_url: "..."
  }
}

// Phase 3: Video analysis complete
{
  event: "video_complete",
  data: {
    platform: "tiktok",
    video_index: 1,
    video_title: "5 minute HIIT workout",
    thumbnail_url: "...",
    key_insights: ["High energy presentation", "Clear workout instructions", "Professional gym setting"],
    themes: ["Fitness", "HIIT", "Quick workouts"]
  }
}

// Phase 4: Inferring content pillars
{
  event: "inferring_pillars",
  data: {
    status: "processing",
    message: "Identifying your content themes..."
  }
}

// Phase 5: Inferring account type
{
  event: "inferring_account_type",
  data: {
    status: "processing",
    message: "Analyzing your content style..."
  }
}

// Phase 6: Complete with Content DNA
{
  event: "complete",
  data: {
    account_type: "fitness_health_cooking_creator",
    account_type_label: "Fitness, Health, and Cooking Creator",
    account_type_confidence: 0.95,
    content_pillars: [
      {
        content: "High-intensity interval training workouts for busy professionals",
        topic: "FITNESS",
        confidence: 0.92,
        supporting_videos: 8
      }
    ],
    content_dna: {
      primary_themes: ["Fitness", "Nutrition", "Wellness"],
      target_audience: "Busy professionals aged 25-40",
      content_style: "Educational with high energy",
      unique_angle: "Time-efficient workouts and meals for busy schedules",
      posting_patterns: {
        frequency: "Daily",
        best_performing_times: ["6-8am", "12-1pm", "5-7pm"]
      },
      engagement_insights: {
        best_performing_topics: ["HIIT workouts", "Meal prep"],
        avg_engagement_rate: 0.045,
        total_videos_analyzed: 5
      }
    },
    analyzed_videos: [...]
  }
}

2. Implementation Details

Backend Service: OnboardingAnalysisService

Key features:

• Use VideoAnalyzer.analyze_video() directly (HTTP path, ~30-90s per video)
• Parallelize up to 3 videos at once (balance speed vs resource usage)
• Stream events as each video completes (progressive disclosure)
• Reuse existing TopicInferenceService with in-memory analyses
• Generate rich "Content DNA" metadata for final reveal
• Cache results in GCS for scraper reuse

3. Content DNA Generation

Generate comprehensive summary including:

• Primary themes: Top 5 themes extracted from video analyses
• Target audience: Inferred from content + engagement patterns
• Content style: Presentation, energy, format analysis
• Unique angle: What makes their content distinctive (LLM-powered)
• Posting patterns: Frequency, best performing times
• Engagement insights: Best topics, avg engagement rate

Frontend Changes

1. Premium Streaming UI

StreamingAnalysisView.tsx - Replace current polling UI

Visual Components:

Progress Header:

• Animated progress bar with gradient
• Phase indicator with smooth transitions
• Video count with animated number changes
• Estimated time remaining

Current Video Card:

• Large thumbnail (16:9 aspect) with shimmer loading
• Video title with text animation entrance
• Platform badge with icon
• Pulsing "analyzing" glow effect
• Particle effects around active video

Completed Videos Grid:

• 2x3 grid (mobile: 2x2)
• Cards fill in with stagger animation
• Each card shows:
• Thumbnail with green checkmark overlay
• Key insights (3-4 bullets, animated in sequence)
• Theme tags (pill-shaped, colored by category)
• Satisfaction pulse animation on complete

Content DNA Reveal (Maximum Wow Factor):

• Hero entrance: Fade up from center with scale animation
• Account type badge: Large, colorful, with confidence ring animation
• Content pillars: Staggered card entrance with glow effect
• DNA Breakdown sections:
• Animated tag cloud for themes
• Persona card for target audience (illustrated icon)
• Content style with typography emphasis
• Unique angle in highlighted callout box
• Mini engagement charts with animated bars
• Confetti explosion when reveal completes
• Haptic feedback on mobile (medium impact)
• Sound effect (optional, tasteful "success" chime)
• Analyzed videos carousel at bottom
• "Continue to Goal Setup" CTA with gradient + pulse

Advanced Animations:

• Smooth transitions between all phases
• Micro-interactions on hover/tap
• Loading states use skeleton screens with shimmer
• Progressive disclosure (elements appear as data arrives)
• Particle effects for video completions
• Confetti for final reveal
• Branded color scheme with gradients

Mobile Enhancements:

• Haptic feedback at key moments:
• Light impact when video analysis starts
• Medium impact when video completes
• Heavy impact when DNA revealed
• Optimized touch targets (min 44x44px)
• Swipeable video carousel on DNA screen
• Bottom sheet for DNA details (if needed)

2. Error Handling UX

Graceful Degradation:

• If 0 videos analyzed → Fall back to API metadata (current behavior)
• If 1-4 videos analyzed → Show partial DNA with note
• If 5+ videos analyzed → Full premium experience
• Show inline error messages for failed videos
• Continue with successful analyses
• Don't block user progression

Error States:

• Video download failed: Skip with subtle notification
• Analysis timeout: Skip after 90s with explanation
• Network disconnected: Retry SSE connection automatically
• No videos available: Skip analysis, manual entry only

Scope

In Scope

• ✅ Backend SSE streaming endpoint
• ✅ Parallel video analysis (3 concurrent)
• ✅ Content DNA generation with LLM
• ✅ Premium streaming UI with animations
• ✅ Video grid visualization
• ✅ Content DNA reveal screen
• ✅ Haptic feedback (mobile)
• ✅ Confetti animation
• ✅ Sound effects (optional, toggleable)
• ✅ Error handling and fallbacks
• ✅ GCS caching for scraper reuse
• ✅ Mobile-responsive design
• ✅ E2E testing

Out of Scope

• ❌ Real-time progress within single video (just video-level progress)
• ❌ User selection of specific videos to analyze
• ❌ Re-analyze button (user can't retry)
• ❌ Multi-language video analysis (English only)
• ❌ Video quality scoring/thresholds
• ❌ A/B testing infrastructure (can add later)

Acceptance Criteria

Backend

• [ ] POST /api/v1/onboarding/analyze-content endpoint with SSE streaming
• [ ] OnboardingAnalysisService implemented with parallel video analysis (max 3 concurrent)
• [ ] Content DNA generation with 3 additional LLM calls (style, angle, audience)
• [ ] Error handling: continue with successful analyses if some fail
• [ ] Logging with LogTopic.ONBOARDING for all phases
• [ ] Video analysis results cached in GCS (mark as "onboarding" source)
• [ ] SSE timeout set to 5 minutes
• [ ] Handles edge cases: 0 videos, < 5 videos, download failures

Frontend

• [ ] StreamingAnalysisView component with SSE connection
• [ ] Progress visualization with phase transitions
• [ ] Video grid with staggered animations
• [ ] Content DNA reveal screen with:
• [ ] Hero entrance animation
• [ ] Confetti explosion
• [ ] Haptic feedback (mobile)
• [ ] Sound effect (optional, toggleable)
• [ ] Particle effects
• [ ] Animated charts/visualizations
• [ ] Error handling UI with graceful degradation
• [ ] Mobile-responsive design (optimized for mobile-first)
• [ ] SSE reconnection on disconnect
• [ ] Integration with existing /onboarding/2 route
• [ ] "Skip" button for users who want to skip analysis

Testing

• [ ] Backend: SSE streaming with multiple concurrent connections
• [ ] Backend: Parallel video analysis (exactly 3 concurrent)
• [ ] Backend: Error handling (video failure, LLM failure, network)
• [ ] Frontend: SSE reconnection behavior
• [ ] Frontend: UI with various result states (0, 1, 3, 5 videos)
• [ ] E2E: Full onboarding flow with real video analysis
• [ ] E2E: Mobile experience with haptics
• [ ] Performance: Page load time, animation smoothness
• [ ] Cross-browser: Chrome, Safari, Firefox

Documentation

• [ ] Update socialgpt/CLAUDE.md with new onboarding flow
• [ ] Document SSE event types and data schemas
• [ ] Add sequence diagram for streaming flow
• [ ] Document animation specifications for design consistency
• [ ] Update onboarding user guide

Technical Considerations

Performance

• Video analysis time: ~30-90s per video
• Parallel limit: 3 concurrent (optimal balance)
• Total time estimate: 5 videos = ~2-3 minutes
• SSE timeout: 5 minutes max (covers edge cases)
• Animation performance: 60fps target, GPU-accelerated transforms

Resource Usage

• VideoAnalyzer: Gemini API (~$0.01-0.05 per video)
• LLM inference: 5 calls total (pillars, account type, style, angle, audience) = ~$0.20
• Total cost per user: ~$0.20-0.45 per onboarding
• Acceptable: High value for onboarding conversion
• GCP quota: Monitor Gemini API rate limits

Caching Strategy

• Cache video analysis results in GCS (same format as scraper)
• Mark analyses with source: "onboarding"
• Scraper skips videos that already have analyses
• 30-day TTL on cached analyses

Error Handling

• 0 videos analyzed → Fall back to current API metadata behavior
• 1-4 videos analyzed → Show partial DNA with note: "Based on N videos"
• 5+ videos analyzed → Full premium experience
• Individual video failure → Skip, show inline notification, continue
• LLM failure → Use fallback defaults, still show results
• Network disconnection → Auto-retry SSE connection 3x

Edge Cases

• User has < 5 videos total → Analyze all available
• User has 0 videos → Skip analysis entirely, manual entry
• Video download fails → Skip that video, continue with others
• Video analysis times out → Skip after 90s, continue
• All videos fail → Fall back to current behavior
• User refreshes page mid-analysis → Restart analysis (analyses cached)

Implementation Phases

Phase 1: Backend SSE Infrastructure (2-3 days)

• Implement SSE endpoint with event streaming
• Add OnboardingAnalysisService
• Parallel video analysis orchestration (3 concurrent)
• Basic error handling

Phase 2: Content DNA Generation (3-4 days)

• Implement DNA analysis logic
• Add 3 LLM calls (style, angle, audience)
• Theme aggregation and engagement insights
• Test with various content types

Phase 3: Frontend Streaming UI (4-5 days)

• Build StreamingAnalysisView component
• SSE connection and state management
• Progress visualization
• Video grid with animations
• Basic DNA reveal screen

Phase 4: Premium Polish & Effects (3-4 days)

• Advanced animations and transitions
• Confetti, particles, haptics
• Sound effects (optional, toggleable)
• Mobile optimization
• Cross-browser testing

Phase 5: Testing & Launch (2-3 days)

• E2E testing (happy path + error cases)
• Performance optimization
• Mobile device testing
• Documentation
• Soft launch with monitoring

Total estimate: 14-19 days (3-4 weeks)

Team size: 1 backend engineer + 1 frontend engineer (parallel work)

Success Metrics

Primary:

• ✅ Onboarding completion rate (target: +15% vs current)
• ✅ Time to complete onboarding step 2 (target: <5 minutes)
• ✅ User satisfaction with content recommendations (survey)

Secondary:

• ✅ % of users who edit inferred pillars (target: <30%)
• ✅ Video analysis success rate (target: >90%)
• ✅ Average videos analyzed per user (target: 4-5)
• ✅ SSE connection stability (target: <5% disconnects)

Cost:

• ✅ Average cost per onboarding (target: <$0.50)
• ✅ Gemini API quota usage (monitor for scaling)

Related Issues

• Scraper optimization for new users
• Video analysis caching strategy
• Content pillar quality improvements
• Onboarding A/B testing framework