Implement Advanced VAD Strategies for Improved Speech Detection

Priority

HIGH

Context

The iOS SDK has multiple VAD strategies (energy-based, ML-based, hybrid). The Kotlin SDK only has basic VAD implementation. Advanced strategies improve detection accuracy.

Current State

• Status: Only basic VAD exists
• Impact: Reduced speech detection accuracy
• iOS Has: Multiple VAD strategies with strategy selection

iOS Reference Implementation

• Strategies: sdk/runanywhere-swift/Sources/RunAnywhere/Components/VAD/Strategies/

iOS Architecture Pattern

public protocol VADStrategy: Sendable {
    func detect(audioData: Data) async -> VADResult
}

public final class EnergyBasedVAD: VADStrategy { ... }
public final class MLBasedVAD: VADStrategy { ... }
public final class HybridVAD: VADStrategy { ... }

Implementation Plan

1. Define Strategy Interfaces (commonMain)

// commonMain/kotlin/com/runanywhere/sdk/components/vad/strategies/VADStrategy.kt
interface VADStrategy {
    suspend fun detect(audioData: ByteArray): VADResult
    val strategyType: StrategyType
}

data class VADResult(
    val isSpeech: Boolean,
    val confidence: Float,
    val energyLevel: Float? = null
)

enum class StrategyType {
    ENERGY_BASED,
    ML_BASED,
    HYBRID
}

// commonMain/kotlin/com/runanywhere/sdk/components/vad/strategies/EnergyBasedVAD.kt
class EnergyBasedVAD(
    private val threshold: Float = 0.03f
) : VADStrategy {
    
    override suspend fun detect(audioData: ByteArray): VADResult {
        val energy = calculateEnergy(audioData)
        val isSpeech = energy > threshold
        
        return VADResult(
            isSpeech = isSpeech,
            confidence = if (isSpeech) minOf(energy / threshold, 1.0f) else 0.0f,
            energyLevel = energy
        )
    }
    
    override val strategyType = StrategyType.ENERGY_BASED
    
    private fun calculateEnergy(audioData: ByteArray): Float {
        var sum = 0.0
        for (i in audioData.indices step 2) {
            val sample = ((audioData[i+1].toInt() shl 8) or (audioData[i].toInt() and 0xFF)).toShort()
            sum += sample * sample
        }
        return (sum / (audioData.size / 2)).toFloat()
    }
}

// commonMain/kotlin/com/runanywhere/sdk/components/vad/strategies/HybridVAD.kt
class HybridVAD(
    private val energyStrategy: EnergyBasedVAD,
    private val mlStrategy: VADStrategy?
) : VADStrategy {
    
    override suspend fun detect(audioData: ByteArray): VADResult {
        val energyResult = energyStrategy.detect(audioData)
        
        // If energy is too low, no need for ML
        if (!energyResult.isSpeech) {
            return energyResult
        }
        
        // Use ML for refinement
        val mlResult = mlStrategy?.detect(audioData)
        
        return if (mlResult != null) {
            VADResult(
                isSpeech = mlResult.isSpeech,
                confidence = (energyResult.confidence + mlResult.confidence) / 2,
                energyLevel = energyResult.energyLevel
            )
        } else {
            energyResult
        }
    }
    
    override val strategyType = StrategyType.HYBRID
}

Files to Create/Modify

New Files (commonMain)

• [ ] components/vad/strategies/VADStrategy.kt
• [ ] components/vad/strategies/EnergyBasedVAD.kt
• [ ] components/vad/strategies/HybridVAD.kt
• [ ] components/vad/models/VADResult.kt

Files to Modify

• [ ] components/vad/VADComponent.kt (use strategies)

Testing Requirements

• [ ] Unit tests for each strategy
• [ ] Accuracy tests with real audio
• [ ] Performance benchmarks
• [ ] Strategy switching tests

Success Criteria

• [ ] Energy-based VAD works correctly
• [ ] Hybrid VAD improves accuracy
• [ ] Strategy selection is configurable
• [ ] Performance is acceptable

Estimated Effort

5-7 days

Dependencies

• Depends on: VADComponent (exists)

🤖 Generated with Claude Code