Implement Routing Policy Engine for Model Selection Optimization

Priority

MEDIUM

Context

The Routing Policy Engine determines whether to use on-device or cloud models based on device capabilities, cost, and privacy. The iOS SDK has a RoutingPolicy system. The Kotlin SDK is missing this optimization layer.

Current State

• Status: Infrastructure does not exist
• Impact: No intelligent routing between on-device and cloud
• iOS Has: Complete routing policy engine

iOS Reference Implementation

• Service: sdk/runanywhere-swift/Sources/RunAnywhere/Core/RoutingPolicy.swift

Implementation Plan

1. Define Common Interfaces (commonMain)

// commonMain/kotlin/com/runanywhere/sdk/core/routing/RoutingPolicy.kt
interface RoutingPolicy {
    suspend fun selectExecutionTarget(
        modelId: String,
        context: RoutingContext
    ): ExecutionTarget
}

data class RoutingContext(
    val deviceCapability: ResourceCapability,
    val privacyLevel: PrivacyLevel,
    val costSensitivity: CostSensitivity,
    val latencyRequirement: LatencyRequirement
)

enum class ExecutionTarget {
    ON_DEVICE,
    CLOUD,
    HYBRID
}

enum class PrivacyLevel {
    PUBLIC, PRIVATE, CONFIDENTIAL
}

enum class CostSensitivity {
    LOW, MEDIUM, HIGH
}

enum class LatencyRequirement {
    REALTIME, INTERACTIVE, BATCH
}

class DefaultRoutingPolicy : RoutingPolicy {
    
    override suspend fun selectExecutionTarget(
        modelId: String,
        context: RoutingContext
    ): ExecutionTarget {
        // Privacy-first: always prefer on-device for confidential data
        if (context.privacyLevel == PrivacyLevel.CONFIDENTIAL) {
            return ExecutionTarget.ON_DEVICE
        }
        
        // Check device capability
        val canRunOnDevice = context.deviceCapability.availableMemoryMB > 500 &&
                            context.deviceCapability.thermalState != ThermalState.CRITICAL
        
        if (!canRunOnDevice) {
            return ExecutionTarget.CLOUD
        }
        
        // Cost optimization
        if (context.costSensitivity == CostSensitivity.HIGH) {
            return ExecutionTarget.ON_DEVICE
        }
        
        // Latency requirements
        return when (context.latencyRequirement) {
            LatencyRequirement.REALTIME -> ExecutionTarget.ON_DEVICE
            LatencyRequirement.INTERACTIVE -> ExecutionTarget.HYBRID
            LatencyRequirement.BATCH -> ExecutionTarget.CLOUD
        }
    }
}

Files to Create/Modify

New Files (commonMain)

• [ ] core/routing/RoutingPolicy.kt
• [ ] core/routing/DefaultRoutingPolicy.kt
• [ ] core/routing/models/RoutingContext.kt
• [ ] core/routing/models/ExecutionTarget.kt

Success Criteria

• [ ] Policy selects optimal execution target
• [ ] Privacy-first routing works
• [ ] Cost optimization is effective
• [ ] Integrated into components

Estimated Effort

1 week

🤖 Generated with Claude Code