feat(auto-blocker): Track novel vs known detections
Summary
The auto-blocker catches IPs attacking our infrastructure in real-time, but we don't currently track whether each detection is novel (first-catch) or known (already in threat feeds).
This is a key metric for:
- Proving original observation (legal/ethical distinction from derivative data)
- Measuring first-catch rate
- Demonstrating value to feed consumers
- Bragging rights
Current Behavior
Auto-blocker catches IP → blocks → reports to AbuseIPDB → indexes
No tracking of whether the IP was already known elsewhere.
Proposed Enhancement
1. Pre-block Novelty Check
Before blocking, check if IP exists in:
- [ ] AbuseIPDB (abuseScore > 0)
- [ ] VirusTotal (malicious detections > 0)
- [ ] Our IOCs index (already indexed)
- [ ] OTX (exists in pulses)
2. New Fields on Block Record
\\\`javascript
{
ip: "1.2.3.4",
// ... existing fields ...
// NEW: Novelty tracking
wasNovel: true, // true if first-catch
noveltyScore: 4, // 0-4 (how many sources we beat)
priorSources: [], // which sources already had it (if any)
firstCatchBy: "auto-blocker", // detection source
noveltyCheckedAt: "2025-12-24T23:00:00Z"
}
\\\`
3. Implementation
\\\`javascript
async function checkNovelty(ip) {
const [abuseipdb, vt, ourIOCs, otx] = await Promise.all([
checkAbuseIPDB(ip),
checkVirusTotal(ip),
checkOurIndex(ip),
checkOTX(ip)
]);
const isNovel = {
abuseipdb: abuseipdb.abuseScore === 0,
virustotal: vt.malicious === 0,
ourIOCs: !ourIOCs.exists,
otx: !otx.exists
};
const noveltyScore = Object.values(isNovel).filter(v => v).length;
const priorSources = Object.entries(isNovel)
.filter(([k, v]) => !v)
.map(([k]) => k);
return {
wasNovel: noveltyScore === 4,
noveltyScore,
priorSources,
firstCatchBy: noveltyScore === 4 ? 'auto-blocker' : null
};
}
\\\`
4. New API Endpoint
\\\`
GET /api/v1/auto-blocker/novelty-stats
Response:
{
"total": 326,
"novel": 287,
"known": 39,
"noveltyRate": "88%",
"beatAbuseIPDB": 301,
"beatVirusTotal": 312,
"beatOTX": 298
}
\\\`
5. Dashboard Integration
Add to Mirepoix:
- Novel detection rate (%)
- First-catch count (24h)
- "Beat the feeds" leaderboard
Files to Modify
- [ ]
lib/auto-blocker.js- Add novelty check before blocking - [ ]
routes/api/v1/behavioral.js- Add novelty stats endpoint - [ ]
routes/api/v1/mirepoix.js- Include novelty metrics
Success Criteria
- Every auto-blocker catch has
wasNovelfield - API endpoint returns novelty statistics
- Can prove X% of detections are original observation
Context
Christmas Eve 2025: Auto-blocker caught 326 IPs in one hour during coordinated China Telecom/Huawei attack. No way to prove how many were first-catch.
That's the gap this fixes.
---
"They attacked us first. We wrote it down. Now we can prove it."
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗