Skip to content

DARPA D2 Qualifier

AI/Computer Vision

From a single photo of a casualty, calls each of the four limbs no injury, wound, amputation, or not testable. Runs offline on one GPU and scored 85.9% on photos it had never seen, against a 78% target.

Computer Vision
PyTorch
DINOv2
RF-DETR
Python
Docker
Medical Imaging
DARPA D2 Qualifier

Key Features

  • Labels all four limbs of a casualty from one photo as no injury, wound, amputation, or not testable
  • 85.9% site accuracy on 212 held-back photos it had never seen, against DARPA's 78% target
  • Cross-validated accuracy climbed from 71.3% to 85.8% over 118 logged experiments
  • Runs offline in one Docker image on one GPU, at 9.8 GB peak memory and 2.3 s for the hardest photo
  • Every trained part is a small head on frozen DINOv2-L tokens, cached once per photo, so most experiments ran in 25 to 40 minutes
  • A fine-tuned RF-DETR wound detector, checked in a paired test, cut wounds missed as no injury from 65 to 55

This is Grayhaven’s entry for DARPA’s D2 qualifier. The task sounds simple. You get one photo of a casualty, and for each arm and leg you say whether it’s fine, wounded, amputated, or not visible. The score is plain accuracy over limbs, with no partial credit.

The rules made it harder. Everything had to run offline in one container, on one GPU with less than 16 GB, inside an hour. Only models with clean licences were allowed, so the shipped system uses DINOv2 and nothing else as its backbone.

How it works

  1. A person detector finds the casualty. It looks at the photo turned four ways, because casualties lie at every angle.
  2. A frozen DINOv2-L encodes the crop once. Every trained part is a small head on those cached tokens.
  3. A localizer finds each limb, and two heads classify each limb close up.
  4. An injury probe and a fine-tuned RF-DETR detector look for wounds inside each limb’s own box. This catches wounds the limb heads call fine, like blood on camouflage trousers.

How it was measured

I developed on 740 photos of 46 casualties and kept 16 casualties, 212 photos, locked away the whole time. After every choice was made, the system scored 85.9% on those photos, close to the 85.8% it scored in cross-validation. DARPA’s target was 78%.

The biggest single gain came from classifying each limb close up, worth about 10 points. Most later gains were small, and their confidence intervals often crossed zero. The full report says so, table by table.

Where it’s weak

Wounds. On the locked-away photos it found only 40% of them, and most of the misses were called no injury, especially leg wounds on clothing. The wound detector was added for exactly this, and a second, paired test cut those misses from 65 to 55.

The submitted model was then retrained on all 62 casualties, so it has no held-out score of its own. The 85.9% belongs to the version just before that retrain.