Evaluation

An independent-data benchmark of the Akator Image Matcher's copy-detection performance — recall and precision measured on Open Images V7 at a scale of one million indexed images.

What the report covers

• Data: Open Images V7 — a public, externally verifiable corpus of natural photographs, so the evaluation is independent of any proprietary data.
• Scale: a one-million-image index (the "haystack") representing a customer database, plus a separate out-of-index pool that supplies the negative queries.
• Method: recall measured across 86 parameter configurations spanning 21 manipulation families in four classes; precision via the false-positive rate on 10,000 out-of-index queries. The system is tested black-box, through its production interface.
• Limitations: the report characterises in full where performance drops — crops beyond about 50%, collages, extreme downscaling or pixelation, and very heavy noise. Images embedded into another picture are found only in part.

Key results

Recall by manipulation, as measured in the report:

• Colour, compression and watermarks: peak recall of 100%.
• Rotation: 96–99% from 10° to 45°.
• Crops: 89–100% up to about 40% cropped.
• Added borders and padding: 74–96% for 5–25% added area.
• Memes: 95%. Strong perspective changes: 97%.
• Images embedded into another picture: 3–77%, depending on how much of the picture they cover. Collages remain hard (2×2 grid: 18%).

All results are conditional on the data distribution and manipulation parameters defined in the report; they are not guarantees for inputs outside those conditions.