Concept · Narrated or established in the cast brief
Matching challenge
For each query image, find gallery samples from the same patient. There may be zero, one or several.
Sources: Challenge research · Chapter 1. Read the full book
Concept · Editorial explanation
Patient-level separation
Patients used for tuning must not reappear in an evaluation meant to measure new cases.
Sources: Challenge research · Chapter 1. Read the full book
Concept · Editorial explanation
Authorised data
Access and use have their own conditions; computing capacity does not grant permission to use data.
Sources: Responsibility matrix · Challenge research. Read the full book
Concept · Editorial explanation
Provenance and versions
Recording origin, transformations and version lets the team identify what ran and repeat a comparison.
Sources: Responsibility matrix · Chapter 1. Read the full book
Concept · Editorial explanation
Experiment
An implemented hypothesis is compared under known conditions; one isolated test does not establish generalisation.
Sources: Responsibility matrix. Read the full book
Concept · Editorial explanation
Validation
Comparison used to choose settings and examine errors; it is not an independent final evaluation.
Sources: Responsibility matrix · Chapter 8 · Chapter 11. Read the full book
Concept · Narrated or established in the cast brief
Mean F1 per query
The challenge metric. Scores are comparable only when measured on the same evaluation under the same rules.
Sources: Challenge research · Chapter 1 · Chapter 9. Read the full book
Concept · Editorial explanation
Agents and specifications
Tools that carry out bounded tasks from instructions and checks; their output still needs review.
Sources: Responsibility matrix · Cast brief. Read the full book
Concept · Editorial explanation
Limited resources
People's time, GPU, memory, storage and API allowance are managed separately.
Sources: Responsibility matrix · Chapter 1. Read the full book