Welcome

This tool will guide you through a structured evaluation of lens models. It consists of three rounds:

1.  A binary classification of successful and failed lens models
2.  A detailed characterisation of each failed model
3.  A short evaluation of which panels are most useful

Test version

This version includes two extra steps compared to future inspections: a detailed failure characterisation (round 2) and a short evaluation of which panels are actually useful (round 3). It will take more time than usual, but the results will help us streamline the process before moving on to the full DR1 sample.

You will evaluate 100 shared objects alongside 100 additional objects. The shared set lets us compare results across participants and check for consistency. The additional objects give broader coverage of the sample.

Progress is saved automatically. You can close this page at any time and resume where you left off by logging in with your participant ID.

Enter your participant ID to begin.

Round 1 of 3

Lens model classification

Classify each lens model as Successful or Failed based on the object cutouts and modeling output panel. Refer to the Criteria button at any time for the full evaluation criteria. You can press Back to revise a previous answer. Failures will be characterised in detail in Round 2.

0 / 0
Object Cutouts
Loading...
Modeling Output
Loading...
Segmentation Maps
Loading...
Round 2 of 3

Failure characterisation

This round covers every object you marked as Failed. For each, select which panel or panels contain the problem and provide a description of what you observe.


For each object you will see the object cutouts, the modeling output, and the segmentation maps. Refer to the Criteria button for examples of common failure modes.


These descriptions will be used to optimize both the modeling and segmentation pipelines.

0 / 0
Object Cutouts
Loading...
Modeling Output
Loading...
Segmentation Maps
Loading...
Modeling issue
Cutout issue
Segmentation issue
Round 3 of 3 Final

Panel usefulness evaluation

This round evaluates a small subset of objects to identify which parts of the inspection panels carry the most diagnostic signal for lens model quality assessment.


When scaling this validation to thousands of objects, knowing which panels and columns are essential allows us to reduce inspection overhead without loss of information. For each object, select the individual columns you used when forming your judgement in Round 1.


Use the notes field for any additional observations on panel usefulness.

Object Cutouts
Loading...
Modeling Output
Loading...
Segmentation Maps
Loading...

Evaluation complete

Thank you for completing all three rounds. Your results have been saved to the server. You can also download your personal CSV below.