Evaluator#

class Evaluator[source]#

Bases: ABC

Evaluates predictions for a set of scenes.

Methods

`__init__`()
`evaluate_predictions`(ground_truth, predictions)	Evaluate predictions against ground truth.
`evaluate_scene`(scene)	Evaluate predictions from a scene's labels store.
`process`(scenes)	Evaluate all given scenes and save the evaluations.

abstract evaluate_predictions(ground_truth: Labels, predictions: Labels) → Any[source]#

Evaluate predictions against ground truth.

Parameters:

Returns:

The evaluation.

Return type:

Any

abstract evaluate_scene(scene: Scene) → Any[source]#

Evaluate predictions from a scene’s labels store.

The predictions are evalated against ground truth labels from the scene’s label source.

Parameters:: scene (Scene) – A scene with a label source and a label store.
Returns:: The evaluation.
Return type:: ClassificationEvaluation

abstract process(scenes: Iterable[Scene]) → None[source]#

Evaluate all given scenes and save the evaluations.