ClassificationEvaluation#

class ClassificationEvaluation[source]#

Bases: ABC

Base class for representing prediction evaluations.

Evaluations can be keyed, for instance, if evaluations happen per class.

class_to_eval_item#

Mapping from class IDs to ``ClassEvaluationItem``s.

Type

Dict[int, ClassEvaluationItem]

scene_to_eval#

Mapping from scene IDs to ``ClassificationEvaluation``s.

Type

Dict[str, ClassificationEvaluation]

avg_item#

Averaged evaluation over all classes.

Type

Optional[Dict[str, Any]]

conf_mat#

Confusion matrix.

Type

Optional[np.ndarray]

__init__()[source]#

Methods

__init__()

compute(ground_truth_labels, prediction_labels)

Compute metrics for a single scene.

compute_avg()

Compute average metrics over all classes.

merge(other[, scene_id])

Merge Evaluation for another Scene into this one.

reset()

Reset the Evaluation.

save(output_uri)

Save this Evaluation to a file.

to_json()

Serialize to a dict or list.

__init__()[source]#
abstract compute(ground_truth_labels, prediction_labels)[source]#

Compute metrics for a single scene.

Parameters
  • ground_truth_labels – Ground Truth labels to evaluate against.

  • prediction_labels – The predicted labels to evaluate.

compute_avg() None[source]#

Compute average metrics over all classes.

Return type

None

merge(other: ClassificationEvaluation, scene_id: Optional[str] = None) None[source]#

Merge Evaluation for another Scene into this one.

This is useful for computing the average metrics of a set of scenes. The results of the averaging are stored in this Evaluation.

Parameters
  • other (ClassificationEvaluation) – Evaluation to merge into this one

  • scene_id (Optional[str], optional) – ID of scene. If specified, (a copy of) other will be saved and be available in to_json()’s output. Defaults to None.

Return type

None

reset()[source]#

Reset the Evaluation.

save(output_uri: str) None[source]#

Save this Evaluation to a file.

Parameters

output_uri (str) – string URI for the file to write.

Return type

None

to_json() Union[dict, list][source]#

Serialize to a dict or list.

Returns

Class-wise and (if available) scene-wise evaluations.

Return type

Union[dict, list]