DatasetConfig#
Note
All Configs are derived from rastervision.pipeline.config.Config
, which itself is a pydantic Model.
- pydantic model DatasetConfig[source]#
Configure train, validation, and test splits for a dataset.
Show JSON schema
{ "title": "DatasetConfig", "description": "Configure train, validation, and test splits for a dataset.", "type": "object", "properties": { "class_config": { "$ref": "#/definitions/ClassConfig" }, "train_scenes": { "title": "Train Scenes", "type": "array", "items": { "$ref": "#/definitions/SceneConfig" } }, "validation_scenes": { "title": "Validation Scenes", "type": "array", "items": { "$ref": "#/definitions/SceneConfig" } }, "test_scenes": { "title": "Test Scenes", "default": [], "type": "array", "items": { "$ref": "#/definitions/SceneConfig" } }, "scene_groups": { "title": "Scene Groups", "description": "Groupings of scenes. Should be a dict of the form: {<group-name>: Set(scene_id_1, scene_id_2, ...)}. Three groups are added by default: \"train_scenes\", \"validation_scenes\", and \"test_scenes\"", "default": {}, "type": "object", "additionalProperties": { "type": "array", "items": { "type": "string" }, "uniqueItems": true } }, "type_hint": { "title": "Type Hint", "default": "dataset", "enum": [ "dataset" ], "type": "string" } }, "required": [ "class_config", "train_scenes", "validation_scenes" ], "additionalProperties": false, "definitions": { "ClassConfig": { "title": "ClassConfig", "description": "Configure class information for a machine learning task.", "type": "object", "properties": { "names": { "title": "Names", "description": "Names of classes. The i-th class in this list will have class ID = i.", "type": "array", "items": { "type": "string" } }, "colors": { "title": "Colors", "description": "Colors used to visualize classes. Can be color strings accepted by matplotlib or RGB tuples. If None, a random color will be auto-generated for each class.", "type": "array", "items": { "anyOf": [ { "type": "string" }, { "type": "array", "items": {} } ] } }, "null_class": { "title": "Null Class", "description": "Optional name of class in `names` to use as the null class. This is used in semantic segmentation to represent the label for imagery pixels that are NODATA or that are missing a label. If None and the class names include \"null\", it will automatically be used as the null class. If None, and this Config is part of a SemanticSegmentationConfig, a null class will be added automatically.", "type": "string" }, "type_hint": { "title": "Type Hint", "default": "class_config", "enum": [ "class_config" ], "type": "string" } }, "required": [ "names" ], "additionalProperties": false }, "RasterTransformerConfig": { "title": "RasterTransformerConfig", "description": "Configure a :class:`.RasterTransformer`.", "type": "object", "properties": { "type_hint": { "title": "Type Hint", "default": "raster_transformer", "enum": [ "raster_transformer" ], "type": "string" } }, "additionalProperties": false }, "RasterSourceConfig": { "title": "RasterSourceConfig", "description": "Configure a :class:`.RasterSource`.", "type": "object", "properties": { "channel_order": { "title": "Channel Order", "description": "The sequence of channel indices to use when reading imagery.", "type": "array", "items": { "type": "integer" } }, "transformers": { "title": "Transformers", "default": [], "type": "array", "items": { "$ref": "#/definitions/RasterTransformerConfig" } }, "bbox": { "title": "Bbox", "description": "User-specified bbox in pixel coords in the form (ymin, xmin, ymax, xmax). Useful for cropping the raster source so that only part of the raster is read from.", "type": "array", "minItems": 4, "maxItems": 4, "items": [ { "type": "integer" }, { "type": "integer" }, { "type": "integer" }, { "type": "integer" } ] }, "type_hint": { "title": "Type Hint", "default": "raster_source", "enum": [ "raster_source" ], "type": "string" } }, "additionalProperties": false }, "LabelSourceConfig": { "title": "LabelSourceConfig", "description": "Configure a :class:`.LabelSource`.", "type": "object", "properties": { "type_hint": { "title": "Type Hint", "default": "label_source", "enum": [ "label_source" ], "type": "string" } }, "additionalProperties": false }, "LabelStoreConfig": { "title": "LabelStoreConfig", "description": "Configure a :class:`.LabelStore`.", "type": "object", "properties": { "type_hint": { "title": "Type Hint", "default": "label_store", "enum": [ "label_store" ], "type": "string" } }, "additionalProperties": false }, "SceneConfig": { "title": "SceneConfig", "description": "Configure a :class:`.Scene` comprising raster data & labels for an AOI.\n ", "type": "object", "properties": { "id": { "title": "Id", "type": "string" }, "raster_source": { "$ref": "#/definitions/RasterSourceConfig" }, "label_source": { "$ref": "#/definitions/LabelSourceConfig" }, "label_store": { "$ref": "#/definitions/LabelStoreConfig" }, "aoi_uris": { "title": "Aoi Uris", "description": "List of URIs of GeoJSON files that define the AOIs for the scene. Each polygon defines an AOI which is a piece of the scene that is assumed to be fully labeled and usable for training or validation. The AOIs are assumed to be in EPSG:4326 coordinates.", "type": "array", "items": { "type": "string" } }, "type_hint": { "title": "Type Hint", "default": "scene", "enum": [ "scene" ], "type": "string" } }, "required": [ "id", "raster_source" ], "additionalProperties": false } } }
- Config
extra: str = forbid
validate_assignment: bool = True
- Fields
- field class_config: ClassConfig [Required]#
- field scene_groups: Dict[str, Set[str]] = {}#
Groupings of scenes. Should be a dict of the form: {<group-name>: Set(scene_id_1, scene_id_2, …)}. Three groups are added by default: “train_scenes”, “validation_scenes”, and “test_scenes”
- field test_scenes: List[SceneConfig] = []#
- field train_scenes: List[SceneConfig] [Required]#
- field validation_scenes: List[SceneConfig] [Required]#
- build()#
Build an instance of the corresponding type of object using this config.
For example, BackendConfig will build a Backend object. The arguments to this method will vary depending on the type of Config.
- classmethod deserialize(inp: str | dict | Config) Self #
Deserialize Config from a JSON file or dict, upgrading if possible.
If
inp
is already aConfig
, it is returned as is.
- dict(with_rv_metadata: bool = False, **kwargs) dict #
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
- classmethod from_dict(cfg_dict: dict) Self #
Deserialize Config from a dict.
- Parameters
cfg_dict (dict) – Dict to deserialize.
- Return type
Self
- classmethod from_file(uri: str) Self #
Deserialize Config from a JSON file, upgrading if possible.
- Parameters
uri (str) – URI to load from.
- Return type
Self
- recursive_validate_config()#
Recursively validate hierarchies of Configs.
This uses reflection to call validate_config on a hierarchy of Configs using a depth-first pre-order traversal.
- revalidate()#
Re-validate an instantiated Config.
Runs all Pydantic validators plus self.validate_config().
Adapted from: https://github.com/samuelcolvin/pydantic/issues/1864#issuecomment-679044432
- to_file(uri: str, with_rv_metadata: bool = True) None #
Save a Config to a JSON file, optionally with RV metadata.
- update(pipeline=None)[source]#
Update any fields before validation.
Subclasses should override this to provide complex default behavior, for example, setting default values as a function of the values of other fields. The arguments to this method will vary depending on the type of Config.
- validate_config()[source]#
Validate fields that should be checked after update is called.
This is to complement the builtin validation that Pydantic performs at the time of object construction.
- validate_list(field: str, valid_options: List[str])#
Validate a list field.
- Parameters
- Raises
ConfigError – if field is invalid
- property all_scenes: List[SceneConfig]#