DatasetConfig#

Note

All Configs are derived from rastervision.pipeline.config.Config, which itself is a pydantic Model.

pydantic model DatasetConfig[source]#

Configure train, validation, and test splits for a dataset.

Show JSON schema

{
   "title": "DatasetConfig",
   "description": "Configure train, validation, and test splits for a dataset.",
   "type": "object",
   "properties": {
      "class_config": {
         "$ref": "#/$defs/ClassConfig"
      },
      "train_scenes": {
         "items": {
            "$ref": "#/$defs/SceneConfig"
         },
         "title": "Train Scenes",
         "type": "array"
      },
      "validation_scenes": {
         "items": {
            "$ref": "#/$defs/SceneConfig"
         },
         "title": "Validation Scenes",
         "type": "array"
      },
      "test_scenes": {
         "default": [],
         "items": {
            "$ref": "#/$defs/SceneConfig"
         },
         "title": "Test Scenes",
         "type": "array"
      },
      "scene_groups": {
         "additionalProperties": {
            "items": {
               "type": "string"
            },
            "type": "array",
            "uniqueItems": true
         },
         "default": {},
         "description": "Groupings of scenes. Should be a dict of the form: {<group-name>: set(scene_id_1, scene_id_2, ...)}. Three groups are added by default: \"train_scenes\", \"validation_scenes\", and \"test_scenes\"",
         "title": "Scene Groups",
         "type": "object"
      },
      "type_hint": {
         "const": "dataset",
         "default": "dataset",
         "enum": [
            "dataset"
         ],
         "title": "Type Hint",
         "type": "string"
      }
   },
   "$defs": {
      "ClassConfig": {
         "additionalProperties": false,
         "description": "Configure class information for a machine learning task.",
         "properties": {
            "names": {
               "description": "Names of classes. The i-th class in this list will have class ID = i.",
               "items": {
                  "type": "string"
               },
               "title": "Names",
               "type": "array"
            },
            "colors": {
               "anyOf": [
                  {
                     "items": {
                        "anyOf": [
                           {
                              "type": "string"
                           },
                           {
                              "items": {},
                              "type": "array"
                           }
                        ]
                     },
                     "type": "array"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "description": "Colors used to visualize classes. Can be color strings accepted by matplotlib or RGB tuples. If None, a random color will be auto-generated for each class.",
               "title": "Colors"
            },
            "null_class": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "description": "Optional name of class in `names` to use as the null class. This is used in semantic segmentation to represent the label for imagery pixels that are NODATA or that are missing a label. If None and the class names include \"null\", it will automatically be used as the null class. If None, and this Config is part of a SemanticSegmentationConfig, a null class will be added automatically.",
               "title": "Null Class"
            },
            "type_hint": {
               "const": "class_config",
               "default": "class_config",
               "enum": [
                  "class_config"
               ],
               "title": "Type Hint",
               "type": "string"
            }
         },
         "required": [
            "names"
         ],
         "title": "ClassConfig",
         "type": "object"
      },
      "LabelSourceConfig": {
         "additionalProperties": false,
         "description": "Configure a :class:`.LabelSource`.",
         "properties": {
            "type_hint": {
               "const": "label_source",
               "default": "label_source",
               "enum": [
                  "label_source"
               ],
               "title": "Type Hint",
               "type": "string"
            }
         },
         "title": "LabelSourceConfig",
         "type": "object"
      },
      "LabelStoreConfig": {
         "additionalProperties": false,
         "description": "Configure a :class:`.LabelStore`.",
         "properties": {
            "type_hint": {
               "const": "label_store",
               "default": "label_store",
               "enum": [
                  "label_store"
               ],
               "title": "Type Hint",
               "type": "string"
            }
         },
         "title": "LabelStoreConfig",
         "type": "object"
      },
      "RasterSourceConfig": {
         "additionalProperties": false,
         "description": "Configure a :class:`.RasterSource`.",
         "properties": {
            "channel_order": {
               "anyOf": [
                  {
                     "items": {
                        "type": "integer"
                     },
                     "type": "array"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "description": "The sequence of channel indices to use when reading imagery.",
               "title": "Channel Order"
            },
            "transformers": {
               "default": [],
               "items": {
                  "$ref": "#/$defs/RasterTransformerConfig"
               },
               "title": "Transformers",
               "type": "array"
            },
            "bbox": {
               "anyOf": [
                  {
                     "maxItems": 4,
                     "minItems": 4,
                     "prefixItems": [
                        {
                           "type": "integer"
                        },
                        {
                           "type": "integer"
                        },
                        {
                           "type": "integer"
                        },
                        {
                           "type": "integer"
                        }
                     ],
                     "type": "array"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "description": "User-specified bbox in pixel coords in the form (ymin, xmin, ymax, xmax). Useful for cropping the raster source so that only part of the raster is read from.",
               "title": "Bbox"
            },
            "type_hint": {
               "const": "raster_source",
               "default": "raster_source",
               "enum": [
                  "raster_source"
               ],
               "title": "Type Hint",
               "type": "string"
            }
         },
         "title": "RasterSourceConfig",
         "type": "object"
      },
      "RasterTransformerConfig": {
         "additionalProperties": false,
         "description": "Configure a :class:`.RasterTransformer`.",
         "properties": {
            "type_hint": {
               "const": "raster_transformer",
               "default": "raster_transformer",
               "enum": [
                  "raster_transformer"
               ],
               "title": "Type Hint",
               "type": "string"
            }
         },
         "title": "RasterTransformerConfig",
         "type": "object"
      },
      "SceneConfig": {
         "additionalProperties": false,
         "description": "Configure a :class:`.Scene` comprising raster data & labels for an AOI.\n    ",
         "properties": {
            "id": {
               "title": "Id",
               "type": "string"
            },
            "raster_source": {
               "$ref": "#/$defs/RasterSourceConfig"
            },
            "label_source": {
               "anyOf": [
                  {
                     "$ref": "#/$defs/LabelSourceConfig"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null
            },
            "label_store": {
               "anyOf": [
                  {
                     "$ref": "#/$defs/LabelStoreConfig"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null
            },
            "aoi_uris": {
               "anyOf": [
                  {
                     "items": {
                        "type": "string"
                     },
                     "type": "array"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "description": "List of URIs of GeoJSON files that define the AOIs for the scene. Each polygon defines an AOI which is a piece of the scene that is assumed to be fully labeled and usable for training or validation. The AOIs are assumed to be in EPSG:4326 coordinates.",
               "title": "Aoi Uris"
            },
            "type_hint": {
               "const": "scene",
               "default": "scene",
               "enum": [
                  "scene"
               ],
               "title": "Type Hint",
               "type": "string"
            }
         },
         "required": [
            "id",
            "raster_source"
         ],
         "title": "SceneConfig",
         "type": "object"
      }
   },
   "additionalProperties": false,
   "required": [
      "class_config",
      "train_scenes",
      "validation_scenes"
   ]
}

Config:

extra: str = forbid
validate_assignment: bool = True

Fields:

class_config (rastervision.core.data.class_config.ClassConfig)
scene_groups (dict[str, set[str]])
test_scenes (list[rastervision.core.data.scene_config.SceneConfig])
train_scenes (list[rastervision.core.data.scene_config.SceneConfig])
type_hint (Literal['dataset'])
validation_scenes (list[rastervision.core.data.scene_config.SceneConfig])

field class_config: ClassConfig [Required]#

field scene_groups: dict[str, set[str]] = {}#: Groupings of scenes. Should be a dict of the form: {<group-name>: set(scene_id_1, scene_id_2, …)}. Three groups are added by default: “train_scenes”, “validation_scenes”, and “test_scenes”

field test_scenes: list[SceneConfig] = []#

field train_scenes: list[SceneConfig] [Required]#

field type_hint: Literal['dataset'] = 'dataset'#

field validation_scenes: list[SceneConfig] [Required]#

build()#

Build an instance of the corresponding type of object using this config.

For example, BackendConfig will build a Backend object. The arguments to this method will vary depending on the type of Config.

classmethod deserialize(inp: str | dict | Config) → Self#

Deserialize Config from a JSON file or dict, upgrading if possible.

If inp is already a Config, it is returned as is.

Parameters:: inp (str | dict | Config) – a URI to a JSON file or a dict.
Return type:: Self

classmethod from_dict(cfg_dict: dict) → Self#

Deserialize Config from a dict.

Parameters:: cfg_dict (dict) – Dict to deserialize.
Return type:: Self

classmethod from_file(uri: str) → Self#

Deserialize Config from a JSON file, upgrading if possible.

Parameters:: uri (str) – URI to load from.
Return type:: Self

get_split_config(split_ind, num_splits)[source]#

recursive_validate_config()#

Recursively validate hierarchies of Configs.

This uses reflection to call validate_config on a hierarchy of Configs using a depth-first pre-order traversal.

revalidate()#

Re-validate an instantiated Config.

Runs all Pydantic validators plus self.validate_config().

to_file(uri: str, with_rv_metadata: bool = True) → None#

Save a Config to a JSON file, optionally with RV metadata.

Parameters:

uri (str) – URI to save to.
with_rv_metadata (bool) – If True, inject Raster Vision metadata such as plugin_versions, so that the config can be upgraded when loaded.

Return type:

None

update(pipeline=None)[source]#

Update any fields before validation.

Subclasses should override this to provide complex default behavior, for example, setting default values as a function of the values of other fields. The arguments to this method will vary depending on the type of Config.

validate_config()[source]#

Validate fields that should be checked after update is called.

This is to complement the builtin validation that Pydantic performs at the time of object construction.

validate_list(field: str, valid_options: list[str])#

Validate a list field.

Parameters:

field (str) – name of field to validate
valid_options (list[str]) – values that field is allowed to take

Raises:

ConfigError – if field is invalid

property all_scenes: list[rastervision.core.data.scene_config.SceneConfig]#