SolverConfig#

Note

All Configs are derived from rastervision.pipeline.config.Config, which itself is a pydantic Model.

pydantic model SolverConfig[source]#

Config related to solver aka optimizer.

Show JSON schema
{
   "title": "SolverConfig",
   "description": "Config related to solver aka optimizer.",
   "type": "object",
   "properties": {
      "lr": {
         "title": "Lr",
         "description": "Learning rate.",
         "default": 0.0001,
         "exclusiveMinimum": 0,
         "type": "number"
      },
      "num_epochs": {
         "title": "Num Epochs",
         "description": "Number of epochs (ie. sweeps through the whole training set).",
         "default": 10,
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "test_num_epochs": {
         "title": "Test Num Epochs",
         "description": "Number of epochs to use in test mode.",
         "default": 2,
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "test_batch_sz": {
         "title": "Test Batch Sz",
         "description": "Batch size to use in test mode.",
         "default": 4,
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "overfit_num_steps": {
         "title": "Overfit Num Steps",
         "description": "Number of optimizer steps to use in overfit mode.",
         "default": 1,
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "sync_interval": {
         "title": "Sync Interval",
         "description": "The interval in epochs for each sync to the cloud.",
         "default": 1,
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "batch_sz": {
         "title": "Batch Sz",
         "description": "Batch size.",
         "default": 32,
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "one_cycle": {
         "title": "One Cycle",
         "description": "If True, use triangular LR scheduler with a single cycle across all epochs with start and end LR being lr/10 and the peak being lr.",
         "default": true,
         "type": "boolean"
      },
      "multi_stage": {
         "title": "Multi Stage",
         "description": "List of epoch indices at which to divide LR by 10.",
         "default": [],
         "type": "array",
         "items": {}
      },
      "class_loss_weights": {
         "title": "Class Loss Weights",
         "description": "Class weights for weighted loss.",
         "type": "array",
         "items": {
            "type": "number"
         }
      },
      "ignore_class_index": {
         "title": "Ignore Class Index",
         "description": "If specified, this index is ignored when computing the loss. See pytorch documentation for nn.CrossEntropyLoss for more details. This can also be negative, in which case it is treated as a negative slice index i.e. -1 = last index, -2 = second-last index, and so on.",
         "type": "integer"
      },
      "external_loss_def": {
         "title": "External Loss Def",
         "description": "If specified, the loss will be built from the definition from this external source, using Torch Hub.",
         "allOf": [
            {
               "$ref": "#/definitions/ExternalModuleConfig"
            }
         ]
      },
      "type_hint": {
         "title": "Type Hint",
         "default": "solver",
         "enum": [
            "solver"
         ],
         "type": "string"
      }
   },
   "additionalProperties": false,
   "definitions": {
      "ExternalModuleConfig": {
         "title": "ExternalModuleConfig",
         "description": "Config describing an object to be loaded via Torch Hub.",
         "type": "object",
         "properties": {
            "uri": {
               "title": "Uri",
               "description": "Local uri of a zip file, or local uri of a directory,or remote uri of zip file.",
               "minLength": 1,
               "type": "string"
            },
            "github_repo": {
               "title": "Github Repo",
               "description": "<repo-owner>/<repo-name>[:tag]",
               "pattern": ".+/.+",
               "type": "string"
            },
            "name": {
               "title": "Name",
               "description": "Name of the folder in which to extract/copy the definition files.",
               "minLength": 1,
               "type": "string"
            },
            "entrypoint": {
               "title": "Entrypoint",
               "description": "Name of a callable present in hubconf.py. See docs for torch.hub for details.",
               "minLength": 1,
               "type": "string"
            },
            "entrypoint_args": {
               "title": "Entrypoint Args",
               "description": "Args to pass to the entrypoint. Must be serializable.",
               "default": [],
               "type": "array",
               "items": {}
            },
            "entrypoint_kwargs": {
               "title": "Entrypoint Kwargs",
               "description": "Keyword args to pass to the entrypoint. Must be serializable.",
               "default": {},
               "type": "object"
            },
            "force_reload": {
               "title": "Force Reload",
               "description": "Force reload of module definition.",
               "default": false,
               "type": "boolean"
            },
            "type_hint": {
               "title": "Type Hint",
               "default": "external-module",
               "enum": [
                  "external-module"
               ],
               "type": "string"
            }
         },
         "required": [
            "entrypoint"
         ],
         "additionalProperties": false
      }
   }
}

Config
  • extra: str = forbid

  • validate_assignment: bool = True

Fields
field batch_sz: PositiveInt = 32#

Batch size.

Constraints
  • exclusiveMinimum = 0

Validated by
field class_loss_weights: Optional[Sequence[float]] = None#

Class weights for weighted loss.

Validated by
field external_loss_def: Optional[ExternalModuleConfig] = None#

If specified, the loss will be built from the definition from this external source, using Torch Hub.

Validated by
field ignore_class_index: Optional[int] = None#

If specified, this index is ignored when computing the loss. See pytorch documentation for nn.CrossEntropyLoss for more details. This can also be negative, in which case it is treated as a negative slice index i.e. -1 = last index, -2 = second-last index, and so on.

Validated by
field lr: PositiveFloat = 0.0001#

Learning rate.

Constraints
  • exclusiveMinimum = 0

Validated by
field multi_stage: List = []#

List of epoch indices at which to divide LR by 10.

Validated by
field num_epochs: PositiveInt = 10#

Number of epochs (ie. sweeps through the whole training set).

Constraints
  • exclusiveMinimum = 0

Validated by
field one_cycle: bool = True#

If True, use triangular LR scheduler with a single cycle across all epochs with start and end LR being lr/10 and the peak being lr.

Validated by
field overfit_num_steps: PositiveInt = 1#

Number of optimizer steps to use in overfit mode.

Constraints
  • exclusiveMinimum = 0

Validated by
field sync_interval: PositiveInt = 1#

The interval in epochs for each sync to the cloud.

Constraints
  • exclusiveMinimum = 0

Validated by
field test_batch_sz: PositiveInt = 4#

Batch size to use in test mode.

Constraints
  • exclusiveMinimum = 0

Validated by
field test_num_epochs: PositiveInt = 2#

Number of epochs to use in test mode.

Constraints
  • exclusiveMinimum = 0

Validated by
field type_hint: Literal['solver'] = 'solver'#
Validated by
build()#

Build an instance of the corresponding type of object using this config.

For example, BackendConfig will build a Backend object. The arguments to this method will vary depending on the type of Config.

build_epoch_scheduler(optimizer: torch.optim.Optimizer, last_epoch: int = -1, **kwargs) Optional[torch.optim.lr_scheduler._LRScheduler][source]#

Returns an LR scheduler tha changes the LR each epoch.

This is used to divide the learning rate by 10 at certain epochs.

Parameters
  • optimizer (optim.Optimizer) – Optimizer to build scheduler for.

  • last_epoch (int) – Last epoch. Defaults to -1.

  • **kwargs – Extra args for the scheduler constructor.

Returns

An epoch scheduler, if applicable. Otherwise, None.

Return type

Optional[torch.optim.lr_scheduler._LRScheduler]

build_loss(num_classes: int, save_dir: Optional[str] = None, hubconf_dir: Optional[str] = None) Callable[source]#

Build and return a loss function based on the config.

Parameters
  • num_classes (int) – Number of classes.

  • save_dir (Optional[str], optional) – Used for building external_loss_def if specified. Defaults to None.

  • hubconf_dir (Optional[str], optional) – Used for building external_loss_def if specified. Defaults to None.

Returns

Loss function.

Return type

Callable

build_optimizer(model: torch.nn.Module, **kwargs) torch.optim.Adam[source]#

Build and return an Adam optimizer for the given model.

Parameters
  • model (nn.Module) – Model to be trained.

  • **kwargs – Extra args for the optimizer constructor.

Returns

An Adam optimzer instance.

Return type

torch.optim.Adam

build_step_scheduler(optimizer: torch.optim.Optimizer, train_ds_sz: int, last_epoch: int = -1, **kwargs) Optional[torch.optim.lr_scheduler._LRScheduler][source]#

Returns an LR scheduler that changes the LR each step.

This is used to implement the “one cycle” schedule popularized by FastAI.

Parameters
  • optimizer (optim.Optimizer) – Optimizer to build scheduler for.

  • train_ds_sz (int) – Size of the training dataset.

  • last_epoch (int) – Last epoch. Defaults to -1.

  • **kwargs – Extra args for the scheduler constructor.

Returns

A step scheduler, if applicable. Otherwise, None.

Return type

Optional[torch.optim.lr_scheduler._LRScheduler]

validator check_no_loss_opts_if_external  »  all fields[source]#
Parameters

values (dict) –

Return type

dict

recursive_validate_config()#

Recursively validate hierarchies of Configs.

This uses reflection to call validate_config on a hierarchy of Configs using a depth-first pre-order traversal.

revalidate()#

Re-validate an instantiated Config.

Runs all Pydantic validators plus self.validate_config().

Adapted from: https://github.com/samuelcolvin/pydantic/issues/1864#issuecomment-679044432

update(*args, **kwargs)#

Update any fields before validation.

Subclasses should override this to provide complex default behavior, for example, setting default values as a function of the values of other fields. The arguments to this method will vary depending on the type of Config.

validate_config()#

Validate fields that should be checked after update is called.

This is to complement the builtin validation that Pydantic performs at the time of object construction.

validate_list(field: str, valid_options: List[str])#

Validate a list field.

Parameters
  • field (str) – name of field to validate

  • valid_options (List[str]) – values that field is allowed to take

Raises

ConfigError – if field is invalid