domain_randomization

default_randomizers

Storage for default a.k.a. nominal domain parameter values and default randomizers

create_conservative_randomizer(env: Union[SimEnv, EnvWrapper]) → DomainRandomizer[source]

Create the default conservative randomizer depending on the passed environment.

Parameters:

env – environment that should be perturbed

Returns:

default conservative randomizer

create_damping_dryfriction_domain_param_map_wamjsc() → Dict[int, str][source]

Create a mapping from indices to domain parameters (as used in the LFI algorithm).

Returns:

dict where the key is the index and the value is the domain parameter name

create_default_domain_param_map_bob() → Dict[int, Tuple[str, str]][source]

Create the default mapping from indices to domain parameters (as used in the BayRn algorithm).

Returns:

dict where the key is the index and the value is a tuple of domain parameter and the associated domain distribution parameter

create_default_domain_param_map_omo() → Dict[int, Tuple[str, str]][source]

Create the default mapping from indices to domain parameters (as used in the BayRn algorithm).

Returns:

dict where the key is the index and the value is a tuple of domain parameter and the associated domain distribution parameter

create_default_domain_param_map_pend() → Dict[int, Tuple[str, str]][source]

Create the default mapping from indices to domain parameters (as used in the BayRn algorithm).

Returns:

dict where the key is the index and the value is a tuple of domain parameter and the associated domain distribution parameter

create_default_domain_param_map_qq() → Dict[int, Tuple[str, str]][source]

Create the default mapping from indices to domain parameters (as used in the BayRn algorithm).

Returns:

dict where the key is the index and the value is a tuple of domain parameter and the associated domain distribution parameter

create_default_domain_param_map_wambic() → Dict[int, Tuple[str, str]][source]

Create the default mapping from indices to domain parameters (as used in the BayRn algorithm).

Returns:

dict where the key is the index and the value is a tuple of domain parameter and the associated domain distribution parameter

create_default_randomizer(env: Union[SimEnv, EnvWrapper]) → DomainRandomizer[source]

Create the default randomizer depending on the passed environment.

Parameters:

env – (wrapped) environment that should be perturbed

Returns:

default randomizer

create_default_randomizer_ant() → DomainRandomizer[source]

Create the default randomizer for the MuJoCo-based AntSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_ant_epsilon(epsilon: float) → DomainRandomizer[source]

Create a randomizer for the MuJoCo-based AntSim whichs domain parameter ranges are controlled by a scalar factor.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_bl() → DomainRandomizer[source]

Create the default randomizer for the BoxLifting.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_bob() → DomainRandomizer[source]

Create the default randomizer for the BallOnBeamSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_bop() → DomainRandomizer[source]

Create the default randomizer for the BallOnPlateSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_bs() → DomainRandomizer[source]

Create the default randomizer for the BoxShelvingSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_cata() → DomainRandomizer[source]

Create the default randomizer for the CatapultSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_cth() → DomainRandomizer[source]
create_default_randomizer_humanoid() → DomainRandomizer[source]

Create the default randomizer for the MuJoCo-based HumanoidSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_humanoid_epsilon(epsilon: float) → DomainRandomizer[source]

Create a randomizer for the MuJoCo-based HumanoidSim whichs domain parameter ranges are controlled by a scalar factor.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_omo() → DomainRandomizer[source]

Create the default randomizer for the OneMassOscillatorSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_pend() → DomainRandomizer[source]

Create the default randomizer for the PendulumSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_pi() → DomainRandomizer[source]

Create the default randomizer for the PlanarInsertSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_qbb() → DomainRandomizer[source]

Create the default randomizer for the QBallBalancerSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_qcp() → DomainRandomizer[source]

Create the default randomizer for the QCartPoleSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_qq() → DomainRandomizer[source]

Create the default randomizer for the QQubeSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_wambic() → DomainRandomizer[source]

Create the default randomizer for the MuJoCo-based WAMBallInCupSim.

Returns:

randomizer based on the nominal domain parameter values

create_default_randomizer_wamjsc() → DomainRandomizer[source]
create_empty_randomizer() → DomainRandomizer[source]

Create an empty randomizer independent of the environment to be filled later (using add_domain_params).

Returns:

empty randomizer

create_example_randomizer_cata() → DomainRandomizer[source]

Create the randomizer for the CatapultSim used for the ‘illustrative example’ in F. Muratore et al, 2019, TAMPI.

Returns:

randomizer based on the nominal domain parameter values

create_uniform_masses_lengths_randomizer_qq(frac_halfspan: float)[source]

Get a uniform randomizer that applies to all masses and lengths of the Quanser Qube according to a fraction of their nominal parameter values

Parameters:

frac_halfspan – fraction of the nominal parameter value

Returns:

DomainRandomizer with uniformly distributed masses and lengths

create_zero_var_randomizer(env: Union[SimEnv, EnvWrapper], eps: float = 1e-08) → DomainRandomizer[source]

Create the randomizer which always returns the nominal domain parameter values.

Note

The variance will not be completely zero as this would lead to invalid distributions (PyTorch checks)

Parameters:

  • env – environment that should be perturbed

  • eps – factor to scale the distributions variance with

Returns:

randomizer with zero variance for all parameters

default_randomizer(env_module, env_class)[source]

Register a default randomizer provider for a given environment type. The environment type is referenced by name to avoid eager loading of all environments when this module is used.

Usage:

@default_randomizer('pyrado.environments.xy.my', 'MyEnv')
    def create_default_randomizer_my() -> DomainRandomizer:
        <implementation>
Parameters:

  • env_module – module in which the env class is defined

  • env_class – environment class name

Returns:

decorator for default randomizer provider function

domain_parameter

class BernoulliDomainParam(val_0: Union[int, float], val_1: Union[int, float], prob_1: float, **kwargs)[source]

Bases: DomainParam

Domain parameter sampled from a Bernoulli distribution

Constructor

Parameters:

  • val_0 – value of event 0

  • val_1 – value of event 1

  • prob_1 – probability of event 1, equals 1 - probability of event 0

  • kwargs – forwarded to DomainParam constructor

adapt(domain_distr_param: str, domain_distr_param_value: Union[float, int])[source]

Update this domain parameter.

Note

This function should by called by the subclasses’ adapt() function.

Parameters:

  • domain_distr_param – distribution parameter to update, e.g. mean or std

  • domain_distr_param_value – new value of the distribution parameter

get_field_names() → List[str][source]

Get union of all hyper-parameters of all domain parameter distributions.

sample(num_samples: int = 1) → List[Tensor][source]

Generate new domain parameter values.

Parameters:

num_samples – number of samples (sets of new parameter values)

Returns:

list of tensors containing the new parameter values

class DomainParam(name: Union[str, List[str]], clip_lo: Optional[Union[int, float]] = -inf, clip_up: Optional[Union[int, float]] = inf, roundint: bool = False)[source]

Bases: object

Class to store and manage (probably multiple) domain parameter a.k.a. physics parameter a.k.a. simulator parameter

Constructor, also see the constructor of DomainRandomizer.

Parameters:

  • name – name of the parameter; can be a list of names if this domain parameters supports multiple parameters, e.g., if the distribution respects correlations; this is usually not the case and the only known subclass with this behavior is SelfPacedDomainParam

  • clip_lo – lower value for clipping

  • clip_up – upper value for clipping

  • roundint – flags if the parameters should be rounded and converted to an integer

adapt(domain_distr_param: str, domain_distr_param_value: Union[float, int, Tensor])[source]

Update this domain parameter.

Note

This function should by called by the subclasses’ adapt() function.

Parameters:

  • domain_distr_param – distribution parameter to update, e.g. mean or std

  • domain_distr_param_value – new value of the distribution parameter

get_field_names() → List[str][source]

Get union of all hyper-parameters of all domain parameter distributions.

sample(num_samples: int = 1) → Union[List[Tensor], List[List[Tensor]]][source]

Generate new domain parameter values.

Parameters:

num_samples – number of samples (sets of new parameter values)

Returns:

list of tensors containing the new parameter values or list of list of tensors containing the new values for each parameter separately; the outer list is for different parameters, the inner list is for the number of samples

class MultivariateNormalDomainParam(mean: Union[int, float, Tensor], cov: Union[list, Tensor], **kwargs)[source]

Bases: DomainParam

Domain parameter sampled from a normal distribution

Constructor

Parameters:

  • mean – nominal parameter value

  • cov – covariance

  • kwargs – forwarded to DomainParam constructor

adapt(domain_distr_param: str, domain_distr_param_value: Tensor)[source]

Update this domain parameter.

Note

This function should by called by the subclasses’ adapt() function.

Parameters:

  • domain_distr_param – distribution parameter to update, e.g. mean or std

  • domain_distr_param_value – new value of the distribution parameter

get_field_names() → List[str][source]

Get union of all hyper-parameters of all domain parameter distributions.

class NormalDomainParam(mean: Union[int, float, Tensor], std: Union[int, float, Tensor], **kwargs)[source]

Bases: DomainParam

Domain parameter sampled from a normal distribution

Constructor

Parameters:

  • mean – nominal parameter value

  • std – standard deviation

  • kwargs – forwarded to DomainParam constructor

adapt(domain_distr_param: str, domain_distr_param_value: Union[float, int])[source]

Update this domain parameter.

Note

This function should by called by the subclasses’ adapt() function.

Parameters:

  • domain_distr_param – distribution parameter to update, e.g. mean or std

  • domain_distr_param_value – new value of the distribution parameter

get_field_names() → List[str][source]

Get union of all hyper-parameters of all domain parameter distributions.

class SelfPacedDomainParam(name: List[str], target_mean: Tensor, target_cov_flat: Tensor, init_mean: Tensor, init_cov_flat: Tensor, clip_lo: float, clip_up: float)[source]

Bases: DomainParam

Constructor

Parameters:

  • name – names of the parameters

  • target_mean – target means of the contextual distribution

  • target_cov_flat – target covariances of the contextual distribution

  • init_mean – initial mean of the contextual distribution

  • init_cov_flat – initial covariances of the contextual distribution

  • clip_lo – lowest values to allow for sampling

adapt(domain_distr_param: str, domain_distr_param_value: Tensor)[source]

Update this domain parameter.

Parameters:

  • domain_distr_param – distribution parameter to update, e.g. mean or cov

  • domain_distr_param_value – new value of the distribution parameter

property context_cov: Tensor

Get the current covariance matrix.

property context_distr: MultivariateNormal

Get the current contextual distribution.

static from_domain_randomizer(domain_randomizer, *, target_cov_factor=1.0, init_cov_factor=0.01)[source]

Creates a self-paced domain parameter having the same initial and target mean and target variance given by the domain randomizer’s variance (scaled by target_cov_factor). The initial variance is also given by the domain randomizer’s variance (scaled by init_cov_factor).

Parameters:

  • domain_randomizer – randomizer to grab the data from

  • target_cov_factor – scaling of the randomizer’s variance to get the target variance; defaults to 1

  • init_cov_factor – scaling of the randomizer’s variance to get the init variance; defaults to 1/100

Returns:

the self-paced domain parameter

get_field_names() → Sequence[str][source]

Get union of all hyper-parameters of all domain parameter distributions.

info() → dict[source]
static make_broadening(name: List[str], mean: List[float], init_cov_portion: float = 0.001, target_cov_portion: float = 0.1, clip_lo: float = -inf, clip_up: float = inf) → SelfPacedDomainParam[source]

Creates a self-paced domain parameter having the same initial and target mean, but a larger variance on the target distribution.

Parameters:

  • name – names of the parameters

  • mean – means of the contextual distribution

  • init_cov_portion – percentage of the mean the initial variance should be set to

  • target_cov_portion – percentage of the mean the target variance should be set to

  • clip_lo – lowest values to allow for sampling

  • clip_up – highest values to allow for sampling

sample(num_samples: int = 1) → List[tensor][source]

Samples num_samples samples of the current context distribution.

property target_cov: Tensor

Get the target covariance matrix.

property target_distr: MultivariateNormal

Get the target distribution.

class UniformDomainParam(mean: Union[int, float, Tensor], halfspan: Union[int, float, Tensor], **kwargs)[source]

Bases: DomainParam

Domain parameter sampled from a normal distribution

Constructor

Parameters:

  • mean – nominal parameter value

  • halfspan – half interval

  • kwargs – forwarded to DomainParam constructor

adapt(domain_distr_param: str, domain_distr_param_value: Union[float, int])[source]

Update this domain parameter.

Note

This function should by called by the subclasses’ adapt() function.

Parameters:

  • domain_distr_param – distribution parameter to update, e.g. mean or std

  • domain_distr_param_value – new value of the distribution parameter

get_field_names() → List[str][source]

Get union of all hyper-parameters of all domain parameter distributions.

domain_randomizer

class DomainRandomizer(*domain_params: DomainParam)[source]

Bases: object

Class for executing the domain randomization

Constructor

Parameters:

domain_params – list or tuple of DomainParam instances

adapt_one_distr_param(domain_param_name: str, domain_distr_param: str, domain_distr_param_value: Union[float, int])[source]

Update the randomizer’s domain parameter distribution for one domain parameter.

Parameters:

  • domain_param_name – name of the domain parameter which’s distribution parameter should be updated

  • domain_distr_param – distribution parameter to update, e.g. mean or std

  • domain_distr_param_value – new value of the distribution parameter

add_domain_params(*domain_params: DomainParam, dp_mapping: Optional[Mapping[int, str]] = None)[source]

Add an arbitrary number of domain parameters with their distributions to the randomizer.

Parameters:

  • domain_params – list or tuple of DomainParam instances

  • dp_mapping – mapping from subsequent integers (starting at 0) to domain parameter names (e.g. mass). This only sets the same and is intended to be used to guarantee the right number and order of domain parameters in the randomizer.

get_params(num_samples: int = -1, fmt: str = 'list', dtype: str = 'numpy') → Union[list, dict][source]

Get the values in the data frame of the perturbed parameters.

Parameters:

  • num_samples – number of samples to be extracted from the randomizer

  • fmt – format (list of dicts or dict of lists) in which the params should be returned

  • dtype – data type in which the params should be returned

Returns:

dict of num_samples perturbed values per specified param or one dict of one perturbed

randomize(num_samples: int)[source]

Draw random parameters from the associated distributions. Internally stores a dict with parameter names as dict-keys and the new parameters in list form as dict values.

Parameters:

num_samples – number of samples to draw for each parameter

rescale_distr_param(param: str, scale: float)[source]

Rescale a parameter for all distributions.

Parameters:

  • param – name of the parameter to change (e.g. std, or cov)

  • scale – scaling factor

transformations

class DomainParamTransform(wrapped_env: Union[SimEnv, EnvWrapper], mask: Union[List[str], Tuple[str]], transformation: BijectiveTransformation)[source]

Bases: EnvWrapper, Serializable

Base class for all domain parameter transformations applied by the environment during setting and getting

These transformations are useful for avoiding infeasible values such as negative masses. When set, a domain parameter is transformed using the inverse_domain_param() function i.e. taking the exp() of the given values. The reasoning behind this is that some learning methods work on the set of real numbers, thus we make them learn in the transformed space, here the log-space, without telling them.

Constructor

Parameters:

  • wrapped_env – environment to wrap

  • mask – every domain parameters which names are in this mask will be transformed. Capitalisation matters.

  • transformation – transformation to apply

UNTRANSFORMED_DOMAIN_PARAMETER_SUFFIX: str = '_untransformed'
property domain_param: dict

These are the environment’s domain parameters, which are synonymous to the parameters used by the simulator to run the physics simulation (e.g., masses, extents, or friction coefficients). The property domain_param includes all parameters that can be perturbed a.k.a. randomized, but there might also be additional parameters.

forward(value: Union[int, float, ndarray, Tensor]) → Union[int, float, ndarray, Tensor][source]

Map a domain parameter value from the actual domain parameter space to the transformed space.

Parameters:

value – domain parameter value in the original space

Returns:

domain parameter value in the transformed space

forward_domain_param(domain_param: dict) → dict[source]

Map a domain parameter set from the actual domain parameter space to the transformed space.

Parameters:

domain_param – domain parameter set in the original space

Returns:

domain parameter set in the transformed space

inverse(value: Union[int, float, ndarray, Tensor]) → Union[int, float, ndarray, Tensor][source]

Map a domain parameter value from the transformed space to the actual domain parameter space.

Parameters:

value – domain parameter value in the transformed space

Returns:

domain parameter value in the original space

inverse_domain_param(domain_param: dict) → dict[source]

Map a domain parameter set from the transformed space to the actual domain parameter space.

Parameters:

domain_param – domain parameter set in the transformed space

Returns:

domain parameter set in the original space

reset(init_state: Optional[ndarray] = None, domain_param: Optional[dict] = None) → ndarray[source]

Reset the environment to its initial state and optionally set different domain parameters.

Parameters:

  • init_state – set explicit initial state if not None

  • domain_param – set explicit domain parameters if not None

Return obs:

initial observation of the state.

property trafo_mask: Union[List[str], Tuple[str]]

Get the mask of transformed domain parameters.

utils

param_grid(param_values: dict) → list[source]

Create a parameter set for every possible combination of parameters.

Parameters:

param_values – dict from parameter names to values for these parameters

Returns:

list of parameter sets

print_domain_params(domain_params: Union[dict, Sequence[dict]])[source]

Print a list of (domain parameter) dicts / a dict (of domain parameters) prettily.

Parameters:

domain_params – list of dicts or a single dict containing the a list of domain parameters

wrap_like_other_env(env_targ: ~typing.Union[~pyrado.environments.sim_base.SimEnv, ~pyrado.environments.real_base.RealEnv], env_src: [<class 'pyrado.environments.sim_base.SimEnv'>, <class 'pyrado.environment_wrappers.base.EnvWrapper'>], use_downsampling: bool = False) → Union[SimEnv, RealEnv][source]

Wrap a given real environment like it’s simulated counterpart (except the domain randomization of course).

Parameters:

  • env_targ – target environment e.g. environment representing the physical device

  • env_src – source environment e.g. simulation environment used for training

  • use_downsampling – apply a wrapper that downsamples the actions if the sampling frequencies don’t match

Returns:

target environment

Module contents