Language

unilab.envs.mdp.rewards¶

Community-style reward terms for the NumPy manager runtime.

Functions

`action_acc_l2`(env)	Penalize the second difference of raw policy actions.
`action_rate_l2`(env)	Penalize the first difference of raw policy actions.
`body_angular_velocity_penalty`(env[, asset_cfg])	Penalize roll/pitch angular velocity of one selected body.
`flat_orientation_l2`(env[, asset_cfg])	Penalize non-flat base orientation.
`is_alive`(env)	Reward environments that have not reached a non-timeout termination.
`is_terminated`(env)	Return one for non-timeout terminations.
`joint_pos_limits`(env[, asset_cfg])	Penalize joint positions if they cross the soft limits.
`joint_vel_l2`(env[, asset_cfg])	Penalize selected joint velocities with an L2-squared kernel.
`root_height`(env[, asset_cfg])	Return the world-frame root height as a scalar reward or metric term.
`track_angular_velocity`(env, std, command_name)	Reward commanded yaw rate while keeping roll/pitch rates near zero.
`track_linear_velocity`(env, std, command_name)	Reward commanded base linear velocity, assuming commanded z is zero.
`upright`(env, std[, asset_cfg])	Gaussian reward for keeping one selected body upright (mjlab `upright`).

Classes

`posture`	Penalize joint deviation from default pose with a per-joint-std Gaussian kernel.
`variable_posture`	`posture` with speed-dependent tolerance: standing/walking/running std maps.

unilab.envs.mdp.rewards.action_acc_l2(env)[source]¶

Penalize the second difference of raw policy actions.

Parameters:: env (ManagerBasedRlEnv)
Return type:: ndarray

unilab.envs.mdp.rewards.action_rate_l2(env)[source]¶

Penalize the first difference of raw policy actions.

Parameters:: env (ManagerBasedRlEnv)
Return type:: ndarray

unilab.envs.mdp.rewards.body_angular_velocity_penalty(env, asset_cfg=SceneEntityCfg(name='robot', joint_names=None, joint_ids=slice(None, None, None), body_names=None, body_ids=slice(None, None, None), geom_names=None, geom_ids=slice(None, None, None), site_names=None, site_ids=slice(None, None, None), actuator_names=None, actuator_ids=slice(None, None, None), tendon_names=None, tendon_ids=slice(None, None, None), camera_names=None, camera_ids=slice(None, None, None), light_names=None, light_ids=slice(None, None, None), material_names=None, material_ids=slice(None, None, None), texture_names=None, texture_ids=slice(None, None, None), pair_names=None, pair_ids=slice(None, None, None), preserve_order=False))[source]¶

Penalize roll/pitch angular velocity of one selected body.

Parameters:

env (ManagerBasedRlEnv)
asset_cfg (SceneEntityCfg)

Return type:

ndarray

unilab.envs.mdp.rewards.flat_orientation_l2(env, asset_cfg=SceneEntityCfg(name='robot', joint_names=None, joint_ids=slice(None, None, None), body_names=None, body_ids=slice(None, None, None), geom_names=None, geom_ids=slice(None, None, None), site_names=None, site_ids=slice(None, None, None), actuator_names=None, actuator_ids=slice(None, None, None), tendon_names=None, tendon_ids=slice(None, None, None), camera_names=None, camera_ids=slice(None, None, None), light_names=None, light_ids=slice(None, None, None), material_names=None, material_ids=slice(None, None, None), texture_names=None, texture_ids=slice(None, None, None), pair_names=None, pair_ids=slice(None, None, None), preserve_order=False))[source]¶

Penalize non-flat base orientation.

Parameters:

env (ManagerBasedRlEnv)
asset_cfg (SceneEntityCfg)

Return type:

ndarray

unilab.envs.mdp.rewards.is_alive(env)[source]¶

Reward environments that have not reached a non-timeout termination.

Parameters:: env (ManagerBasedRlEnv)
Return type:: ndarray

unilab.envs.mdp.rewards.is_terminated(env)[source]¶

Return one for non-timeout terminations.

Parameters:: env (ManagerBasedRlEnv)
Return type:: ndarray

unilab.envs.mdp.rewards.joint_pos_limits(env, asset_cfg=SceneEntityCfg(name='robot', joint_names=None, joint_ids=slice(None, None, None), body_names=None, body_ids=slice(None, None, None), geom_names=None, geom_ids=slice(None, None, None), site_names=None, site_ids=slice(None, None, None), actuator_names=None, actuator_ids=slice(None, None, None), tendon_names=None, tendon_ids=slice(None, None, None), camera_names=None, camera_ids=slice(None, None, None), light_names=None, light_ids=slice(None, None, None), material_names=None, material_ids=slice(None, None, None), texture_names=None, texture_ids=slice(None, None, None), pair_names=None, pair_ids=slice(None, None, None), preserve_order=False))[source]¶

Penalize joint positions if they cross the soft limits.

Parameters:

env (ManagerBasedRlEnv)
asset_cfg (SceneEntityCfg)

Return type:

ndarray

unilab.envs.mdp.rewards.joint_vel_l2(env, asset_cfg=SceneEntityCfg(name='robot', joint_names=None, joint_ids=slice(None, None, None), body_names=None, body_ids=slice(None, None, None), geom_names=None, geom_ids=slice(None, None, None), site_names=None, site_ids=slice(None, None, None), actuator_names=None, actuator_ids=slice(None, None, None), tendon_names=None, tendon_ids=slice(None, None, None), camera_names=None, camera_ids=slice(None, None, None), light_names=None, light_ids=slice(None, None, None), material_names=None, material_ids=slice(None, None, None), texture_names=None, texture_ids=slice(None, None, None), pair_names=None, pair_ids=slice(None, None, None), preserve_order=False))[source]¶

Penalize selected joint velocities with an L2-squared kernel.

Parameters:

env (ManagerBasedRlEnv)
asset_cfg (SceneEntityCfg)

Return type:

ndarray

class unilab.envs.mdp.rewards.posture[source]¶

Bases: ManagerTermBase

Penalize joint deviation from default pose with a per-joint-std Gaussian kernel.

params["std"] maps joint-name regexes to per-joint standard deviations; the reward is exp(-mean(error^2 / std^2)) over the selected joints.

Parameters:

cfg (ManagerTermBaseCfg)
env (ManagerBasedRlEnv)

__init__(cfg, env)[source]¶

Parameters:

cfg (ManagerTermBaseCfg)
env (ManagerBasedRlEnv)

unilab.envs.mdp.rewards.root_height(env, asset_cfg=SceneEntityCfg(name='robot', joint_names=None, joint_ids=slice(None, None, None), body_names=None, body_ids=slice(None, None, None), geom_names=None, geom_ids=slice(None, None, None), site_names=None, site_ids=slice(None, None, None), actuator_names=None, actuator_ids=slice(None, None, None), tendon_names=None, tendon_ids=slice(None, None, None), camera_names=None, camera_ids=slice(None, None, None), light_names=None, light_ids=slice(None, None, None), material_names=None, material_ids=slice(None, None, None), texture_names=None, texture_ids=slice(None, None, None), pair_names=None, pair_ids=slice(None, None, None), preserve_order=False))[source]¶

Return the world-frame root height as a scalar reward or metric term.

Parameters:

env (ManagerBasedRlEnv)
asset_cfg (SceneEntityCfg)

Return type:

ndarray

unilab.envs.mdp.rewards.track_angular_velocity(env, std, command_name, asset_cfg=SceneEntityCfg(name='robot', joint_names=None, joint_ids=slice(None, None, None), body_names=None, body_ids=slice(None, None, None), geom_names=None, geom_ids=slice(None, None, None), site_names=None, site_ids=slice(None, None, None), actuator_names=None, actuator_ids=slice(None, None, None), tendon_names=None, tendon_ids=slice(None, None, None), camera_names=None, camera_ids=slice(None, None, None), light_names=None, light_ids=slice(None, None, None), material_names=None, material_ids=slice(None, None, None), texture_names=None, texture_ids=slice(None, None, None), pair_names=None, pair_ids=slice(None, None, None), preserve_order=False))[source]¶

Reward commanded yaw rate while keeping roll/pitch rates near zero.

Parameters:

env (ManagerBasedRlEnv)
std (float)
command_name (str)
asset_cfg (SceneEntityCfg)

Return type:

ndarray

unilab.envs.mdp.rewards.track_linear_velocity(env, std, command_name, asset_cfg=SceneEntityCfg(name='robot', joint_names=None, joint_ids=slice(None, None, None), body_names=None, body_ids=slice(None, None, None), geom_names=None, geom_ids=slice(None, None, None), site_names=None, site_ids=slice(None, None, None), actuator_names=None, actuator_ids=slice(None, None, None), tendon_names=None, tendon_ids=slice(None, None, None), camera_names=None, camera_ids=slice(None, None, None), light_names=None, light_ids=slice(None, None, None), material_names=None, material_ids=slice(None, None, None), texture_names=None, texture_ids=slice(None, None, None), pair_names=None, pair_ids=slice(None, None, None), preserve_order=False))[source]¶

Reward commanded base linear velocity, assuming commanded z is zero.

Parameters:

env (ManagerBasedRlEnv)
std (float)
command_name (str)
asset_cfg (SceneEntityCfg)

Return type:

ndarray

unilab.envs.mdp.rewards.upright(env, std, asset_cfg=SceneEntityCfg(name='robot', joint_names=None, joint_ids=slice(None, None, None), body_names=None, body_ids=slice(None, None, None), geom_names=None, geom_ids=slice(None, None, None), site_names=None, site_ids=slice(None, None, None), actuator_names=None, actuator_ids=slice(None, None, None), tendon_names=None, tendon_ids=slice(None, None, None), camera_names=None, camera_ids=slice(None, None, None), light_names=None, light_ids=slice(None, None, None), material_names=None, material_ids=slice(None, None, None), texture_names=None, texture_ids=slice(None, None, None), pair_names=None, pair_ids=slice(None, None, None), preserve_order=False))[source]¶

Gaussian reward for keeping one selected body upright (mjlab upright).

The reward is exp(-||pg_xy||^2 / std^2) where pg is the world gravity vector expressed in the selected body’s link frame. Flat-ground form only: mjlab’s optional terrain-normal sensors are not ported.

Parameters:

env (ManagerBasedRlEnv)
std (float)
asset_cfg (SceneEntityCfg)

Return type:

ndarray

class unilab.envs.mdp.rewards.variable_posture[source]¶

Bases: ManagerTermBase

posture with speed-dependent tolerance: standing/walking/running std maps.

The per-joint std is selected from std_standing / std_walking / std_running by the total command speed (planar norm plus yaw magnitude) against walking_threshold and running_threshold.

Parameters:

cfg (ManagerTermBaseCfg)
env (ManagerBasedRlEnv)

__init__(cfg, env)[source]¶

Parameters:

cfg (ManagerTermBaseCfg)
env (ManagerBasedRlEnv)