Language

unilab.algos.torch.hora.ppo¶

Classes

HoraPPO

PPO variant that constructs a shared HORA actor-critic backbone.

class unilab.algos.torch.hora.ppo.HoraPPO[source]¶

Bases: FinalObservationAwarePPO

PPO variant that constructs a shared HORA actor-critic backbone.

Parameters:

actor (HoraActorModel)
critic (HoraCriticModel)
storage (RolloutStorage)
num_learning_epochs (int)
num_mini_batches (int)
clip_param (float)
gamma (float)
lam (float)
value_loss_coef (float)
entropy_coef (float)
learning_rate (float)
max_grad_norm (float)
optimizer (str)
use_clipped_value_loss (bool)
schedule (str)
desired_kl (float)
normalize_advantage_per_mini_batch (bool)
device (str)
rnd_cfg (dict | None)
symmetry_cfg (dict | None)
multi_gpu_cfg (dict | None)
enable_compile (bool)

__init__(actor, critic, storage, num_learning_epochs=5, num_mini_batches=4, clip_param=0.2, gamma=0.99, lam=0.95, value_loss_coef=1.0, entropy_coef=0.01, learning_rate=0.001, max_grad_norm=1.0, optimizer='adam', use_clipped_value_loss=True, schedule='adaptive', desired_kl=0.01, normalize_advantage_per_mini_batch=False, device='cpu', rnd_cfg=None, symmetry_cfg=None, multi_gpu_cfg=None, enable_compile=False)[source]¶

Parameters:

actor (HoraActorModel)
critic (HoraCriticModel)
storage (RolloutStorage)
num_learning_epochs (int)
num_mini_batches (int)
clip_param (float)
gamma (float)
lam (float)
value_loss_coef (float)
entropy_coef (float)
learning_rate (float)
max_grad_norm (float)
optimizer (str)
use_clipped_value_loss (bool)
schedule (str)
desired_kl (float)
normalize_advantage_per_mini_batch (bool)
device (str)
rnd_cfg (dict | None)
symmetry_cfg (dict | None)
multi_gpu_cfg (dict | None)
enable_compile (bool)

static construct_algorithm(obs, env, cfg, device)[source]¶

Parameters:

obs (TensorDict)
env (VecEnv)
cfg (dict)
device (str)

Return type:

PPO

process_env_step(obs, rewards, dones, extras)[source]¶

Parameters:

obs (TensorDict)
rewards (Tensor)
dones (Tensor)
extras (dict[str, Tensor | TensorDict])

Return type:

None