unilab.algos.torch.offpolicy.runner.OffPolicyRunner

class unilab.algos.torch.offpolicy.runner.OffPolicyRunner[source]

Bases: AsyncRunner

Unified runner for SAC and TD3.

Parameters:

  • env_name (str)

  • algo_type (str)

  • num_envs (int)

  • replay_buffer_n (int)

  • batch_size (int)

  • learning_starts (int)

  • updates_per_step (int)

  • policy_frequency (int)

  • sync_collection (bool)

  • env_steps_per_sync (int)

  • device (str | None)

  • actor_hidden_dim (int)

  • use_layer_norm (bool)

  • obs_normalization (bool)

  • sim_backend (str)

  • env_cfg_override (dict | None)

  • actor_kwargs (dict | None)

  • seed (int | None)

  • trace_enabled (bool)

  • trace_output_dir (str | None)

  • trace_thread_time (bool)

  • trace_cuda_events (bool)

Methods

__init__(learner, env_name, algo_type[, ...])

close()

learn([max_iterations, save_interval, ...])

Unified training loop for off-policy algorithms.
__init__(learner, env_name, algo_type, num_envs=4096, replay_buffer_n=1024, batch_size=8192, learning_starts=0, updates_per_step=8, policy_frequency=4, sync_collection=True, env_steps_per_sync=1, device=None, actor_hidden_dim=512, use_layer_norm=True, obs_normalization=False, sim_backend='mujoco', env_cfg_override=None, actor_kwargs=None, seed=None, trace_enabled=False, trace_output_dir=None, trace_thread_time=False, trace_cuda_events=True)[source]
Parameters:

  • env_name (str)

  • algo_type (str)

  • num_envs (int)

  • replay_buffer_n (int)

  • batch_size (int)

  • learning_starts (int)

  • updates_per_step (int)

  • policy_frequency (int)

  • sync_collection (bool)

  • env_steps_per_sync (int)

  • device (str | None)

  • actor_hidden_dim (int)

  • use_layer_norm (bool)

  • obs_normalization (bool)

  • sim_backend (str)

  • env_cfg_override (dict | None)

  • actor_kwargs (dict | None)

  • seed (int | None)

  • trace_enabled (bool)

  • trace_output_dir (str | None)

  • trace_thread_time (bool)

  • trace_cuda_events (bool)

learn(max_iterations=1500, save_interval=50, log_dir='logs', logger_type='tensorboard')[source]

Unified training loop for off-policy algorithms.

Parameters:

  • max_iterations (int)

  • save_interval (int)

  • log_dir (str)

  • logger_type (str)

Return type:

None

close()[source]
Return type:

None