unilab.algos.torch.flash_sac

FlashSAC algorithm package.

class unilab.algos.torch.flash_sac.FlashSACActor[source]

Bases: Module

Parameters:
zeta_cdf: torch.Tensor
__init__(num_blocks, input_dim, hidden_dim, action_dim, noise_zeta_mu=2.0, noise_zeta_max=16, device='cpu')[source]

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:
normalize_parameters()[source]
Return type:

None

get_mean_and_std(observations, training)[source]
Parameters:
Return type:

tuple[Tensor, Tensor]

forward(observations, training)[source]

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:
Return type:

tuple[Tensor, dict[str, Tensor]]

as_export_module()[source]

Return a single-input/single-output wrapper suitable for torch.onnx.export.

Return type:

Module

explore(obs, dones=None, deterministic=False)[source]
Parameters:
Return type:

Tensor

class unilab.algos.torch.flash_sac.FlashSACDoubleCritic[source]

Bases: Module

Parameters:
__init__(num_blocks, input_dim, hidden_dim, num_bins, min_v, max_v, num_qs=2, device='cpu')[source]

Initialize internal Module state, shared by both nn.Module and ScriptModule.

Parameters:
normalize_parameters()[source]
Return type:

None

forward(observations, actions, training)[source]

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:
Return type:

tuple[Tensor, dict[str, Tensor]]

class unilab.algos.torch.flash_sac.FlashSACLearner[source]

Bases: object

Parameters:

  • obs_dim (int)

  • action_dim (int)

  • critic_obs_dim (int)

  • device (str)

  • gamma (float)

  • tau (float)

  • actor_lr (float)

  • critic_lr (float)

  • actor_hidden_dim (int)

  • critic_hidden_dim (int)

  • actor_num_blocks (int)

  • critic_num_blocks (int)

  • num_atoms (int)

  • critic_min_v (float)

  • critic_max_v (float)

  • temp_initial_value (float)

  • temp_target_sigma (float)

  • temp_target_entropy (float | None)

  • actor_bc_alpha (float)

  • actor_noise_zeta_mu (float)

  • actor_noise_zeta_max (int)

  • learning_rate_init (float)

  • learning_rate_peak (float)

  • learning_rate_end (float)

  • learning_rate_warmup_steps (int)

  • learning_rate_decay_steps (int)

  • normalize_reward (bool)

  • normalized_g_max (float)

  • n_step (int)

  • obs_normalization (bool)

  • use_amp (bool)

  • amp_dtype (str)

  • use_compile (bool)

__init__(obs_dim, action_dim, critic_obs_dim, device='cpu', gamma=0.99, tau=0.01, actor_lr=0.0003, critic_lr=0.0003, actor_hidden_dim=128, critic_hidden_dim=256, actor_num_blocks=2, critic_num_blocks=2, num_atoms=101, critic_min_v=-5.0, critic_max_v=5.0, temp_initial_value=0.01, temp_target_sigma=0.15, temp_target_entropy=None, actor_bc_alpha=0.0, actor_noise_zeta_mu=2.0, actor_noise_zeta_max=16, learning_rate_init=0.0003, learning_rate_peak=0.0003, learning_rate_end=0.00015, learning_rate_warmup_steps=0, learning_rate_decay_steps=500000, normalize_reward=True, normalized_g_max=5.0, n_step=1, obs_normalization=False, use_amp=False, amp_dtype='auto', use_compile=False)[source]
Parameters:

  • obs_dim (int)

  • action_dim (int)

  • critic_obs_dim (int)

  • device (str)

  • gamma (float)

  • tau (float)

  • actor_lr (float)

  • critic_lr (float)

  • actor_hidden_dim (int)

  • critic_hidden_dim (int)

  • actor_num_blocks (int)

  • critic_num_blocks (int)

  • num_atoms (int)

  • critic_min_v (float)

  • critic_max_v (float)

  • temp_initial_value (float)

  • temp_target_sigma (float)

  • temp_target_entropy (float | None)

  • actor_bc_alpha (float)

  • actor_noise_zeta_mu (float)

  • actor_noise_zeta_max (int)

  • learning_rate_init (float)

  • learning_rate_peak (float)

  • learning_rate_end (float)

  • learning_rate_warmup_steps (int)

  • learning_rate_decay_steps (int)

  • normalize_reward (bool)

  • normalized_g_max (float)

  • n_step (int)

  • obs_normalization (bool)

  • use_amp (bool)

  • amp_dtype (str)

  • use_compile (bool)

update_reward_stats(rewards, dones)[source]
Parameters:
Return type:

None

update_critic(batch)[source]
Parameters:

batch (dict[str, Tensor])

Return type:

dict[str, float]

update_actor(batch)[source]
Parameters:

batch (dict[str, Tensor])

Return type:

dict[str, float]

soft_update_target()[source]
Return type:

None

get_state_dict()[source]
Return type:

dict[str, Any]

load_state_dict(state_dict)[source]
Parameters:

state_dict (dict[str, Any])

Return type:

None

class unilab.algos.torch.flash_sac.FlashSACRunner[source]

Bases: OffPolicyRunner

Parameters:

  • env_name (str)

  • env_cfg_override (dict[str, Any] | None)

  • device (str | None)

  • num_envs (int)

  • replay_buffer_n (int)

  • batch_size (int)

  • learning_starts (int)

  • updates_per_step (int)

  • policy_frequency (int)

  • sync_collection (bool)

  • env_steps_per_sync (int)

  • gamma (float)

  • tau (float)

  • actor_lr (float)

  • critic_lr (float)

  • obs_normalization (bool)

  • actor_hidden_dim (int)

  • critic_hidden_dim (int)

  • num_atoms (int)

  • use_amp (bool)

  • sim_backend (str)

  • actor_num_blocks (int)

  • critic_num_blocks (int)

  • actor_bc_alpha (float)

  • actor_noise_zeta_mu (float)

  • actor_noise_zeta_max (int)

  • critic_min_v (float)

  • critic_max_v (float)

  • target_sigma (float)

  • target_entropy (float | None)

  • temp_initial_value (float)

  • learning_rate_init (float)

  • learning_rate_peak (float)

  • learning_rate_end (float)

  • learning_rate_warmup_steps (int)

  • learning_rate_decay_steps (int)

  • normalize_reward (bool)

  • normalized_g_max (float)

  • n_step (int)

  • amp_dtype (str)

  • use_compile (bool)

  • seed (int | None)

  • trace_enabled (bool)

  • trace_output_dir (str | None)

  • trace_thread_time (bool)

  • trace_cuda_events (bool)

__init__(env_name, env_cfg_override=None, device=None, num_envs=2048, replay_buffer_n=512, batch_size=2048, learning_starts=0, updates_per_step=1, policy_frequency=2, sync_collection=True, env_steps_per_sync=1, gamma=0.99, tau=0.01, actor_lr=0.0003, critic_lr=0.0003, obs_normalization=False, actor_hidden_dim=128, critic_hidden_dim=256, num_atoms=101, use_amp=False, sim_backend='mujoco', actor_num_blocks=2, critic_num_blocks=2, actor_bc_alpha=0.0, actor_noise_zeta_mu=2.0, actor_noise_zeta_max=16, critic_min_v=-5.0, critic_max_v=5.0, target_sigma=0.15, target_entropy=None, temp_initial_value=0.01, learning_rate_init=0.0003, learning_rate_peak=0.0003, learning_rate_end=0.00015, learning_rate_warmup_steps=0, learning_rate_decay_steps=500000, normalize_reward=True, normalized_g_max=5.0, n_step=1, amp_dtype='auto', use_compile=False, seed=None, trace_enabled=False, trace_output_dir=None, trace_thread_time=False, trace_cuda_events=True)[source]
Parameters:

  • env_name (str)

  • env_cfg_override (dict[str, Any] | None)

  • device (str | None)

  • num_envs (int)

  • replay_buffer_n (int)

  • batch_size (int)

  • learning_starts (int)

  • updates_per_step (int)

  • policy_frequency (int)

  • sync_collection (bool)

  • env_steps_per_sync (int)

  • gamma (float)

  • tau (float)

  • actor_lr (float)

  • critic_lr (float)

  • obs_normalization (bool)

  • actor_hidden_dim (int)

  • critic_hidden_dim (int)

  • num_atoms (int)

  • use_amp (bool)

  • sim_backend (str)

  • actor_num_blocks (int)

  • critic_num_blocks (int)

  • actor_bc_alpha (float)

  • actor_noise_zeta_mu (float)

  • actor_noise_zeta_max (int)

  • critic_min_v (float)

  • critic_max_v (float)

  • target_sigma (float)

  • target_entropy (float | None)

  • temp_initial_value (float)

  • learning_rate_init (float)

  • learning_rate_peak (float)

  • learning_rate_end (float)

  • learning_rate_warmup_steps (int)

  • learning_rate_decay_steps (int)

  • normalize_reward (bool)

  • normalized_g_max (float)

  • n_step (int)

  • amp_dtype (str)

  • use_compile (bool)

  • seed (int | None)

  • trace_enabled (bool)

  • trace_output_dir (str | None)

  • trace_thread_time (bool)

  • trace_cuda_events (bool)

Modules

double_buffer

FlashSAC builder for the CPU-pinned double-buffer replay path.

layers

FlashSAC layers and lightweight normalization helpers.

learner

FlashSAC learner adapted to UniLab's off-policy contract.

network

FlashSAC actor, critic, and temperature modules.

runner

FlashSAC runner using the shared off-policy runtime.

update

FlashSAC update helpers.