FlashSAC

FlashSAC runs through src/unilab/scripts/train_flashsac.py in its own config tree. Select it with --algo flashsac; defaults are inlined in src/unilab/conf/flashsac/config.yaml, and the implementation lives under unilab.rl.algos.flash_sac.

It shares the off-policy runner design with SAC, but does not use the same default networks: the actor uses a block-based structure and the critic uses a distributional (categorical) Q variant.

Quick Start

uv run train --algo flashsac --task g1_walk_flat --sim mujoco
uv run train --algo flashsac --task go2_joystick_flat --sim mujoco training.no_play=true

Key Fields

For the off-policy playback path (src/unilab/scripts/train_flashsac.py / CLI --algo flashsac), set training.export_onnx=false to skip policy.onnx export while still recording playback video. See Evaluation and Playback.

  • algo.algo_log_name=flash_sac

  • algo.num_envs=1024

  • algo.max_iterations=5000

  • algo.tau=0.01

  • algo.save_interval=1000

  • algo.algo_params.actor_num_blocks=2

  • algo.algo_params.critic_num_blocks=2

Learner acceleration controls

The default single-NVIDIA learner enables all three acceleration layers. They remain separate configuration choices:

  • algo.algo_params.use_compile controls CUDA Inductor compilation. Set it to false for the eager per-update compatibility path; this also disables the dependent whole-cycle graph.

  • algo.algo_params.compile_full_objectives selects complete actor/critic objectives instead of the smaller loss-tensor compile regions. It does not control CUDA Graph capture.

  • algo.algo_params.use_whole_cycle_cuda_graph controls learner-owned whole-cycle CUDA Graph capture. Set only this field to false to retain Inductor compilation without whole-cycle capture.

For example, run the eager debugging path with:

uv run train --algo flashsac --task g1_walk_flat --sim mujoco \
  algo.algo_params.use_compile=false

FlashSAC requires synchronized collection and the same sole replay path as SAC: bounded host ingress plus one complete replay ring on a CUDA or Apple MPS learner device. CPU and XPU training are unsupported.

The log root is logs/flash_sac/<task>/.

Checkpoint compatibility

Every FlashSAC checkpoint records an actor_semantics payload (schema_version/kind/builder/kwargs/obs_dim/action_dim/ action_transform, written by unilab.rl.algos.flash_sac.actor_spec). Both load paths — training resume and play — validate it fail-closed before any weights are restored, for the built-in actor and for custom actors injected through algo.algo_params.actor_config alike.

Pre-contract checkpoints (a missing or null actor_semantics entry) are no longer supported for any actor, including the built-in actor: loading fails closed with an error that states the minimum supported format. The only recovery path is to retrain the policy with a current release. The supported baseline starts with the first release after 1.3.5 that carries this change; it is not backported to any 1.3.5 patch release.