unisim.backend.isaacsim.backend

Host-side IsaacSim/IsaacLab subprocess backend.

IsaacSim is deliberately kept out of the UniLab interpreter: the supported IsaacSim 5.1 wheels require Python 3.11, while the main project supports a different Python range. The NumPy-facing contract and lifecycle come from the backend-neutral subprocess_ipc owner; this module supplies IsaacSim runtime discovery, the worker entrypoint, clone-origin validation, and the eval-owned Kit/viewer/camera capability boundary.

Classes

IsaacSimBackend

Thin host client for the IsaacLab/PhysX worker.

IsaacSimModelInfo

Opaque metadata returned by the IsaacSim worker handshake.

Exceptions

IsaacSimRenderError

Raised when the requested IsaacSim render mode cannot be initialized.

IsaacSimWorkerError

Raised when the external IsaacSim/IsaacLab worker fails.
class unisim.backend.isaacsim.backend.IsaacSimBackend[source]

Bases: MjcfSubprocessBackend

Thin host client for the IsaacLab/PhysX worker.

The shared client owns pipe framing, shared-memory slot allocation, timeout/crash diagnostics, XML cold-path metadata, and all NumPy state views. Physics remains the default (render_mode=None). Eval/play profiles pass a concrete render intent through the cold INIT payload so Kit selects the correct experience before simulation materialization. The worker owns all camera/viewport operations; this class validates the NumPy-facing frame contract.

Parameters:
__init__(scene, num_envs, sim_dt, *, render_mode=None, render_width=1280, render_height=720, solver_position_iteration_count=None, solver_velocity_iteration_count=None, bounce_threshold_velocity=None, contact_offset=None, rest_offset=None, max_depenetration_velocity=None, gpu_max_rigid_contact_count=None, gpu_max_rigid_patch_count=None, tensor_cuda_ipc=False, share_friction_materials=False, env_spacing=None, **kwargs)[source]
Parameters:
get_dr_capabilities()[source]

Advertise only reset and wrench terms implemented by mapped scenes.

Host-plane sets cover torch.cpu payloads; the device plane (torch.cuda) carries the CUDA IPC PD-gain lane only.

Return type:

DomainRandomizationCapabilities

get_reset_term_default(term)[source]

Return per-environment variant-assigned mapped reset defaults.

Parameters:

term (str)

Return type:

ndarray

apply_interval_randomization(plan)[source]

Apply a scheduled interval randomization plan.

Generic dispatch: each op yielded by plan.iter_ops() is validated against the builtin term specs (custom terms pass through) and routed to the backend-owned handler table returned by _interval_term_handlers(). Op payloads are torch-first: Torch tensors or TensorIntervalPayload values produced by the caller; host adapters convert tensors at their private boundary through unisim.dr._host, and a missing producer-sampled payload fails closed. A term without a handler fails closed with NotImplementedError naming the backend class and the term. Backends that need per-plan prologue/epilogue semantics (for example clearing staged external forces before the ops accumulate) keep a thin override that calls this base implementation.

Parameters:

plan (IntervalRandomizationPlan)

Return type:

None

apply_body_force(body_ids, force, torque=None)[source]

Stage a world-frame wrench for the next mapped-scene step.

The force acts at each target body’s center of mass and the optional torque is about that center. Repeated submissions accumulate until the next successful step consumes them.

Parameters:
Return type:

None

set_pre_step_control(fn)[source]

Register a callback for mapped scenes; model-file callbacks stay rejected.

Parameters:

fn (Any | None)

Return type:

None

get_tensor_execution()[source]

Declare the optional tensor lifecycle without discovering SDKs.

Return type:

TensorExecution

get_tensor_capabilities()[source]

Return fail-closed tensor methods and negotiable state fields.

Return type:

TensorLifecycleCapabilities

get_public_state_widths()[source]

Return the mapped public qpos/qvel arena widths.

Return type:

PublicStateWidths

get_sensor_inventory()[source]

Return the complete CUDA IPC tensor-sensor namespace without host slots.

Return type:

tuple[SensorDescriptor, ...]

get_state_views(fields=None, device=None)[source]

Return backend-owned state through the declared tensor lifecycle.

Tensor views are Torch tensors. Returned values are logically read-only and mutation is undefined.

DEVICE_RESIDENT adapters return stable live views on the backend’s exact accelerator device; device=None selects that device and any other device is rejected. HOST_BRIDGE adapters return explicit copies from authoritative host state; device=None selects host CPU and callers pass an explicit accelerator device for H2D copies. Unlike the removed 1.x NumPy snapshot API, successful tensor adapters do not return NumPy arrays. Consume the declared tensor execution mode rather than probing the array implementation.

Parameters:
Return type:

Mapping[str, Any]

get_sensor_view(name, device=None)[source]

Return one named sensor view on the declared tensor lifecycle.

Parameters:
Return type:

Any

get_tracked_body_views(body_names=None, device=None)[source]

Return four ordered tracked-body blocks from one public IPC read.

Parameters:
Return type:

TrackedBodyStateViews

step_tensor(ctrl, nsteps=1)[source]

Advance physics from a backend-declared accelerator control tensor.

ctrl is a contiguous float32 Torch tensor with shape (num_envs, num_actuators), lives on the adapter-required device, and must be finite. The method consumes it before synchronizing and returning.

Parameters:
Return type:

dict | None

set_state_tensor(env_indices, qpos, qvel, randomization=None)[source]

Set selected state through the adapter-declared tensor lifecycle.

env_indices is a contiguous one-dimensional int64 Torch tensor with unique values in [0, num_envs); qpos and qvel are contiguous float32 tensors with shapes (len(env_indices), nq) and (len(env_indices), nv). All three share one adapter-accepted device and must be finite. Adapters may use a bounded synchronization for fail-closed row validation. DEVICE_RESIDENT adapters otherwise avoid a host detour; HOST_BRIDGE adapters make their explicit accelerator-to-host boundary measurable before CPU state is updated. Inputs are consumed before return.

randomization is the unified torch-first TensorResetRandomizationPayload: every populated field is a dense selected-row Torch tensor table. Tensors on torch.cpu form the host plane, checked against DomainRandomizationCapabilities.supported_reset_terms; tensors on any other device form the device plane, checked against supported_device_reset_terms (empty by default, so undeclared device support fails closed naming the term). Non-delta terms hold final absolute values applied as selected-row in-place scatters; base_mass_delta and base_com_offset hold deltas merged with the adapter baseline. Host adapters convert tensors at their private NumPy boundary through unisim.dr._host.

Parameters:

  • env_indices (Any)

  • qpos (Any)

  • qvel (Any)

  • randomization (TensorResetRandomizationPayload | None)

Return type:

dict | None

set_state_tensor_mask(reset_mask, qpos, qvel, randomization=None, *, full_refresh=False)[source]

Commit full-width state through a backend-owned mask projection.

Parameters:

  • reset_mask (Any)

  • qpos (Any)

  • qvel (Any)

  • randomization (TensorResetRandomizationPayload | None)

  • full_refresh (bool)

Return type:

dict | None

bind_packed_reset(*, device=None)[source]

Begin a packed transaction over the external CUDA IPC reset boundary.

Parameters:

device (Any | None)

Return type:

Any

close()[source]

Shut down the worker and release shared memory. Idempotent.

Return type:

None

get_body_mass()[source]

Return native per-environment body masses in public body-id order.

Return type:

ndarray

get_body_ipos(env_ids=None)[source]

Return canonical defaults or native selected materialization COM offsets.

Parameters:

env_ids (Sequence[int] | ndarray | None)

Return type:

ndarray

get_geom_names()[source]

Return qualified geometry names in frozen public geometry order.

Return type:

tuple[str, ...]

get_geom_body_ids()[source]

Return owning public body IDs in frozen geometry order.

Return type:

ndarray

get_geom_contact_masks()[source]

Return normalized native collision state in public geometry order.

Return type:

tuple[ndarray, ndarray]

get_geom_friction()[source]

Return native effective per-environment friction in public order.

Columns are IsaacSim [static_friction, dynamic_friction, 0]. PhysX has no native equivalent for MuJoCo’s torsional/rolling coefficients.

Return type:

ndarray

get_play_capabilities()[source]

Return the Kit viewer/camera plus physics-state playback capabilities.

Both scene profiles implement the playback contract. The mapped entity scene serves worker-maintained public qpos/qvel rows; the model-file whole-model profile assembles the same snapshot from its root/dof wire slots once the construction-time joint-order validation passes. Model-file scenes whose MJCF topology the synthetic 7/6-root wire cannot replay (fixed base, ball joints, reordered freejoints) keep failing closed here.

Return type:

PlayCapabilities

get_physics_state_layout()[source]

Return the [time, qpos, qvel] snapshot layout (no mocap tail yet).

Return type:

PhysicsStateLayout

get_physics_state()[source]

Assemble contract [time, qpos, qvel] rows from the public slots.

The worker refreshes the public shared-memory rows after every step and reset, so the snapshot is a host-side copy of those rows plus the leading time column; no new IPC frame or worker-side asset access is involved. The worker protocol carries no simulation clock, so time comes from the per-env host accumulator advanced on every public step path and zeroed on full state resets, mirroring the mujoco backend’s time semantics.

The mapped profile copies the MJCF-order qpos/qvel rows verbatim. The model-file profile projects its wire slots: root_state carries the synthetic floating root as [pos, quat wxyz, world lin vel (link origin), world ang vel] and dof_state the scalar joints in MJCF document order, so the snapshot’s MuJoCo-convention qvel angular block is the world angular velocity rotated into the body frame — exactly the convention the model-file set_state path consumes, which keeps snapshot restore a lossless round trip.

Return type:

ndarray

set_physics_state(state)[source]

Restore a get_physics_state snapshot through the set_state path.

Parameters:

state (ndarray)

Return type:

None

get_playback_model(env_index=None)[source]

Return the MJCF playback source for detached MuJoCo replay.

The mapped profile keeps the shared implementation (fixed-variant scenes require an explicit env_index). The model-file profile replays through its whole-MJCF construction source; the adapter rejects model-file fixed-variant plans at construction, so one source serves every row.

Parameters:

env_index (int | None)

Return type:

Any

resolve_play_render_plan(*, play_render_mode, play_steps, output_video)[source]

Resolve high-level playback mode into backend-owned render parameters.

Parameters:
Return type:

PlayRenderPlan

bind_renderer(spacing=1.0, *, offset_mode='grid', headless=False, capture=False, width=1280, height=720, camera_kwargs=None)[source]

Initialize the worker-side viewer and/or capture camera.

spacing/offset_mode are accepted for contract parity and ignored: envs are already laid out on the worker sim’s grid.

Parameters:
Return type:

None

capture_video_frame()[source]

Capture one RGB frame from the worker’s camera sensor.

Return type:

ndarray

class unisim.backend.isaacsim.backend.IsaacSimModelInfo[source]

Bases: SubprocessModelInfo

Opaque metadata returned by the IsaacSim worker handshake.

Parameters:
__init__(num_dof, num_bodies, dof_names, body_names, gravity, use_gpu_pipeline)
Parameters:
exception unisim.backend.isaacsim.backend.IsaacSimRenderError[source]

Bases: RuntimeError

Raised when the requested IsaacSim render mode cannot be initialized.

exception unisim.backend.isaacsim.backend.IsaacSimWorkerError[source]

Bases: IsaacGymWorkerError

Raised when the external IsaacSim/IsaacLab worker fails.

IsaacGymWorkerError is retained as a compatibility ancestor because early subprocess adapters exposed that exception as the public worker failure type. The concrete class remains distinct, so callers can still distinguish IsaacSim failures with an exact type check while existing except IsaacGymWorkerError handlers continue to work.

Parameters: