visionsim.simulate package¶

Submodules¶

visionsim.simulate.blender module¶

visionsim.simulate.blender.require_connected_client(func: Callable[Concatenate[BlenderClient, _P], Any]) → Callable[Concatenate[BlenderClient, _P], Any][source]¶

Decorator which ensures a client is connected.

Parameters:: func (Callable[Concatenate[BlenderClient, _P], Any]) – Function to decorate
Raises:: RuntimeError – raised if client is not connected.
Returns:: Decorated function.
Return type:: Callable[Concatenate[BlenderClient, _P], Any]

visionsim.simulate.blender.require_connected_clients(func: Callable[Concatenate[BlenderClients, _P], Any]) → Callable[Concatenate[BlenderClients, _P], Any][source]¶

Decorator which ensures all clients are connected.

Parameters:: func (Callable[Concatenate[BlenderClients, _P], Any]) – Function to decorate
Raises:: RuntimeError – raised if at least one client is not connected.
Returns:: Decorated function.
Return type:: Callable[Concatenate[BlenderClients, _P], Any]

visionsim.simulate.blender.require_initialized_service(func: Callable[Concatenate[BlenderService, _P], Any]) → Callable[Concatenate[BlenderService, _P], Any][source]¶

Decorator which ensures the render service was initialized.

Parameters:: func (Callable[Concatenate[BlenderService, _P], Any]) – Function to decorate
Raises:: RuntimeError – raised if client.initialize has not been previously called.
Returns:: Decorated function.
Return type:: Callable[Concatenate[BlenderService, _P], Any]

visionsim.simulate.blender.validate_camera_moved(func: Callable[Concatenate[BlenderService, _P], Any]) → Callable[Concatenate[BlenderService, _P], Any][source]¶

Decorator which emits a warning if the camera was not moved.

Parameters:: func (Callable[Concatenate[BlenderService, _P], Any]) – Function to decorate
Returns:: Decorated function.
Return type:: Callable[Concatenate[BlenderService, _P], Any]

Bases: Server

Expose a BlenderService to the outside world via RPCs.

Example

Once created, it can be started, which will block and await for an external connection from a BlenderClient:

server = BlenderServer()
server.start()

However, this needs to be called within blender’s runtime. Instead one can use BlenderServer.spawn() to spawn one or more blender instances, each with their own server.

Initialize a BlenderServer instance

Parameters:

hostname (bytes | str | None, optional) – the host to bind to. By default, the ‘wildcard address’ is used to listen on all interfaces. If not properly secured, the server can receive traffic from unintended or even malicious sources. Defaults to None (wildcard).
port (bytes | str | int | None, optional) – the TCP port to bind to. Defaults to 0 (bind to a random open port).
service (type[BlenderService], optional) – the service to expose, must be a BlenderService subclass. Defaults to BlenderService.
extra_config (dict, optional) – the configuration dictionary that is passed to the RPyC connection. Defaults to {"allow_all_attrs": True, "allow_setattr": True}.
**kwargs – Additional keyword arguments which are passed to the rpyc.utils.server.Server constructor.

Raises:

RuntimeError – a BlenderServer needs to be instantiated from within a blender instance.
ValueError – the exposed service must be BlenderService or subclass.

static spawn(jobs: int = 1, timeout: float = -1.0, log: str | os.PathLike | FILE | tuple[FILE, FILE] = -3, autoexec: bool = False, executable: str | os.PathLike | None = None) → Iterator[tuple[list[subprocess.Popen], list[tuple[str, int]]]][source]¶

Spawn one or more blender instances and start a BlenderServer in each.

This is roughly equivalent to calling blender -b --python blender.py in many subprocesses, where blender.py initializes and starts a server instance. Proper logging and termination of these processes is also taken care of.

Note

The returned processes and connection settings are not guaranteed to be in the same order.

Warning

If log is a file handle or descriptor, such as redirecting Blender logs to subprocess.STDOUT, the writing process might get overwhelmed which can cause silent errors, dropped logs and locked processes. It is thus not recommended for long render jobs to set log to anything but DEVNULL or a directory.

Parameters:

jobs (int, optional) – number of jobs to spawn. Defaults to 1.
timeout (float, optional) – try to discover spawned instances for timeout (in seconds) before giving up. If negative, a port will be randomly selected and assigned to the spawned server, bypassing the need for discovery and timeouts. Note that when a port is assigned this context manager will immediately yield, even if the server is not yet ready to accept incoming connections. Defaults to assigning a port to spawned server (-1 seconds).
log (str | os.PathLike | FILE | tuple[FILE, FILE], optional) – path to log directory, file handle, descriptor or tuple thereof. Stdout and stderr will be captured and saved if supplied. Defaults to subprocess.DEVNULL for both stdout/stderr.
autoexec (bool, optional) – if true, allow execution of any embedded python scripts within blender. For more, see blender’s CLI documentation. Defaults to False.
executable (str | os.PathLike | None, optional) – path to Blender’s executable. Defaults to looking for blender on $PATH, but is useful when targeting a specific blender install, or when it’s installed via a package manager such as flatpak. Setting it to “flatpak run –die-with-parent org.blender.Blender” might be required when using flatpaks. Defaults to None (system PATH).

Raises:

TimeoutError – raise if unable to discover spawned servers in timeout seconds and kill any spawned processes.

Yields:

tuple[list[subprocess.Popen], list[tuple[str, int]]] –

A tuple containing:

list[subprocess.Popen]: List of subprocess.Popen corresponding to all spawned servers.
list[tuple[str, int]]: List of connection setting for each server, where each element is a (hostname, port) tuple.

static spawn_registry() → tuple[Process, UDPRegistryClient][source]¶

Spawn a registry server and client to aid in server discovery, or return cached result. While this method can be called directly, it will be invoked automatically by discover() and spawn().

Returns:

A tuple containing:

Process: process running the global registry server,
rpyc.utils.registry.UDPRegistryClient: global registry client

Return type:

tuple[Process, rpyc.utils.registry.UDPRegistryClient]

static discover() → list[tuple[str, int]][source]¶

Discover any BlenderServers that are already running and return their connection parameters.

Note

A discoverable server might already be in use and can refuse connection attempts.

Returns:: List of connection setting for each server, where each element is a (hostname, port) tuple.
Return type:: list[tuple[str, int]]

class visionsim.simulate.blender.BlenderService[source]¶

Bases: Service

Server-side API to interact with blender and render novel views.

Most of the methods of a BlenderClient instance are remote procedure calls to a connected blender service. These methods are prefixed by exposed_.

ALIASES: tuple[str] = ('BLENDER',)¶

__init__() → None[source]¶

Initialize render service.

Raises:: RuntimeError – raised if not within blender’s runtime.

on_connect(conn: Connection) → None[source]¶

Called when the connection is established

Parameters:: conn (rpyc.Connection) – Connection object

on_disconnect(conn: Connection) → None[source]¶

Called when the connection has already terminated. Resets blender runtime. (must not perform any IO on the connection)

Parameters:: conn (rpyc.Connection) – Connection object

reset() → None[source]¶

Cleans up and resets blender runtime.

De-initialize service by restoring blender to it’s startup state, ensuring any cached attributes are cleaned (otherwise objects will be stale), and resetting any instance variables that were previously initialized.

register_output_type(subpath: str, node: bpy.types.CompositorNodeOutputFile, slot: bpy.types.NodeOutputFileSlotFile | bpy.types.NodeCompositorFileOutputItem, **camera_defaults) → None[source]¶

Register a new output datatype. If this is not called by an include_ method, the metadata for that datatype will not be saved to the database and the path to which the data is saved will not be updated at every render.

Warning

You must pass in the slot instance that was returned when a new file_output_item was created and not simply one of the node.file_output_items as these are readonly!

Parameters:

subpath (str) – Path suffix, from root data directory, of the new datatype (eg: “previews/depths”)
node (bpy.types.CompositorNodeOutputFile) – Output file node responsible for saving
slot (bpy.types.NodeOutputFileSlotFile | bpy.types.NodeCompositorFileOutputItem) – Slot of node which will save the output data.
**camera_defaults – Any addition camera information that will be added by default. Commonly, the number of output channels is passed in (eg: c=4 for RGBA).

Raises:

RuntimeError – raised if output type has already been registered.

property scene: bpy.types.Scene¶: Get current blender scene

property tree: bpy.types.CompositorNodeTree¶: Get current scene’s node tree

property render_layers: bpy.types.CompositorNodeRLayers[source]¶: Get and cache render layers node, create one if needed.

property view_layer: bpy.types.ViewLayer¶: Get current view layer

property camera: bpy.types.Camera¶: Get active camera, detect when it changes.

get_parents(obj: bpy.types.Object) → list[bpy.types.Object][source]¶

Recursively retrieves parent objects of a given object in Blender

Parameters:: obj – Object to find parent of.
Returns:: Parent objects of obj.
Return type:: list[bpy.types.Object]

exposed_with_logger(log: Logger) → None[source]¶

Use supplied logger, if logger is initialized in client, messages will log to the client.

Parameters:: log (logging.Logger) – Logger to use for messages

exposed_initialize(blend_file: str | PathLike, root_path: str | PathLike, **kwargs) → None[source]¶

Initialize BlenderService and load blendfile.

Parameters:

blend_file (str | os.PathLike) – path of scene file to load.
root_path (str | os.PathLike) – path at which to save rendered results.
**kwargs – Additional keyword arguments to be passed to bpy.ops.wm.open_mainfile.

exposed_iter_fcurves(actions: list[bpy.types.Action] | None = None) → Iterator[bpy.types.FCurve][source]¶

Yield fcurves of all actions.

This abstracts away the API for accessing fcurves which changed to using channelbags in v4.4, see release notes here.

Parameters:: actions (list[bpy.types.Action] | None, optional) – Only yield fcurves from these actions if specified, otherwise use all actions. Defaults to None.
Yields:: Iterator[bpy.types.FCurve] – an fcurve object from the scene or action

exposed_get_original_fps() → float[source]¶

Get effective framerate (fps/fps_base).

Returns:: Frame rate of scene.
Return type:: float

exposed_animation_range() → range[source]¶

Get animation range of current scene as range(start, end+1, step).

Returns:: Range of frames in animation.
Return type:: range

exposed_animation_range_tuple() → tuple[int, int, int][source]¶

Get animation range of current scene as a tuple of (start, end, step).

Returns:: Frame start, end, and step of animation.
Return type:: tuple[int, int, int]

exposed_include_composites(file_format: FILE_FORMATS | None = None, color_mode: COLOR_MODES | None = None, exr_codec: EXR_CODECS | None = None, bit_depth: Literal[8, 16, 32] | None = None) → None[source]¶

Sets up Blender to include the outputs of any existing compositor nodes groups.

Note: A default arguments of None means do not change setting inherited from the blendfile’s Output settings.

Parameters:

file_format (str | None, optional) – Format to save composited render as. Options vary depending on the version of Blender, with the following being broadly available: (‘BMP’, ‘IRIS’, ‘PNG’, ‘JPEG’, ‘JPEG2000’, ‘TARGA’, ‘TARGA_RAW’, ‘CINEON’, ‘DPX’, ‘OPEN_EXR’, ‘HDR’, ‘TIFF’, ‘WEBP’). Defaults to None.
color_mode (str | None, optional) – Typically one of (‘BW’, ‘RGB’, ‘RGBA’). Defaults to None.
exr_codec (str | None, optional) – Codec used to compress exr file. Only used when file_format="OPEN_EXR", options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to None.
bit_depth (int | None, optional) – Bit depth per channel, also referred to as color-depth. Options depend on the chosen file format, with 8, 16 and 32bits being common. Defaults to None.

exposed_include_frames(file_format: FILE_FORMATS = 'PNG', color_mode: COLOR_MODES = 'RGB', exr_codec: EXR_CODECS = 'DWAA', bit_depth: Literal[8, 16, 32] = 8) → None[source]¶

Sets up Blender compositor to include ground truth rendered images, bypassing any existing compositor nodes.

Note

For linear intensity renders, use the “OPEN_EXR” format with and 32 or 16 bits.

Parameters:

file_format (str, optional) – Format to save ground truth render as. Options vary depending on the version of Blender, with the following being broadly available: (‘BMP’, ‘IRIS’, ‘PNG’, ‘JPEG’, ‘JPEG2000’, ‘TARGA’, ‘TARGA_RAW’, ‘CINEON’, ‘DPX’, ‘OPEN_EXR’, ‘HDR’, ‘TIFF’, ‘WEBP’). Defaults to “PNG”.
color_mode (str, optional) – Typically one of (‘BW’, ‘RGB’, ‘RGBA’). Defaults to “RGB”.
exr_codec (str, optional) – Codec used to compress exr file. Only used when file_format="OPEN_EXR", options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “DWAA”.
bit_depth (int, optional) – Bit depth per channel, also referred to as color-depth. Options depend on the chosen file format, with 8, 16 and 32 bits being common. Defaults to 8 bits.

Raises:

ValueError – raised when file-format not understood.

exposed_include_depths(preview: bool = True, file_format: FILE_FORMATS = 'OPEN_EXR', exr_codec: EXR_CODECS = 'DWAA', bit_depth: Literal[16, 32] = 32) → None[source]¶

Sets up Blender compositor to include depth map for rendered images.

Note

The preview colormap is re-normalized on a per-frame basis, to visually compare across frames, apply colorization after rendering using the CLI.

Parameters:

preview (bool, optional) – If true, colorized depth maps, helpful for quick visualizations, will be generated alongside ground-truth depth maps. Defaults to True.
file_format (str, optional) – Format of depth maps, one of “OPEN_EXR” or “HDR”. Defaults to “OPEN_EXR”.
exr_codec (str, optional) – Codec used to compress exr file. Only used when file_format="OPEN_EXR", options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “DWAA”.
bit_depth (int, optional) – Bit depth per channel, also referred to as color-depth. Options depend on the chosen file format, with 8, 16 and 32 bits being common. Defaults to 32 bits.

Raises:

ValueError – raised when file-format not understood.

exposed_include_normals(preview: bool = True, exr_codec: EXR_CODECS = 'DWAA', bit_depth: Literal[16, 32] = 32) → None[source]¶

Sets up Blender compositor to include normal map for rendered images.

Parameters:

preview (bool, optional) – If true, colorized normal maps will also be generated with each vector component being remapped from [-1, 1] to [0-255] where XYZ coordinates are mapped channel-wise to RGB. Defaults to True.
exr_codec (str, optional) – Codec used to compress exr file. Options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “DWAA”.
bit_depth (int, optional) – Bit depth per channel, also referred to as color-depth. Either 16 or 32 bits. Defaults to 32 bits.

exposed_include_flows(preview: bool = True, direction: Literal['forward', 'backward', 'both'] = 'forward', exr_codec: EXR_CODECS = 'DWAA', bit_depth: Literal[16, 32] = 32) → None[source]¶

Sets up Blender compositor to include optical flow for rendered images.

Parameters:

preview (bool, optional) – If true, also save preview visualizations of flow. Defaults to True.
direction (str, optional) – One of ‘forward’, ‘backward’ or ‘both’. Direction of flow to colorize for preview visualization. Only used when preview is true, otherwise both forward and backward flows are saved. Defaults to “forward”.
exr_codec (str, optional) – Codec used to compress exr file. Options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “DWAA”.
bit_depth (int, optional) – Bit depth per channel, also referred to as color-depth. Options depend on the chosen file format, with 8, 16 and 32 bits being common. Defaults to 32 bits.

Note

The preview colormap is re-normalized on a per-frame basis, to visually compare across frames, apply colorization after rendering using the CLI.

Raises:

ValueError – raised when direction is not understood.
RuntimeError – raised when motion blur is enabled as flow cannot be computed.

exposed_include_segmentations(preview: bool = True, shuffle: bool = True, seed: int = 1234, exr_codec: EXR_CODECS = 'DWAA', bit_depth: Literal[16, 32] = 32) → None[source]¶

Sets up Blender compositor to include segmentation maps for rendered images.

The preview visualization simply assigns a color to each object ID by mapping the objects ID value to a hue using a HSV node with saturation=1 and value=1 (except for the background which will have a value of 0 to ensure it is black).

Parameters:

preview (bool, optional) – If true, also save preview visualizations of segmentation. Defaults to True.
shuffle (bool, optional) – Shuffle preview colors, helps differentiate object instances. Defaults to True.
seed (int, optional) – Random seed used when shuffling colors. Defaults to 1234.
exr_codec (str, optional) – Codec used to compress exr file. Options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “DWAA”.
bit_depth (int, optional) – Bit depth per channel, also referred to as color-depth. Either 16 or 32 bits. Defaults to 32 bits.

Raises:

RuntimeError – raised when not using CYCLES, as other renderers do not support a segmentation pass.

exposed_include_materials(preview: bool = True, shuffle: bool = True, seed: int = 1234, exr_codec: EXR_CODECS = 'DWAA', bit_depth: Literal[16, 32] = 32) → None[source]¶

Sets up Blender compositor to include material ID maps for rendered images.

The preview visualization simply assigns a color to each material ID by mapping the materials ID value to a hue using a HSV node with saturation=1 and value=1 (except for the background which will have a value of 0 to ensure it is black).

Parameters:

preview (bool, optional) – If true, also save preview visualizations of material IDs. Defaults to True.
shuffle (bool, optional) – Shuffle preview colors, helps differentiate material instances. Defaults to True.
seed (int, optional) – Random seed used when shuffling colors. Defaults to 1234.
exr_codec (str, optional) – Codec used to compress exr file. Options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “DWAA”.
bit_depth (int, optional) – Bit depth per channel, also referred to as color-depth. Either 16 or 32 bits. Defaults to 32 bits.

Raises:

RuntimeError – raised when not using CYCLES, as other renderers do not support a material ID pass.

exposed_include_diffuse_pass(file_format: FILE_FORMATS = 'OPEN_EXR', color_mode: COLOR_MODES = 'RGB', exr_codec: EXR_CODECS = 'DWAA', bit_depth: Literal[8, 16, 32] = 32) → None[source]¶

Sets up Blender compositor to include diffuse light passes for rendered images.

For CYCLES, this includes: Diffuse Direct, Diffuse Indirect and Diffuse Color. For EEVEE, this includes: Diffuse Light and Diffuse Color.

Parameters:

file_format (str, optional) – Format to save diffuse passes as. Defaults to “OPEN_EXR”.
color_mode (str, optional) – Typically one of (‘BW’, ‘RGB’, ‘RGBA’). Defaults to “RGB”.
exr_codec (str, optional) – Codec used to compress exr file. Only used when file_format="OPEN_EXR". Defaults to “DWAA”.
bit_depth (int, optional) – Bit depth per channel. Defaults to 32 bits.

exposed_include_specular_pass(file_format: FILE_FORMATS = 'OPEN_EXR', color_mode: COLOR_MODES = 'RGB', exr_codec: EXR_CODECS = 'DWAA', bit_depth: Literal[8, 16, 32] = 32) → None[source]¶

Sets up Blender compositor to include specular light passes for rendered images.

For CYCLES, this includes: Glossy Direct, Glossy Indirect and Glossy Color. For EEVEE, this includes: Specular Light and Specular Color.

Parameters:

file_format (str, optional) – Format to save specular passes as. Defaults to “OPEN_EXR”.
color_mode (str, optional) – Typically one of (‘BW’, ‘RGB’, ‘RGBA’). Defaults to “RGB”.
exr_codec (str, optional) – Codec used to compress exr file. Only used when file_format="OPEN_EXR". Defaults to “DWAA”.
bit_depth (int, optional) – Bit depth per channel. Defaults to 32 bits.

exposed_include_points(preview: bool = True, exr_codec: EXR_CODECS = 'DWAA', bit_depth: Literal[16, 32] = 32) → None[source]¶

Sets up Blender compositor to include a world-space point map for each frame.

Note

The point map corresponds to world-space positions, like those used in VGGT [1], and not the camera-centric positions used in DUSt3R [2].

Parameters:

preview (bool, optional) – If true, colorized point maps will also be generated, where colors are assigned based on the absolute fractional world coordinates. Defaults to True.
exr_codec (str, optional) – Codec used to compress exr file. Defaults to “DWAA”.
bit_depth (int, optional) – Bit depth per channel. Either 16 or 32 bits. Defaults to 32 bits.

exposed_load_addons(*addons: str) → None[source]¶

Load blender addons by name (case-insensitive).

Parameters:: *addons (str) – name of addons to load.

exposed_set_resolution(height: tuple[int] | list[int] | int | None = None, width: int | None = None) → None[source]¶

Set frame resolution (height, width) in pixels. If a single tuple is passed, instead of using keyword arguments, it will be parsed as (height, width).

Parameters:

height (tuple[int] | list[int] | int | None, optional) – Height of render in pixels. Defaults to value from file.
width (int | None, optional) – Width of render in pixels. Defaults to value from file.

Raises:

ValueError – raised if resolution is not understood.

exposed_use_motion_blur(enable: bool) → None[source]¶

Enable/disable motion blur.

Parameters:: enable (bool) – If true, enable motion blur.
Raises:: RuntimeError – raised when motion blur is enabled as flow cannot be computed.

exposed_use_animations(enable: bool) → None[source]¶

Enable/disable all animations.

Parameters:: enable (bool) – If true, enable animations.

Enables/activates cycles render devices and settings.

Note: A default arguments of None means do not change setting inherited from blendfile.

Parameters:

device_type (str, optional) – Name of device to use, one of “cpu”, “cuda”, “optix”, “metal”, etc. See blender docs for full list. Defaults to None.
use_cpu (bool, optional) – Boolean flag to enable CPUs alongside GPU devices. Defaults to None.
adaptive_threshold (float, optional) – Set noise threshold upon which to stop taking samples. Defaults to None.
max_samples (int, optional) – Maximum number of samples per pixel to take. Defaults to None.
use_denoising (bool, optional) – If enabled, a denoising pass will be used. Defaults to None.

Raises:

RuntimeError – raised when no devices are found.
ValueError – raised when setting use_cpu is required.

Returns:

Name of activated devices.

Return type:

list[str]

exposed_unbind_camera(clear_animations: bool = True) → None[source]¶

Remove constraints, animations and parents from main camera.

Note: In order to undo this, you’ll need to re-initialize.

Parameters:: clear_animations (bool, optional) – If true clear animation data for camera.

exposed_move_keyframes(scale=1.0, shift=0.0) → None[source]¶

Adjusts keyframes in Blender animations, keypoints are first scaled then shifted.

Parameters:

scale (float, optional) – Factor used to rescale keyframe positions along x-axis. Defaults to 1.0.
shift (float, optional) – Factor used to shift keyframe positions along x-axis. Defaults to 0.0.

Raises:

RuntimeError – raised if trying to move keyframes beyond blender’s limits.

exposed_set_current_frame(frame_number: int) → None[source]¶

Set current frame number. This might advance any animations.

Parameters:: frame_number (int) – index of frame to skip to.

exposed_camera_info() → dict[str, Any][source]¶

Return a dictionary with camera intrinsics.

Returns:: dictionary containing camera parameters.
Return type:: dict[str, Any]

exposed_camera_extrinsics() → npt.NDArray[np.floating][source]¶

Get the 4x4 transform matrix encoding the current camera pose.

Returns:: Current camera pose in matrix form.
Return type:: npt.NDArray[np.floating]

exposed_position_camera(location: TypeAliasForwardRef('npt.ArrayLike') | None = None, rotation: TypeAliasForwardRef('npt.ArrayLike') | None = None, look_at: TypeAliasForwardRef('npt.ArrayLike') | None = None, in_order: bool = True) → None[source]¶

Positions and orients camera according to specified parameters. All transformations are local, use unbind_camera to ensure position is set in world coordinates.

Note: Only one of look_at or rotation can be set at once.

Parameters:

location (npt.ArrayLike, optional) – Location to place camera in 3D space. Defaults to none.
rotation (npt.ArrayLike, optional) – Rotation matrix for camera. Defaults to none.
look_at (npt.ArrayLike, optional) – Location to point camera. Defaults to none.
in_order (bool, optional) – If set, assume current camera pose is from previous/next frame and ensure new rotation set by look_at is compatible with current position. Without this, a rotations will stay in the [-pi, pi] range and this wrapping will mess up interpolations. Only used when look_at is set. Defaults to True.

Raises:

ValueError – raised if camera orientation is over-defined.

exposed_rotate_camera(angle: float) → None[source]¶

Rotate camera around it’s optical axis, relative to current orientation. All transformations are local, use unbind_camera to ensure position is set in world coordinates.

Parameters:: angle – Relative amount to rotate by (clockwise, in radians).

exposed_set_camera_keyframe(frame_num: int, matrix: TypeAliasForwardRef('npt.ArrayLike') | None = None) → None[source]¶

Set camera keyframe at given frame number. If camera matrix is not supplied, currently set camera position/rotation/scale will be used, this allows users to set camera position using position_camera and rotate_camera.

Parameters:

frame_num (int) – index of frame to set keyframe for.
matrix (npt.ArrayLike | None, optional) – 4x4 camera transform, if not supplied, use current camera matrix. Defaults to None.

exposed_set_animation_range(start: int | None = None, stop: int | None = None, step: int | None = None) → None[source]¶

Set animation range for scene.

Parameters:

start (int | None, optional) – frame start, inclusive. Defaults to None.
stop (int | None, optional) – frame stop, exclusive. Defaults to None.
step (int | None, optional) – frame interval. Defaults to None.

exposed_render_current_frame(allow_skips=True, dry_run=False) → None[source]¶

Generates a single frame in Blender at the current camera location, return the file paths for that frame, potentially including depth, normals, etc.

Note

This method renders the current frame as-is, it assumes the camera position, frame number and all other parameters have been set.

Parameters:

allow_skips (bool, optional) – if true, blender will not re-render and overwrite existing frames. This does not however apply to depth/normals/etc, which cannot be skipped. Defaults to True.
dry_run (bool, optional) – if true, nothing will be rendered at all. Defaults to False.

exposed_render_frame(frame_number: int, allow_skips=True, dry_run=False) → None[source]¶

Same as first setting current frame then rendering it.

Warning

Calling this has the side-effect of changing the current frame.

Parameters:

frame_number (int) – frame to render
allow_skips (bool, optional) – if true, blender will not re-render and overwrite existing frames. This does not however apply to depth/normals/etc, which cannot be skipped. Defaults to True.
dry_run (bool, optional) – if true, nothing will be rendered at all. Defaults to False.

exposed_render_frames(frame_numbers: Iterable[int], allow_skips=True, dry_run=False, update_fn: UpdateFn | None = None) → None[source]¶

Render all requested frames and return associated transforms dictionary.

Parameters:

frame_numbers (Iterable[int]) – frames to render.
allow_skips (bool, optional) – if true, blender will not re-render and overwrite existing frames. This does not however apply to depth/normals/etc, which cannot be skipped. Defaults to True.
dry_run (bool, optional) – if true, nothing will be rendered at all. Defaults to False.
update_fn (UpdateFn, optional) – callback function to track render progress. Will first be called with total kwarg, indicating number of steps to be taken, then will be called with advance=1 at every step. Closely mirrors the rich.Progress API. Defaults to None.

Raises:

RuntimeError – raised if trying to render frames beyond blender’s limits.

exposed_render_animation(frame_start: int | None = None, frame_end: int | None = None, frame_step: int | None = None, allow_skips=True, dry_run=False, update_fn: UpdateFn | None = None) → None[source]¶

Determines frame range to render, sets camera positions and orientations, and renders all frames in animation range.

Note: All frame start/end/step arguments are absolute quantities, applied after any keyframe moves.: If the animation is from (1-100) and you’ve scaled it by calling move_keyframes(scale=2.0) then calling render_animation(frame_start=1, frame_end=100) will only render half of the animation. By default the whole animation will render when no start/end and step values are set.

Parameters:

frame_start (int, optional) – Starting index (inclusive) of frames to render as seen in blender. Defaults to None, meaning value from .blend file.
frame_end (int, optional) – Ending index (inclusive) of frames to render as seen in blender. Defaults to None, meaning value from .blend file.
frame_step (int, optional) – Skip every nth frame. Defaults to None, meaning value from .blend file.
allow_skips (bool, optional) – Same as render_current_frame.
dry_run (bool, optional) – Same as render_current_frame.
update_fn (UpdateFn, optional) – Same as render_frames.

Raises:

ValueError – raised if scene and camera are entirely static.

exposed_save_file(path: str | PathLike) → None[source]¶

Save the opened blender file. This is useful for introspecting the state of the compositor/scene/etc.

Parameters:: path (str | os.PathLike) – path where to save blendfile.
Raises:: ValueError – raised if file already exists.

class visionsim.simulate.blender.BlenderClient(addr: tuple[str, int], timeout: float = 10.0)[source]¶

Bases: object

Client-side API to interact with blender and render novel views.

The BlenderClient is responsible for communicating with (and potentially spawning) separate BlenderServer`s that will actually perform the rendering via a :class:`BlenderService.

The client acts as a context manager, it will connect to it’s server when the context is entered and cleanly disconnect and close the connection in case of errors or when exiting the with-block.

Many useful methods to interact with blender are provided, such as set_resolution or render_animation. These methods are dynamically generated when the client connects to the server. Available methods are directly inherited from BlenderService (or whichever service the server is exposing), specifically any service method starting with exposed_ will be accessible to the client at runtime. For example, BlenderClient.include_depths is a remote procedure call to BlenderService.exposed_include_depths().

__init__(addr: tuple[str, int], timeout: float = 10.0) → None[source]¶

Initialize a client with known address of server. Note: Using auto_connect() or spawn() is often more convenient.

Parameters:

addr (tuple[str, int]) – Connection tuple containing the hostname and port
timeout (float, optional) – Maximum time in seconds the client will attempt to connect to the server for before an error is thrown. Only used when entering context manager. Defaults to 10 seconds.

addr: tuple[str, int]¶

conn: Connection | None¶

awaitable: AsyncResult | None¶

process: Popen | None¶

timeout: float¶

classmethod auto_connect(timeout: float = 10.0) → Self[source]¶

Automatically connect to available server.

Use BlenderServer.discover() to find available server within timeout.

Note: This doesn’t actually connect to the server instance, the connection happens: when the context manager is entered. This simply creates a client instance with the connection settings (i.e: hostname, port) of an existing server. The connection might still fail when entering the with-block.

Parameters:: timeout (float, optional) – try to discover server instance for timeout (in seconds) before giving up. Defaults to 10.0 seconds.
Raises:: TimeoutError – raise if unable to discover server in timeout seconds.
Returns:: client instance initialized with connection settings of existing server.
Return type:: Self

Spawn and connect to a blender server. The spawned process is accessible through the client’s process attribute.

Parameters:

timeout (float, optional) – try to discover spawned instances for timeout (in seconds) before giving up. If negative, a port will be randomly selected and assigned to the spawned server, bypassing the need for discovery and timeouts. Note that when a port is assigned this context manager will immediately yield, even if the server is not yet ready to accept incoming connections. Defaults to assigning a port to spawned server (-1 seconds).
log (str | os.PathLike | FILE | tuple[FILE, FILE], optional) – path to log directory, file handle, descriptor or tuple thereof. Stdout and stderr will be captured and saved if supplied. Defaults to subprocess.DEVNULL for both stdout/stderr.
autoexec (bool, optional) – if true, allow execution of any embedded python scripts within blender. For more, see blender’s CLI documentation. Defaults to False.
executable (str | os.PathLike | None, optional) – path to Blender’s executable. Defaults to looking for blender on $PATH, but is useful when targeting a specific blender install, or when it’s installed via a package manager such as flatpak. Setting it to “flatpak run –die-with-parent org.blender.Blender” might be required when using flatpaks. Defaults to None (system PATH).

Yields:

Self – the connected client

render_animation_async(*args, **kwargs) → AsyncResult[source]¶

Asynchronously call render_animation and return an rpyc.AsyncResult.

Parameters:

*args – Same as BlendService.exposed_render_animation()
*kwargs – Same as BlendService.exposed_render_animation()

Returns:

Result encapsulating the return value of render_animation.: After wait``ing for the render to finish, it can be accessed using the ``.value attribute.

Return type:

rpyc.AsyncResult

render_frames_async(*args, **kwargs) → AsyncResult[source]¶

Asynchronously call render_frames and return an rpyc.AsyncResult.

Parameters:

*args – Same as BlendService.exposed_render_frames()
*kwargs – Same as BlendService.exposed_render_frames()

Returns:

Result encapsulating the return value of render_frames.: After wait``ing for the render to finish, it can be accessed using the ``.value attribute.

Return type:

rpyc.AsyncResult

wait() → None[source]¶: Block and await any async results.

class visionsim.simulate.blender.BlenderClients(*objs: Iterator[BlenderClient | tuple[str, int]])[source]¶

Bases: tuple

Collection of BlenderClient instances.

Most methods in this class simply call the equivalent method of each client, that is, calling clients.set_resolution is equivalent to calling set_resolution for each client in clients. Some special methods, namely the render_frames() and render_animation() methods will instead distribute the rendering load to all clients.

Finally, entering each client’s context-manager, and closing each client connection is ensured by using this class’ context-manager.

__init__(*objs) → None[source]¶

Initialize collection of BlenderClient from iterable of clients, or their connection settings.

Parameters:: *objs (Iterator[BlenderClient | tuple[str, int]]) – BlenderClient instances or their hostnames and ports.

stack: ExitStack¶

Spawn and connect to one or more blender servers. The spawned processes are accessible through the client’s process attribute.

Parameters:

jobs (int, optional) – number of jobs to spawn. Defaults to 1.
timeout (float, optional) – try to discover spawned instances for timeout (in seconds) before giving up. If negative, a port will be randomly selected and assigned to the spawned server, bypassing the need for discovery and timeouts. Note that when a port is assigned this context manager will immediately yield, even if the server is not yet ready to accept incoming connections. Defaults to assigning a port to spawned server (-1 seconds).
log (str | os.PathLike | FILE | tuple[FILE, FILE], optional) – path to log directory, file handle, descriptor or tuple thereof. Stdout and stderr will be captured and saved if supplied. Defaults to subprocess.DEVNULL for both stdout/stderr.
autoexec (bool, optional) – if true, allow execution of any embedded python scripts within blender. For more, see blender’s CLI documentation. Defaults to False.
executable (str | os.PathLike | None, optional) – path to Blender’s executable. Defaults to looking for blender on $PATH, but is useful when targeting a specific blender install, or when it’s installed via a package manager such as flatpak. Setting it to “flatpak run –die-with-parent org.blender.Blender” might be required when using flatpaks. Defaults to None (system PATH).

Yields:

Self – the connected clients

Spawns a multiprocessing-like worker pool, each with their own BlenderClient instance. The function supplied to pool.map/imap/starmap and their async variants will be automagically passed a client instance as their first argument that they can use for rendering.

Example

def render(client, blend_file):
    root = Path("renders") / Path(blend_file).stem
    client.initialize(blend_file, root)
    client.render_animation()

if __name__ == "__main__":
    with BlenderClients.pool(2) as pool:
        pool.map(render, ["monkey.blend", "cube.blend", "metaballs.blend"])

Note

Here we use multiprocess instead of the builtin multiprocessing library to take advantage of the more advanced dill serialization (as opposed to the standard pickling).

Parameters:

jobs (int, optional) – number of jobs to spawn. Defaults to 1.
timeout (float, optional) – try to discover spawned instances for timeout (in seconds) before giving up. If negative, a port will be randomly selected and assigned to the spawned server, bypassing the need for discovery and timeouts. Note that when a port is assigned this context manager will immediately yield, even if the server is not yet ready to accept incoming connections. Defaults to assigning a port to spawned server (-1 seconds).
log (str | os.PathLike | FILE | tuple[FILE, FILE], optional) – path to log directory, file handle, descriptor or tuple thereof. Stdout and stderr will be captured and saved if supplied. Defaults to subprocess.DEVNULL for both stdout/stderr.
autoexec (bool, optional) – if true, allow execution of any embedded python scripts within blender. For more, see blender’s CLI documentation. Defaults to False.
executable (str | os.PathLike | None, optional) – path to Blender’s executable. Defaults to looking for blender on $PATH, but is useful when targeting a specific blender install, or when it’s installed via a package manager such as flatpak. Setting it to “flatpak run –die-with-parent org.blender.Blender” might be required when using flatpaks. Defaults to None (system PATH).
conns – List of connection tuples containing the hostnames and ports of existing servers. If specified, the pool will use these servers (and jobs and other spawn arguments will be ignored) instead of spawning new ones.

Yields:

multiprocess.Pool –

A multiprocess.Pool instance which has had it’s applicator methods: (map/imap/starmap/etc) monkey-patched to inject a client instance as first argument.

common_animation_range() → range[source]¶

Get animation range shared by all clients as range(start, end+1, step).

Raises:: RuntimeError – animation ranges for all clients are expected to be the same.
Returns:: Range of frames in animation.
Return type:: range

common_animation_range_tuple() → tuple[int, int, int][source]¶

Get animation range shared by all clients as a tuple of (start, end, step).

Raises:: RuntimeError – animation ranges for all clients are expected to be the same.
Returns:: Frame start, end, and step of animation.
Return type:: tuple[int, int, int]

render_frames(frame_numbers: Collection[int], allow_skips: bool = True, dry_run: bool = False, update_fn: UpdateFn | None = None) → None[source]¶

Render all requested frames by distributing workload across connected clients and return associated transforms dictionary.

Warning

Assumes all clients are initialized in the same manner, that is, to the same blendfile, with the same animation range, render settings, etc.

Parameters:

frame_numbers (Collection[int]) – frames to render.
allow_skips (bool, optional) – if true, blender will not re-render and overwrite existing frames. This does not however apply to depth/normals/etc, which cannot be skipped. Defaults to True.
dry_run (bool, optional) – if true, nothing will be rendered at all. Defaults to False.
update_fn (UpdateFn, optional) –
callback function to track render progress. Will first be called with total kwarg, indicating number of steps to be taken, then will be called with advance=1 at every step. Closely mirrors the rich.Progress API. Defaults to None.

Raises:

RuntimeError – raised if trying to render frames beyond blender’s limits.

render_animation(frame_start: int | None = None, frame_end: int | None = None, frame_step: int | None = None, allow_skips=True, dry_run=False, update_fn: UpdateFn | None = None) → None[source]¶

Determines frame range to render, sets camera positions and orientations, and renders all frames in animation range by distributing workload onto all connected clients.

Note: All frame start/end/step arguments are absolute quantities, applied after any keyframe moves.: If the animation is from (1-100) and you’ve scaled it by calling move_keyframes(scale=2.0) then calling render_animation(frame_start=1, frame_end=100) will only render half of the animation. By default the whole animation will render when no start/end and step values are set.

Parameters:

frame_start (int, optional) – Starting index (inclusive) of frames to render as seen in blender. Defaults to None, meaning value from .blend file.
frame_end (int, optional) – Ending index (inclusive) of frames to render as seen in blender. Defaults to None, meaning value from .blend file.
frame_step (int, optional) – Skip every nth frame. Defaults to None, meaning value from .blend file.
allow_skips (bool, optional) – Same as render_current_frame.
dry_run (bool, optional) – Same as render_current_frame.
update_fn (UpdateFn, optional) – Same as render_frames.

Raises:

ValueError – raised if scene and camera are entirely static.

save_file(path: str | PathLike) → None[source]¶

Save opened blender file. This is useful for introspecting the state of the compositor/scene/etc.

Note: Only saves file once (from a single connected client), assumes all clients have: been initialized in the same manner.

Parameters:: path (str | os.PathLike) – path where to save blendfile.
Raises:: ValueError – raised if file already exists.

wait() → None[source]¶: Wait for all clients at once.

visionsim.simulate.config module¶

class visionsim.simulate.config.CompositesConfig(file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] | None = None, color_mode: Literal['BW', 'RGB', 'RGBA'] | None = None, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] | None = None, bit_depth: int | None = None)[source]¶

Bases: object

Same as FramesConfig(), except with all defaults set to None such that they are inherited from the blendfile. For more information see include_composites.

file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] | None = None¶: File format used to save composited frames

color_mode: Literal['BW', 'RGB', 'RGBA'] | None = None¶

grayscale, color or color+alpha

Type:: Mode to save composited frames in

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] | None = None¶: Encoding used to compress EXRs, only used when file_format='OPEN_EXR'

bit_depth: int | None = None¶: Bit depth for intensity frames. Usually 8 for pngs, 32 or 16 bits for OPEN_EXR

__init__(file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] | None = None, color_mode: Literal['BW', 'RGB', 'RGBA'] | None = None, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] | None = None, bit_depth: int | None = None) → None¶

class visionsim.simulate.config.FramesConfig(file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] = 'PNG', color_mode: Literal['BW', 'RGB', 'RGBA'] = 'RGB', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[8, 16, 32] = 8)[source]¶

Bases: object

Same as CompositesConfig(), except with sensible defaults. For more information see include_frames.

file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] = 'PNG'¶: File format used to save ground truth frames

color_mode: Literal['BW', 'RGB', 'RGBA'] = 'RGB'¶

grayscale, color or color+alpha

Type:: Mode to save ground truth frames in

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA'¶: Encoding used to compress EXRs, only used when file_format='OPEN_EXR'

bit_depth: Literal[8, 16, 32] = 8¶: Bit depth for intensity frames. Usually 8 for pngs, 32 or 16 bits for OPEN_EXR

__init__(file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] = 'PNG', color_mode: Literal['BW', 'RGB', 'RGBA'] = 'RGB', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[8, 16, 32] = 8) → None¶

class visionsim.simulate.config.DepthsConfig(preview: bool = True, file_format: Literal['OPEN_EXR', 'HDR'] = 'OPEN_EXR', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32)[source]¶

Bases: object

For more information see include_depths.

preview: bool = True¶: Also save colorized depth maps as PNGs

file_format: Literal['OPEN_EXR', 'HDR'] = 'OPEN_EXR'¶: File format used to save depth maps

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA'¶: Encoding used to compress EXRs, only used when file_format='OPEN_EXR'

bit_depth: Literal[16, 32] = 32¶: Bit depth used for saving depth maps. Usually 32 or 16 bits for OPEN_EXR and 32 for HDR

__init__(preview: bool = True, file_format: Literal['OPEN_EXR', 'HDR'] = 'OPEN_EXR', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32) → None¶

class visionsim.simulate.config.NormalsConfig(preview: bool = True, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32)[source]¶

Bases: object

For more information see include_normals.

preview: bool = True¶: Also save colorized normal maps as PNGs

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA'¶: Encoding used to compress EXRs

bit_depth: Literal[16, 32] = 32¶: Bit depth used for saving normal maps

__init__(preview: bool = True, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32) → None¶

class visionsim.simulate.config.FlowsConfig(preview: bool = True, direction: Literal['forward', 'backward', 'both'] = 'forward', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32)[source]¶

Bases: object

For more information see include_flows.

preview: bool = True¶: Also save colorized flow maps as PNGs

direction: Literal['forward', 'backward', 'both'] = 'forward'¶: Direction of flow to colorize for preview visualization. Only used when preview is true

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA'¶: Encoding used to compress EXRs

bit_depth: Literal[16, 32] = 32¶: Bit depth used for saving flow maps

__init__(preview: bool = True, direction: Literal['forward', 'backward', 'both'] = 'forward', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32) → None¶

class visionsim.simulate.config.SegmentationsConfig(preview: bool = True, shuffle: bool = True, seed: int = 1234, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32)[source]¶

Bases: object

For more information see include_segmentations.

preview: bool = True¶: Also save colorized segmentation maps as PNGs

shuffle: bool = True¶: Shuffle preview colors, helps differentiate object instances

seed: int = 1234¶: Random seed used when shuffling colors

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA'¶: Encoding used to compress EXRs

bit_depth: Literal[16, 32] = 32¶: Bit depth used for saving segmentation maps

__init__(preview: bool = True, shuffle: bool = True, seed: int = 1234, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32) → None¶

class visionsim.simulate.config.MaterialsConfig(preview: bool = True, shuffle: bool = True, seed: int = 1234, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32)[source]¶

Bases: object

For more information see include_materials.

preview: bool = True¶: Also save colorized material passes as PNGs

shuffle: bool = True¶: Shuffle preview colors, helps differentiate material instances

seed: int = 1234¶: Random seed used when shuffling colors

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA'¶: Encoding used to compress EXRs

bit_depth: Literal[16, 32] = 32¶: Bit depth used for saving material maps

__init__(preview: bool = True, shuffle: bool = True, seed: int = 1234, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32) → None¶

class visionsim.simulate.config.DiffusePassConfig(file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] = 'OPEN_EXR', color_mode: Literal['BW', 'RGB', 'RGBA'] = 'RGB', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[8, 16, 32] = 32)[source]¶

Bases: object

For more information see include_diffuse_pass.

file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] = 'OPEN_EXR'¶: File format used to save diffuse passes

color_mode: Literal['BW', 'RGB', 'RGBA'] = 'RGB'¶

grayscale, color or color+alpha

Type:: Mode to save diffuse passes in

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA'¶: Encoding used to compress EXRs

bit_depth: Literal[8, 16, 32] = 32¶: Bit depth used for saving diffuse passes

__init__(file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] = 'OPEN_EXR', color_mode: Literal['BW', 'RGB', 'RGBA'] = 'RGB', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[8, 16, 32] = 32) → None¶

class visionsim.simulate.config.SpecularPassConfig(file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] = 'OPEN_EXR', color_mode: Literal['BW', 'RGB', 'RGBA'] = 'RGB', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[8, 16, 32] = 32)[source]¶

Bases: object

For more information see include_specular_pass.

file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] = 'OPEN_EXR'¶: File format used to save specular passes

color_mode: Literal['BW', 'RGB', 'RGBA'] = 'RGB'¶

grayscale, color or color+alpha

Type:: Mode to save specular passes in

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA'¶: Encoding used to compress EXRs

bit_depth: Literal[8, 16, 32] = 32¶: Bit depth used for saving specular passes

__init__(file_format: Literal['BMP', 'IRIS', 'PNG', 'JPEG', 'JPEG2000', 'TARGA', 'CINEON', 'DPX', 'OPEN_EXR', 'HDR', 'TIFF', 'WEBP'] = 'OPEN_EXR', color_mode: Literal['BW', 'RGB', 'RGBA'] = 'RGB', exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[8, 16, 32] = 32) → None¶

class visionsim.simulate.config.PointsConfig(preview: bool = True, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32)[source]¶

Bases: object

For more information see include_points.

preview: bool = True¶: Also save colorized point maps as PNGs

exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA'¶: Encoding used to compress EXRs

bit_depth: Literal[16, 32] = 32¶: Bit depth used for saving point maps

__init__(preview: bool = True, exr_codec: Literal['NONE', 'PXR24', 'ZIP', 'PIZ', 'RLE', 'ZIPS', 'DWAA', 'DWAB'] = 'DWAA', bit_depth: Literal[16, 32] = 32) → None¶

class visionsim.simulate.config.RenderConfig(executable: 'Path | None' = None, height: 'int | None' = None, width: 'int | None' = None, include_composites: 'bool' = False, composites: 'CompositesConfig' = <factory>, include_frames: 'bool' = True, frames: 'FramesConfig' = <factory>, include_depths: 'bool' = False, depths: 'DepthsConfig' = <factory>, include_normals: 'bool' = False, normals: 'NormalsConfig' = <factory>, include_flows: 'bool' = False, flows: 'FlowsConfig' = <factory>, include_segmentations: 'bool' = False, segmentations: 'SegmentationsConfig' = <factory>, include_materials: 'bool' = False, materials: 'MaterialsConfig' = <factory>, include_diffuse_pass: 'bool' = False, diffuse_pass: 'DiffusePassConfig' = <factory>, include_specular_pass: 'bool' = False, specular_pass: 'SpecularPassConfig' = <factory>, include_points: 'bool' = False, points: 'PointsConfig' = <factory>, include_all: 'bool' = False, previews: 'bool' = True, keyframe_multiplier: 'float' = 1.0, timeout: 'int' = -1, autoexec: 'bool' = True, device_type: "Literal['cpu', 'cuda', 'optix', 'metal']"='optix', adaptive_threshold: 'float' = 0.05, max_samples: 'int' = 256, use_denoising: 'bool' = True, log_dir: 'Path' = PosixPath('logs'), allow_skips: 'bool' = True, unbind_camera: 'bool' = False, use_animations: 'bool' = True, use_motion_blur: 'bool | None' = None, addons: 'list[str] | None' = None, jobs: 'int' = 1, autoscale: 'bool' = False, max_job_vram: 'MemSize | None' = None)[source]¶

Bases: object

executable: Path | None = None¶: Path to blender executable

__init__(executable: Path | None = None, height: int | None = None, width: int | None = None, include_composites: bool = False, composites: CompositesConfig = <factory>, include_frames: bool = True, frames: FramesConfig = <factory>, include_depths: bool = False, depths: DepthsConfig = <factory>, include_normals: bool = False, normals: NormalsConfig = <factory>, include_flows: bool = False, flows: FlowsConfig = <factory>, include_segmentations: bool = False, segmentations: SegmentationsConfig = <factory>, include_materials: bool = False, materials: MaterialsConfig = <factory>, include_diffuse_pass: bool = False, diffuse_pass: DiffusePassConfig = <factory>, include_specular_pass: bool = False, specular_pass: SpecularPassConfig = <factory>, include_points: bool = False, points: PointsConfig = <factory>, include_all: bool = False, previews: bool = True, keyframe_multiplier: float = 1.0, timeout: int = -1, autoexec: bool = True, device_type: Literal['cpu', 'cuda', 'optix', 'metal']='optix', adaptive_threshold: float = 0.05, max_samples: int = 256, use_denoising: bool = True, log_dir: Path = PosixPath('logs'), allow_skips: bool = True, unbind_camera: bool = False, use_animations: bool = True, use_motion_blur: bool | None = None, addons: list[str] | None = None, jobs: int = 1, autoscale: bool = False, max_job_vram: Annotated[int, ~tyro.constructors._primitive_spec.PrimitiveConstructorSpec(nargs=1, metavar=BYTES, instance_from_str=~visionsim.types._bytes_from_str, is_instance=~visionsim.types.<lambda>, str_from_instance=~visionsim.types._bytes_to_str, choices=None, _action=None)] | None = None) → None¶

height: int | None = None¶: Height of rendered frames

width: int | None = None¶: Width of rendered frames

include_composites: bool = False¶: If true, enable composited outputs

composites: CompositesConfig¶: Composited frames configuration options

include_frames: bool = True¶: If true, enable ground truth frame outputs

frames: FramesConfig¶: Ground truth frames configuration options

include_depths: bool = False¶: If true, enable depth map outputs

depths: DepthsConfig¶: Depth maps configuration options

include_normals: bool = False¶: If true, enable normal map outputs

normals: NormalsConfig¶: Normal maps configuration options

include_flows: bool = False¶: If true, enable optical flow outputs

flows: FlowsConfig¶: Optical flow configuration options

include_segmentations: bool = False¶: If true, enable segmentation map outputs

segmentations: SegmentationsConfig¶: Segmentation maps configuration options

include_materials: bool = False¶: If true, enable material map outputs

materials: MaterialsConfig¶: Material maps configuration options

include_diffuse_pass: bool = False¶: If true, enable diffuse light pass outputs

diffuse_pass: DiffusePassConfig¶: Diffuse light passes configuration options

include_specular_pass: bool = False¶: If true, enable specular light pass outputs

specular_pass: SpecularPassConfig¶: Specular light passes configuration options

include_points: bool = False¶: If true, enable world-space point map outputs

points: PointsConfig¶: Point maps configuration options

include_all: bool = False¶: If true, enable all ground truth outputs

previews: bool = True¶: If false, disable all preview visualizations of auxiliary outputs

keyframe_multiplier: float = 1.0¶

2.0 will slow down time

Type:: Stretch keyframes by this amount, eg

timeout: int = -1¶: Maximum allowed time in seconds to wait to connect to render instance

autoexec: bool = True¶

potentially dangerous)

Type:: If true, allow python execution of embedded scripts (warning

device_type: Literal['cpu', 'cuda', 'optix', 'metal'] = 'optix'¶: Name of device to use, one of “cpu”, “cuda”, “optix”, “metal”, etc

adaptive_threshold: float = 0.05¶: Noise threshold of rendered images, for higher quality frames make this threshold smaller. The default value is intentionally a little high to speed up renders

max_samples: int = 256¶: Maximum number of samples per pixel to take

use_denoising: bool = True¶: If enabled, a denoising pass will be used

log_dir: Path = PosixPath('logs')¶: Directory to use for logging

allow_skips: bool = True¶: If true, skip rendering a frame if it already exists

unbind_camera: bool = False¶: Free the camera from it’s parents, any constraints and animations it may have. Ensures it uses the world’s coordinate frame and the provided camera trajectory

use_animations: bool = True¶: Allow any animations to play out, if false, scene will be static

use_motion_blur: bool | None = None¶: Enable realistic motion blur. cannot be used if also rendering optical flow

addons: list[str] | None = None¶: List of extra addons to enable

jobs: int = 1¶: Number of concurrent render jobs

autoscale: bool = False¶: Set number of jobs automatically based on available VRAM and max_job_vram when enabled

max_job_vram: _bytes_to_str, choices=None, _action=None)] | None = None¶: Maximum allowable VRAM per job in bytes (limit is not enforced, simply used for autoscale)

visionsim.simulate.install module¶

visionsim.simulate.job module¶

visionsim.simulate.job.render_job(client: BlenderClient | BlenderClients, blend_file: str | PathLike, root: str | PathLike, *, config: RenderConfig = RenderConfig(executable=None, height=None, width=None, include_composites=False, composites=CompositesConfig(file_format=None, color_mode=None, exr_codec=None, bit_depth=None), include_frames=True, frames=FramesConfig(file_format='PNG', color_mode='RGB', exr_codec='DWAA', bit_depth=8), include_depths=False, depths=DepthsConfig(preview=True, file_format='OPEN_EXR', exr_codec='DWAA', bit_depth=32), include_normals=False, normals=NormalsConfig(preview=True, exr_codec='DWAA', bit_depth=32), include_flows=False, flows=FlowsConfig(preview=True, direction='forward', exr_codec='DWAA', bit_depth=32), include_segmentations=False, segmentations=SegmentationsConfig(preview=True, shuffle=True, seed=1234, exr_codec='DWAA', bit_depth=32), include_materials=False, materials=MaterialsConfig(preview=True, shuffle=True, seed=1234, exr_codec='DWAA', bit_depth=32), include_diffuse_pass=False, diffuse_pass=DiffusePassConfig(file_format='OPEN_EXR', color_mode='RGB', exr_codec='DWAA', bit_depth=32), include_specular_pass=False, specular_pass=SpecularPassConfig(file_format='OPEN_EXR', color_mode='RGB', exr_codec='DWAA', bit_depth=32), include_points=False, points=PointsConfig(preview=True, exr_codec='DWAA', bit_depth=32), include_all=False, previews=True, keyframe_multiplier=1.0, timeout=-1, autoexec=True, device_type='optix', adaptive_threshold=0.05, max_samples=256, use_denoising=True, log_dir=PosixPath('logs'), allow_skips=True, unbind_camera=False, use_animations=True, use_motion_blur=None, addons=None, jobs=1, autoscale=False, max_job_vram=None), frame_start: int | None = None, frame_end: int | None = None, frame_step: int | None = None, output_blend_file: str | PathLike | None = None, dry_run: bool = False, update_fn: UpdateFn | None = None) → None[source]¶

Render a sequence from a given blender-file.

Parameters:

client (BlenderClient | BlenderClients) – The blender client(s) which will be used for rendering. These should already be connected to a BlenderServer, and will get automagically passed in when using this function with BlenderClients.pool or similar.
blend_file (str | os.PathLike) – Path to blender file to use.
root (str | os.PathLike) – Location at which to save all outputs.
config (RenderConfig) – Render configuration.
frame_start (int | None, optional) – Frame index to start capture at (inclusive). If None, use start of animation range.
frame_end (int | None, optional) – frame number to stop capture at (inclusive). If None, use end of animation range.
frame_step (int | None, optional) – Step with which to capture frames. If None, use step of animation range.
output_blend_file (str | os.PathLike | None, optional) – If set, write the modified blend file to this path. Helpful for troubleshooting. Defaults to not saving.
dry_run (bool, optional) – If enabled, do not render any frames or ground truth annotations.
update_fn (UpdateFn | None, optional) –
callback function to track render progress. Will first be called with total kwarg, indicating number of steps to be taken, then will be called with advance=1 at every step. Closely mirrors the rich.Progress API.

visionsim.simulate.schema module¶

Module contents¶

visionsim.simulate.install_dependencies(executable: str | PathLike | None = None, editable: bool = False) → CompletedProcess[source]¶

Install additional packages into blender`s runtime.

Parameters:

executable (str | os.PathLike | None, optional) – Same as BlenderServer.spawn. Defaults to None.
editable – (bool, optional): If set, install current visionsim as editable in blender. Only works if visionsim is already installed as editable locally.