Video modes

Interactive Video Modes

The "streaming" and "synchronous" render modes are designed for the interactive use case and support low-latency hardware H.264 encoding and decoding to make the most of the available bandwidth. These two render modes operate using a server-side render loop which is updated with client-side changes when calling render on the Iray Bridge render context. The server-side render loop will keep rendering in the background and picks up changes from the client, renders, and streams back images with the changes or with more iterations back to the client on a video stream.

The "synchronous" video mode works exactly like local rendering in that calling render with a transaction will return a canvas containing a rendering of the scene in the state of that transaction. This is required when for instance blending the result with other render contexts or if the resulting rendered frame must contain the changes in the passed in transaction for some other reason. While sometimes required, this comes at a potentially big loss of performance, especially for high latency connections. The reason for this is that the frame rate is not only dependent on how fast the server can render a frame, but also on the round trip time to the server, the time it takes to encode and decode the frame, and the time it takes to send the frame over the network. For a connection to a server with a round trip of 100 ms the frame rate is capped at 10 fps, even without considering any other cost.

The "streaming" video mode is designed to overcome this limitation by allowing the returned canvas to lag a little bit behind. Each call to the render method of the Iray Bridge render context will send any scene changes to the server, but instead of waiting for a frame that contains those changes the most recent frame from the server will be returned immediately. The server-side render loop constantly renders and streams frames to the client and will eventually pick up the changes. The client application needs to keep calling the render method of the Iray Bridge render context to keep streaming changes to the server and to display the most recent frame from the server. This works very well for interactive use cases and, together will hardware H.264 encoding and decoding, can achieve frame rates very close to the actual frame rate on the server if network conditions are good enough. The "streaming" video mode is not sensitive to network lag, but the bandwidth must be high enough to support the bit-rate of the video stream. Another factor that can cause performance problems is if the network tends to buffer data and deliver it in bursts.

The "streaming" and "synchronous" video modes also support other video formats such as jpg, as well as lossless formats like png and exr, but these image formats are very expensive to encode and decode and also use a lot more bandwidth than H.264, so for the best interactive experience, H.264 is recommended. It is possible to switch at any time between the supported video modes so streaming can be used until a frame is needed that is guaranteed to be up-to-date at which point "synchronous" can be used for that frame, and then switch back to "streaming".

The interactive video modes send frames over a video stream that only encodes a single canvas at a time making it impractical to separate out different aspects of the light in the same rendering as can be done using path expressions (LPEs). Simultaneous rendering of multiple canvases is, however, supported by the "batch" video mode.

Batch Video Mode

The "batch" video mode is intended to be used for final renderings with no or infrequent updates on the client before the final image is ready. This mode always uses a lossless video format for transmission of rendered frames. In contrast to the interactive video modes, the batch mode does not use Iray Bridge video streaming and does support multiple canvases rendered in one pass. This mode is therefore ideal for use together with light path expressions (LPEs).