Meta Developers

OpenXR does not allow direct control over display time for images in swapchains, and it does not use frame indexes to manage the display sequence. It provides predicted frame times, but not frame numbers. To submit frames, you must call xrWaitFrame, xrBeginFrame, and xrEndFrame in that order.

The xrWaitFrame function call provides a frame timing mechanism. This call synchronizes frame submission with the display, returning a predicted display time (predictedDisplayTime) as a timestamp for the next predicted time a composited frame is expected to display. Render the frame for the predicted state of the world at that time. Call xrWaitFrame before rendering the frame, as it is a blocking call that waits for the previous frame to be marked as ready-to-render by xrBeginFrame.

The xrWaitFrame call allows the runtime to control the frame cadence. Each xrWaitFrame call must be matched with an xrBeginFrame call for the same frame. Call xrBeginFrame before starting frame rendering, as it marks the beginning of the rendering process for that frame.

Important: Match each xrBeginFrame call with its associated xrWaitFrame call to render the next frame. Failing to match these calls (e.g., calling xrWaitFrame twice without an xrBeginFrame call in between) will cause the second xrWaitFrame call to block indefinitely.

The xrEndFrame call attempts to submit the frame by submitting all composition layers. When calling xrEndFrame for the same frame, pass the predicted display time returned by xrWaitFrame. This allows the runtime to determine what the frame was rendered for, as there are no frame numbers.

The wait-begin-end Cycle

After xrWaitFrame, the frame is not rendered until xrBeginFrame is called, but it does have a predicted display time. At this point, you can implement your own timing logic and manage GPU work. Then, xrBeginFrame marks the frame as ready-to-render. This frame will be the only one that:

Has a predictedDisplayTime timestamp from xrWaitFrame.
Has not been marked as ready-to-render by xrBeginFrame.

OpenXR does not support consecutive xrWaitFrame calls without an xrBeginFrame call in between. Without an xrBeginFrame call, the runtime cannot mark the frame for rendering, as there are no frame indexes. This also means that one frame can have a predictedDisplayTime from xrWaitFrame before xrBeginFrame marks the start of its rendering.

Note: Although predictedDisplayTime is always increasing, it is not a unique identifier for frames. This timestamp is an expected display time, and the runtime may update the actual display time to support the app.

You must call xrWaitFrame, xrBeginFrame, and xrEndFrame functions in that order as if they are single threaded. If the app calls xrBeginFrame without having previously called xrEndFrame, the previous frame is no longer useful and gets discarded.

This wait-begin-end cycle happens in Khronos’s hello_xr sample app as follows:

XrFrameWaitInfo frameWaitInfo{XR_TYPE_FRAME_WAIT_INFO};
XrFrameState frameState{XR_TYPE_FRAME_STATE};
CHECK_XRCMD(xrWaitFrame(m_session, &frameWaitInfo, &frameState));

XrFrameBeginInfo frameBeginInfo{XR_TYPE_FRAME_BEGIN_INFO};
CHECK_XRCMD(xrBeginFrame(m_session, &frameBeginInfo));

std::vector<XrCompositionLayerBaseHeader*> layers;
...

XrFrameEndInfo frameEndInfo{XR_TYPE_FRAME_END_INFO};
frameEndInfo.displayTime = frameState.predictedDisplayTime;
frameEndInfo.environmentBlendMode = m_environmentBlendMode;
frameEndInfo.layerCount = (uint32_t)layers.size();
frameEndInfo.layers = layers.data();
CHECK_XRCMD(xrEndFrame(m_session, &frameEndInfo));

Where m_session is an XrSession handle. The layers pointer vector to the XrCompositionLayerBaseHeader struct defines current and future structs that contain information about the composition layer:

XrCompositionLayerProjection layer{XR_TYPE_COMPOSITION_LAYER_PROJECTION};
std::vector<XrCompositionLayerProjectionView> projectionLayerViews;
if (frameState.shouldRender == XR_TRUE) {
    if (RenderLayer(frameState.predictedDisplayTime, projectionLayerViews, layer)) {
        layers.push_back(reinterpret_cast<XrCompositionLayerBaseHeader*>(&layer));
    }
}

The XrCompositionLayerProjection struct represents projected images rendered per view and the XrCompositionLayerProjectionView struct contains info such as field of view, location, and orientation of a projection element. The logic of rendering layers then follows. In hello_xr, this is defined in the RenderLayer function.

Rendering Layers

OpenXR apps draw frames by submitting layers. Compositors combine the layers and GPU draws the frame.

When the hello_xr app runs, it receives the predicted location of the camera/view/head so that it can render from the correct place. The app locates the space and the output is an XrSpaceLocation struct. The space gets located by calling the xrLocateSpace function which receives the pose of the origin of an XrSpace within a base XrSpace at a given time (actual or predicted).

To do so, the app uses the XrViewLocateInfo struct and the xrLocateViews function to return the viewer pose and projection parameters for rendering each view to use in a projection layer.

XrViewLocateInfo viewLocateInfo{XR_TYPE_VIEW_LOCATE_INFO};
viewLocateInfo.viewConfigurationType = m_viewConfigType;
viewLocateInfo.displayTime = predictedDisplayTime;
viewLocateInfo.space = m_appSpace;

res = xrLocateViews(m_session, &viewLocateInfo, &viewState, viewCapacityInput, &viewCountOutput, m_views.data());
CHECK_XRRESULT(res, "xrLocateViews");

The app draws a cube at the center of a visualized space. It loops through all visualized spaces, locates the space, and then draws the cube. Being just a sample app, hello_xr draws cubes at these locations, so that users can see where the origin of these common spaces are. However, this is not something an app would commonly do.

// For each locatable space that we want to visualize, render a 25cm cube.
std::vector<Cube> cubes;

for (XrSpace visualizedSpace : m_visualizedSpaces) {
    XrSpaceLocation spaceLocation{XR_TYPE_SPACE_LOCATION};
    res = xrLocateSpace(visualizedSpace, m_appSpace, predictedDisplayTime, &spaceLocation);
    CHECK_XRRESULT(res, "xrLocateSpace");
    if (XR_UNQUALIFIED_SUCCESS(res)) {
        if ((spaceLocation.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) != 0 &&
            (spaceLocation.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT) != 0) {
            cubes.push_back(Cube{spaceLocation.pose, {0.25f, 0.25f, 0.25f}});
        }
    } else {
        Log::Write(Log::Level::Verbose, Fmt("Unable to locate a visualized reference space in app space: %d", res));
    }
}

Where m_visualizedSpaces is a vector of XrSpace, initially defined as:

std::vector<XrSpace> m_visualizedSpaces;

XrSpace handles represent space. In OpenXR, functions that return coordinates require an XrSpace parameter to define the reference frame for these coordinates. Conversely, passing coordinates to a function requires XrSpace so that the runtime interprets these coordinates.

Note: There’s a difference between an XrSpace handle and a reference space. In OpenXR, apps can locate a space in multiple reference spaces, by asking about the location of one space in any other space. In some scenarios, the runtime won’t know and there might not be an answer. Reference spaces are a special category of runtime-defined spaces with a well-defined behavior (related to the real world in all current uses). Commonly, calling xrLocateSpace will be to locate one space in a reference space (possibly with an offset).

All code snippets in this document belong to hello_xr sample app, which is developed by The Khronos Group Inc. and licensed under the Apache License, Version 2.0.

Did you find this page helpful?