Developer Analytics: An Introduction

All developers rely on analytics to launch, optimize and monitor their applications. This is typically a daily, if not an hourly activity. Data rocks!

Most developers will have a third-party or custom analytics solutions for measuring core application usage like session behavior. These solutions answer questions like: “how long does it take a person to get to the next level” or “what percentage of people take a particular action within the application.” Third-party or custom analytics triggers are identified in the game mechanics and provide insightful data on how you can improve things like game play or sentiment around your application. Because these solution are robust and give you all the necessary information you need for optimizing your application holistically, Oculus Analytics focus on platform-specific metrics to help you make more informed decisions about your business on Oculus.

Visit the Developer Dashboard and you'll see “Analytics” in the left-side navigation for each of your applications. This section provides you important information about your app's performance on Oculus. We hope to provide the necessary platform-specific metrics to ultimately help you make more informed decisions based on how your app is performing in the Oculus Store and on Oculus devices, how Oculus users are using your app and what they think of your application and other core metrics. In addition, the rich insights include data filters, real-time processing, app version filtering and the ability to export the data to merge these analytics with your core app metrics.

Data filters provide the ability to dive deeper into your key analytics. Filters for mobile applications by carrier and various device breakdowns, while PC developers can view key analytics by GPU or CPU breakdowns. All devs can view and sort analytics by a variety of date ranges, hour or day, app version, OS, or country.

Real-time data is very important when making time-sensitive decisions about your application. You can learn about things like unexpected crashes or CPU spikes hourly. We hope these insights help you iterate faster to make your applications more reliable, enjoyable, and effective.

You can export the data to csv to better synthesize with internal metrics for a more holistic view of app performance.

The following provides definitions for each metric and how they can be filtered based on whether your application is free or paid.

Lifetime installs: Installs are the total number of times your application has been installed. This includes users that previously uninstalled your app and users installing on multiple devices.

Lifetime IAP revenue: In-app (IAP) revenue is net factoring in taxes and discounts, but excluding Oculus platform fees and refunds. This revenue is an approximate value and should not be used for accounting purposes.

Lifetime active users: Tracks total unique users. Each user is only counted once.

Lifetime copies sold: This view tracks the total number of app purchases in the Oculus Store. It does not include entitlements from keys redeemed, employee offers, press, release channels and organization members.

Lifetime revenue: Total revenue from copies sold. This Revenue is net factoring in taxes and discounts, but excluding Oculus platform fees and refunds. This revenue is an approximate value and should not be used for accounting purposes.

Lifetime active users: Tracks total unique users. Each user is only counted once.

Average device frame rate: Tracks the average frame rate. Low frame rates can indicate performance issues, judder, and stuttering. Users who experience poor frame rates are likely to perceive your app negatively.

Frame rate over session time: Tracks the average frame rate on a timeline that represents the first 180 minutes of the average session. When used with Breakdown set to Device Name, it is a powerful tool that helps you understand and optimize your app's real world performance on different devices.

Average battery burn rate: Tracks the battery consumption of your mobile app. High battery burn rates are undesirable because apps with high power consumption are more prone to overheating. Average play sessions are shorter because the device runs out of power faster and users typically dislike apps that have high battery usage. Using the Breakdown feature, you can isolate which device types of Gear VR are affected most by overheating, or data specific to Oculus Go.

Available memory: Tracks the amount of available memory while your mobile app is running. To see how different devices compare, use the Device Name breakdown. Available memory on a device can indicate how much memory remains for future app development. Unexpectedly low available memory can indicate memory leaks, problems with garbage collection, unstopped services, or other memory-related problems.

Client Lost Frames Last 7 Days: Tracks the percentage of Application Frames Dropped. Application Frames Dropped increments each time the application fails to submit a new set of layers using ovr_EndFrame before the compositor is executed before each V-Sync (Vertical Synchronization). This is identical to App Missed Submit Count in the Application Render Timing HUD. A high percentage of client lost frames indicates performance issues. Users who experience poor performance are likely to perceive your app negatively

Percentage of Compositor Lost Frames Per Frame: Tracks the percentage of Compositor Frames Dropped. Compositor Frames Dropped Increments each time the compositor fails to present a new rendered frame at V-Sync (Vertical Synchronization). This is identical to Compositor Missed V-Sync Count in the Compositor Render Timing HUD. A high percentage of compositor lost frames indicates performance issues. Users who experience poor performance are likely to perceive your app negatively

Median CPU Milliseconds Per Frame: Tracks the median CPU milliseconds spent to render a frame. A high value of CPU milliseconds per frame indicates performance issues. Users who experience poor performance are likely to perceive your app negatively.

Median GPU Milliseconds Per Frame: Tracks the median GPU milliseconds spent to render a frame. A high value of GPU milliseconds per frame indicates performance issues. Users who experience poor performance are likely to perceive your app negatively.

Overheat events: Tracks the number of times your mobile app has overheated a device. Overheating events shorten the average session length of your users and have negatively affect your application's perceived quality. They can also be a source of negative reviews which can affect sales.

Number of users with install failures: Tracks the number of users who couldn't install your mobile app the first time. This metric does not track failures for updates to your application. Install failures are typically caused by lack of storage space. If your app is so large as to have a significant number of install failures, consider options, such as DLC for reducing the size of your app.

Gamepad usage: Tracks the percentage of users using gamepad controllers with your mobile app. Many users do not have gamepads, so consider implementing touchpad-compatible modes to access the widest audience.

Headphones usage: Tracks the percentage of users using headphones with your mobile app. If your app experience requires headphone usage, consider displaying an awareness message after your application has loaded to its main menu suggesting that users "Use headphones for the best VR experience."

Gear VR Controller Usage: Tracks the percentage of users using Gear VR controllers with your Gear VR application.

7 day active users: Number of unique users who used your app in the last 7 days. Each user is only counted once during the time period. 7 Day Active provides a better overview of engagement by smoothing out the fluctuations between weekend and weekday usage. If your app has low active users, consider adding social features like Voice Chat (VoIP), Events, Leaderboards, Destinations + Rich Presence, and more.

Average session length: Tracks the amount of time a user spends during the average app session. Session length is an engagement indicator. If your app has short session lengths, consider adding social features that encourage your audience to stay in your app for longer periods of time. If the device breakdown reveals significantly reduced session lengths for specific devices, this may indicate a performance or overheating issue specific to that device type and we would recommend checking for errors and optimizing the performance for the devices encountering these issues.

Active users: Tracks daily unique users. Each user is only counted once daily and all data is aggregated to the Pacific Time Zone. Activity is an engagement indicator. Following user activity lets you measure the impact of new versions and releases of your app. If your app has low active users, consider adding social features, marketing your application, or adding additional content with DLCs to drive user interest.

Number of uninstalls: Tracks the number of times your app was uninstalled.

Number of installs: Installs are the total number of times your application has been installed on a device. This includes users that previously uninstalled your app and are installing again and users installing on multiple devices. The number of installs indicates the adoption rate of your app. A downward trend in adoption can be countered by providing incentives for installing your app such as discounts and advertising.

Number of entitlements: The number of entitlements is a sum of all users who have been granted access to the application. Entitlements come from copies sold, keys redeemed, users in release channels, and members of your organization. There are certain cases where temporary entitlements will be granted to Oculus employees and press.

Number of keys redeemed: Compares the number of keys generated with those keys which have been redeemed by users. These metrics are only available for apps with generated keys.

In app revenue: In-app (IAP) revenue is net tax and discounts, excluding Oculus platform fees. This revenue is an approximate value and should not be used for accounting purposes. Chart only includes IAP data starting from November 10th 2016. Compare your IAP statistics to active users and session length to gain insight into IAP trends. This metric is only available for apps with IAP.

Refunds: Tracks the lost revenue from refunded app copies and refunded IAP.

Revenue: Revenue is net tax and discounts, excluding in-app revenue and Oculus platform fees. This revenue is an approximate value and should not be used for accounting purposes. Chart only includes revenue data starting from November 10th 2016. Revenue trends influence your decisions on making pricing or marketing changes. For example, if revenue is trending down, you may consider investing more in marketing or reducing the price of the app.

Lifetime click through rate: Percentage of notifications that were clicked on. Generally the higher the CTR the more effective your notifications have been at bringing people to your app. CTR = (Lifetime Number of clicks / Lifetime Number of sends) * 100(%).

Notifications sent: Tracks the number of notifications that have been sent to your users.

Impressions: Impressions are the number of times your notifications have been viewed in a user's notifications feed.

Clicks: Clicks are the number of times your notifications have been clicked on in a user's notifications feed.

Unsubscribed: Number of people who have unsubscribed for your app's notifications. People who have unsubscribed will no longer receive notifications from your app.

Lists and displays user reviews of your application sorted by most helpful, most recent, or most critical. Also gives you the ability to respond to individual reviews.

Our analytics views can be set to display data from a variety of date ranges. You can sort by a custom date range and a life-to-date view as shown below, in addition to yearly, quarterly, monthly, all the way to today's date for full granular visibility into your application's performance on the Oculus Store.

In the custom view example above and to the left, we selected February 1st and then February 24th to gather data for this specified chart range. Hitting update afterwards populates the chart with the requested data in your dashboard.

The life-to-date date range on the right provides all data points for your application from the date it went live on the Oculus Store to the date you queried the data.

The following example is an imaginary Gear VR application with a purchase price of $1.99 per copy sold. Our example application, Roll-a-Rectangle PRO, has now been published on the Oculus Store for two months.

Below are the three main financial analytic views showing 28 days of data for January 2018 for Roll-a-Rectangle PRO that propagate in the Overview section of its Analytics section. The charts below show the copies sold, total revenue* over time, and the number of active users engaging with the application.

* Revenue in this example does not reflect taxes, discounts, or platform fees.

From the first 28 days that our example application has been live on the Oculus Store, it has sold 100 copies and accrued $199.00 in revenue with a total of 80 active users. The chart is also able to display dialed in metrics by the day, and shows us that January 3rd we sold 54 copies, accrued $106.46 in revenue and that 37 users were actively playing the application that day. Since this is the first month on the store for our example application, each chart reflects a 0% change in month over month data.

One month later and after taking into account some user reviews on our application asking for more content to be added to the application, an updated version of Roll-a-Rectangle PRO is released with new content in the first week of February 2018.

The update results in an increase of installs on February 11th with a total of 97 copies sold, $193.03 in accrued revenue, and 84 active users present in the application. Keeping track of active users after releasing new content is a good indicator of the retentiveness of this updated content, as more active users in your application means users are continually revisiting your application over time.

In our example below, active users started the month of February with a moderate increase and then rose quickly with the new content addition to a high of 84 active users, ultimately base-lining with a few more surges in daily active users toward the end of the month. We can infer from this scenario that new and existing users interacted with this application to a greater extent than the previous month, but the content may not have been compelling enough to drive user retention on a stable or increased vector.

The performance overview analytic charts are helpful in diagnosing hardware issues with your application. These charts are averages of all tallied data from each unique device spending at least one minute of time in your application.

For our example application, our average FPS was not too far off from the 60 FPS required of Gear VR applications and only 1.42% of all devices suffered an overheating issue in February 2018. Average time spent for this same month was 23 minutes a user and remained fairly steady throughout.

Correlations can be drawn from decreases in frame rate and increase in overheat events (if your application built for mobile devices like Gear VR and Oculus Go) with that of time spent in your application. For instance, if your application happened to be reporting a higher than average instance of overheating in a set time frame and average time spent in your application is also steadily going down in that same time frame, it would be wise to check your application to assess this issue and fix it if need be.

The user overview analytics charts display results correlating with user engagement with your application through notification campaigns you run, the average and type of review score your application is rated from 1-5 stars, and a breakdown from highest to lowest percentage of where your user base originates from, which is helpful for targeting any advertising campaigns you may wish to run for your application.

For February 2018, our example application has served 11 notification impressions to users this month for the new content that was published in February, has an average of 4.32 stars out of 22 total reviews, and shows that a majority of users are originating from the United States and the United Kingdom.

Our analytics suite is most useful as general guidance on how your application performs on the Oculus Store and should not be fully relied on for official financial accounting purposes. Data being reported in the charts is an approximation within 5% and may be delayed. The current day's data points contain partial data up to current pacific time.

Question: What is the maximum amount of time that we track lifetime statistics for?

Answer: Lifetime values are limited in the line graphs until November 2016 and when we need to make a note that the line will constantly shift to +2 years of data. Current date - two years because of FB infra. Numbers at top always are lifetime, but the line charts are limited because of a different database.

Question: Why do we baseline at 28 days?

Answer: We believe this timeframe provides the most relevant guidance for developers.

Question: Why is my free application showing a decrease in unique downloads since the new analytics rolled out?

Answer: There are some changes that were put into effect with the new analytics that are causing this discrepancy. We used to show entitlements on the top line for free apps. We are now showing installs. The difference is that users can get an entitlement by clicking on the "free" button and then never finish installing the app.

Question: What metrics are used to define an Active User?

Answer: We track the individual unique oculus IDs that buy or install your application. If a users happens to have multiple devices, this will still be reflected as one unique active user in the statistics.

Question: How do we capture data on lost frames in an application?

Answer: Real time capture of lost frames in your application is sourced from our SDK. Data is captured and logged every minute of application playtime per user. There are two different time of performance capture: client capture (developers can affect this behavior) and directly taken from the compositor.

Question: How accurate is our time spent in application data?

Answer: We use an arithmetic mean, so the data may be skewed in some cases. There needs to be greater than 60 seconds of in app behavior prior to our analytics generating an accurate read. After 3-4 hours we stop counting time spent.