A Look at Omniverse Kit, Nvidia’s Metaverse SDK – InApps 2022

Last week I looked at potential developer platforms for the metaverse, a 3D version of the web that has been much discussed this year. Although Facebook ignited the latest round of hype about the metaverse last month, another company, Nvidia, is much further along in its development of an actual metaverse platform. Nvidia’s Omniverse has been running since 2019, so in today’s post I’m going to take a look at its developer toolkit: Omniverse Kit.

As I noted last week, the main purpose of Omniverse is to allow engineers to collaborate on building a physical product by working together on a digital representation of that product. It’s a simulation and collaboration platform targeted at design and manufacturing companies.

Omniverse Kit is how a developer can create apps for the Omniverse. However, it is just one of five main components to the overall developer platform. The other four parts are Nucleus, Connect, Simulation and RTX Renderer (RTX is a high-end professional visual computing platform by Nvidia). The latter three components are fairly self-explanatory, but Nucleus takes some explaining. First, here is the architectural diagram:

In simple terms, Nucleus is a set of services that Omniverse apps, among other things, interact with. In a video introduction to Omniverse Kit, Damien Fagnou, a senior director of software at Nvidia, said that Nucleus “represents a coordination point between a lot of different applications.”

The Developer Stack

For developers, the Omniverse Kit stack looks like this:

As Fagnou noted in the video, “extreme modularity” and extensions are two key concepts of the Omniverse Kit. To explain, he showed how this works in an internal application built using Kit: Omniverse Create, a tool used for content creation on the platform. Rather than being one large, monolithic app, Create was built using many extensions (which are basically software modules).

“Instead of being one application, it is made of extensions,” said Fagnou. “Those extensions are called building blocks that you can assemble in many ways to create different types of apps. At the higher level, they are all written in Python to assemble the UI, the workflow and the general feature set.”

As one simple example of how this works in practice, the search module in Create is an extension that is connected to the Nucleus indexing service.

It’s All Built on USD

All of Omniverse, including Kit, is built on top of Pixar’s open source Universal Scene Description (USD). It’s both a file format and “a powerful scene representation with an API.”

As explained in another Nvidia video, Omniverse was the first application to be built entirely around USD. “This is not a format for import and export,” said Dirk Van Gelder, Nvidia’s director of software development (and before that, a 23-year veteran of Pixar), “the actual document that you’re working on is USD itself.”

An example is the easiest way to explain it. In one scene that Van Gelder showed in his video, he moved an animated horse around; each time the horse moved, a value changed in the USD.

Physics and the Role of Python

One of the critical parts of a 3D metaverse platform is its “physics,” meaning the way it simulates the physical world. Nvidia has several simulation SDKs, one of which is the physics SDK PhysX.

One of the core features of PhysX is the Character Controller (CCT), “a video-game-style movement controller that permits a user to move around a world without being able to go through Physics-collision-enabled objects” (walls, other people, etc.).

While developers can easily use a tool like Omniverse Create to add and edit a CCT, they can also “tweak” the CCT behavior through Python scripting — for example, the documentation states, “you can add a key binding and scripting to make the CCT jump.”

So Python developers perhaps have a leg up on the competition if they want to make the jump from web development to metaverse development. Nvidia’s recent job vacancies give an indication of what’s required if you’re a developer interested in going down the metaverse pathway. As well as a computer science degree, applicants to one recent Omniverse team job posting required “strong C++ and Python skills” and “strong knowledge of real-time systems such as games.”

Microservices

Finally, it’s worth taking note of the microservices architecture of Omniverse Kit. In a video by Jozef van Eenbergen, a senior software engineering manager at Nvidia, he explained that microservices are closely tied into the extensions of Kit.

“A service can be part of an extension, use an extension or be a simple script executed by Kit,” he said.

One example of how microservices are used in Kit is in the generation of thumbnail images. “Communication happens over http and the content browser instructs a set of thumbnail services to generate thumbnails if they do not exist,” said van Eenbergen.

Conclusion: Omniverse Is a Step Ahead

As you can see, the Omniverse developer platform is wide-ranging and has a lot of moving parts. But if you’re a developer familiar with Python and/or C++, and you have an interest in computer graphics, it’s not that much of a leap to pivot your career to the metaverse.

As of now, Nividia’s Omniverse seems like the perfect way for developers to segue into the 3D web, which as I mentioned at the top, is a trending market. Although Facebook is hiring too in its newly minted “metaverse product group,” it doesn’t yet have a developer platform. Nvidia, on the other hand, has a fully functioning metaverse platform built on top of proven open source technology from Pixar (USD).

Indeed, perhaps the two to three years head start that Nvidia has on Facebook and others will result in the Omniverse becoming the leading gateway to the metaverse for developers. That remains to be seen, since Omniverse is currently targeted at an enterprise market, but either way I’d encourage interested developers to check out the Omniverse Kit.