NVIDIA generative AI tech can let you have real convos with NPCs

NVIDIA has introduced Avatar Cloud Engine (ACE), a technology that lets gamers have natural conversations with non-playable characters (NPCs) in games. The company showcased the tech at Computex 2023 with a demo called Kairos, where a player chats with an NPC named Jin in a futuristic Ramen shop.

The demo shows how the player can ask Jin about his life and get relevant answers. For example, the player says: “Hey Jin, how are you?” Jin replies: “Unfortunately, not so good. I am worried about the crime around here. It’s gotten bad lately. My ramen shop got caught in the crossfire.”

The dialogue is not very engaging, but the idea is to enable natural, context-based interactions. NVIDIA developed the demo with Convai to promote ACE. It can run on the cloud or locally (on NVIDIA hardware, of course). And it uses NVIDIA NeMo to create and customize large language models with game-specific information and filters to prevent inappropriate conversations. It also uses Riva, a speech recognition and speech-to-text tool, and NVIDIA’s Omniverse Audio2Face to generate realistic facial animations for game characters based on speech.

The demo was made in Unreal Engine 5 to highlight NVIDIA’s ray-tracing and other GPU features. The graphics are more impressive than the AI dialogue, but there is potential for improvement. NVIDIA did not announce any games that would use the tech, but Stalker 2: Heart of Chernobyl and Fort Solis will use Omniverse Audio2Face.

Source