we saw in action the ChatGPT which helps you finish your games and much more

We went to Nvidia’s European headquarters in Munich to test Nvidia’s latest AI innovations. It was its Project G-Assist that caught our eye, an assistant with amazing potential for gamers, but not only.

For his April Fool’s Day 2017, Nvidia presented the fictional project “G-Assist”a USB flash drive in the shape of a GeForce GTX graphics card that “ leverages NVIDIA’s cutting-edge artificial intelligence to bring you the next revolution in gaming “. A technology to play for you, in short.

Eight years later, Nvidia has become one of the most valuable companies in the world, the undisputed leader in AI for businesses, but also for gamers since the arrival of DLSS in 2022. If the dream of G-Assist is not (yet) a reality, the project was talked about again at Computex 2024 as part of a promising technology demo: a real AI assistant to help you in your games and applications.

As part of a trip to Berlin to the European headquarters of Nvidia, we were able to once again experience this “proof of concept” which still raises so many questions.

A universal tutorial for your games

If this “G-Assist Project” is presented to us as a simple technological demonstration, its operation is very real and is based in part on the architecture of GeForce RTX graphics cards. The software thus takes into account the player’s requests, whether sent by text or voice, to send it in this case to a remote server in the United States. The model thus takes into account the player’s request and the game context, namely the game window currently being displayed.

So in the game ARK: Survival Ascended who partnered with Nvidia for this demonstration, the player can then ask for help to get started in the game, including how to tame a dinosaur and which weapon to use. The assistant then detects the objects present in the quick inventory, but also the types of dinosaurs present on the screen to guide the player towards the procedure to follow.

Similarly, by displaying their full character sheet and the rest of their inventory, we could then ask which character indicator to level up first. G-Assist then told us to level up health and stamina first, advice that is also valid for many RPGs in the first hours of play. We can easily imagine the same scenario in a game like Fire Ringin which G-Assist could advise leveling up on the Strength or Dexterity categories depending on the player’s preferred weapons.

If the answers remain only vocal and textual, Nvidia presented us with a fairly wide range of possibilities: it will be possible, in collaboration with the developers, to display visual indications, images of the game or diagrams to enrich the help provided by G-Assist. It would even be possible to show entire game sequences according to the request, based in particular on the chaptering of the YouTube videos used to train the LLM.

It is this work with developers and publishers that will be decisive in the realization of the project, because they will be the link between G-Assist and the community. Most of the wikis and knowledge bases on which the assistant will be based are overwhelmingly the work of the players themselves, so it is their solutions that will be the basis for training the AI ​​model behind G-Assist. These same contents are also cited as sources for more complex answers to players wanting to go further.

Detecting performance issues

But this is not the only use case for G-Assist, which can give you advice to optimize the performance of your games according to your graphics settings as well as your configuration. Here, the tool acts as a deep integration of GeForce Experience, and more recently Nvidia App, based on the Game Ready profiles integrated into the Nvidia driver with each update.

But the integration goes even further, showing you an in-depth analysis of your performance and energy consumption history while offering you optimization measures. If you were used to using a tool like Rivatuner to come to your own conclusion, G-Assist wants to do the work for you here.

G-Assist acts as an assistant connected to Nvidia App or Geforce Experience, defining the graphic profile you want according to your requests. No need to go to the control panel, the Nvidia settings, but also the game settings are changed in real time. It can even offer you overclocking or even undervolting settings to better manage the consumption of your hardware.

We regret, however, that the tool does not allow you to check certain Windows parameters, such as the different programs launched in the background or the power mode. However, Nvidia representatives on site gave us to understand that it might be possible to connect third-party LLMs to G-Assist, so we can imagine compatibility with the Windows Copilot assistant in a fantasy version of the assistant.

An assistant for your games… and your applications?

Let’s remember that in this form, G-Assist is only an embryo of a solution still being designed at Nvidia. Its functionalities are not yet fixed, the firm counting on the cooperation of developers to allow its software to reach its full potential.

Remember Clippy in Microsoft Office? G-Assist could well be its worthy successor, and this, on any application. Imagine yourself on Photoshop or Blender with an assistant helping you step by step to learn the basics of 3D creation or photo editing.

Currently, you can ask ChatGPT, Copilot or Gemini for help in mastering your software, the different models looking for their answers in the official manuals and wikis. The integration of G-Assist would be a real plus in their learning or simply the occasional help provided to their users.

Here again, Nvidia is not closing any doors on the possibilities offered by G-Assist. Its hybrid operation between cloud and local opens it to a multitude of use cases. It is still necessary, once again, to count on the involvement of developers. In the meantime, Nvidia’s secret weapon remains an idea that will still require a little work before it becomes a reality.