Back
R&D
The experiments, offered on different platforms and immersive screens, will complement the work carried out during the previous edition on immersive spaces, in particular by integrating the playback of matches in 3D.
The set up put in place for this 2023 edition has been imagined and designed by France Télévisions, with French and foreign technological partners. Five demonstrators, developed by the teams of the innovation department, around 3 emerging subjects are exhibited on the terrace of the RGLab throughout the fortnight:
- Playback of matches in immersive spaces using motion capture
- Volumetric video on immersive devices
- AI as a virtual assistant
At the previous edition of the RGLab, the innovation department presented an immersive social space at Stade 2 - Roland Garros on the VRChat social network in virtual reality. The alleys of the stadium and the Philippe Chatrier court were reconstructed in 3D to provide the setting for mini-games, a museum, a watchparty area and gatherings around the event. However, these initial demonstrations of virtual worlds did not faithfully reproduce what happens on the Roland Garros courts. That's what's going to be shown this year.
Playback of matches in 3D
A first experience of Roblox
Creating a first experience on Roblox for Roland Garros will enable France Télévisions to familiarise itself with this platform, which attracts a very young audience, to better understand it and imagine future offers. The experiment that has been developed represents a bridge between France Télévisions' traditional coverage of sporting events and an interactive, fun medium.
The innovation department has teamed up with Hawk-Eye Innovations, supplier of match data to the French Tennis Federation, to reconstruct, in 3D on Roblox, tennis matches that took place on the Philippe Chatrier court. Hawk-Eye tracks the positions of players on the court, the trajectory of the ball, and analyses match data with precision to determine, at any given moment, what action is taking place on the court. Using this centre of mass tracking data, the Innovation team has developed an application available on Roblox that allows avatars to replay this data on a P. Chatrier court faithfully remodelled in 3D.
In the future, this technological brick could be at the heart of a complete experience on Roblox, with the possibility of adding games, video extracts produced by France Télévisions and community events based around sporting competitions. The platform is also an ideal playground for showcasing large-scale cultural or musical events that bring together audiences of all ages. It is also a new opportunity for France Télévisions' youth and education departments to reach this target group, which is potentially turning away from traditional TV.
*Roblox is a free massively multiplayer online video game (PEGI 7) aimed at children and teenagers (target age 9/16), launched in 2006 and available on PC, mobile (80% of usage), tablet and home console. This is a sandbox-style game, enabling users to create worlds and games for other users to visit. More than 50 million games are available on the platform.
Immerse yourself further on VRChat
This year, on the same Chatrier court in virtual reality, France Télévisions is presenting a demonstrator enabling visitors to see and replay sequences of matches captured at Roland Garros and rendered in 3D using motion capture technology developed by the company GVCap.
Two experiences are available on VR headsets:
The first consists of watching a match between two tennis players from your avatar on the Chatrier court, with the option of moving to any position on the court to get the best possible view. Two avatars with hyper-realistic movements faithfully replay the points captured by GVCap's technology. This technology, which is based on the use of commercially available cameras (such as GoPro), captures the movements of the players and their rackets, their precise position on the court and the trajectory of the ball. By processing these images using GVCAP's AI technology, the data can be displayed on avatars and the match replayed virtually as it was in real life.
Motion capture technologies will evolve to enable this data to be processed in real time. In the near future, it will be possible to access this virtual match live.
The second experience on offer allows you to replay the match with your avatar against one of the players on the pitch. With a choice of 4 levels of difficulty, you can take on the player with the real tracking data (the opposing player's position and the ball's trajectory) as captured during the match played on the Chatrier court. Depending on the difficulty level chosen, you have to be able to return the ball as quickly as possible to score the most points. The most experienced players even go so far as to use the actual speed of the ball... 😊
*VRChat is a virtual reality social network. This platform allows its users to create avatars and virtual worlds to interact with other users online, via virtual reality headsets or on a computer in desktop mode.
Volumetric video with 4DViews
To take immersion and realism even further, France Télévisions has teamed up with 4DViews to offer sequences of technical tennis moves in volumetric video.
The innovation team went to the 4DViews film sets in Grenoble to capture several video sequences of a tennis player in motion in order to extract several ultra-realistic extracts in 3D.
The tennis player (Malo) was filmed and rendered in volumetric video thanks to Holosys. The HOLOSYS is 4Dviews’ volumetric capture system which enables storytellers to bring real humans into digital experiences. The system records human performances in a 5-metre wide capture zone from multiple viewpoints with 48 cropped-4K cameras, creating high-quality volumetric video from meshes and textures. With applications ranging across the sports, training, and entertainment industries, digital twins created using the HOLOSYS can be seamlessly integrated into AR, VR, and traditional media formats.
These volumetric files were then processed and encoded using 4Dfx and 4Dcoder software respectively, before being integrated and read using the 4DViews SDK in the Unity real-time 3D engine.
These sequences were then integrated by the France Télévisions teams into two demonstrators enabling the various technical gestures captured to be viewed as close as possible to the 3D model:
- On a VR headset: with the option of moving around the 3D model and viewing the movements from the angle of your choice,
- SONY Elf-SR2 Spatial Reality Display: This 27-inch 4K 3D screen is unique in that it displays the 3D model in the 3D environment of the P. Chatrier court, without the need for glasses. The screen simultaneously displays two images, one for each eye, and a camera at the top of the screen tracks the user's face and eyes in particular. In this way, the tennis player in volumetric video appears in relief on the court, and the user can change perspective around him and move around the image without constraints.
Beyond professional applications, these technologies foreshadow the screens of the future, particularly those of PCs and tablets, which will provide access to these virtual worlds by offering an initial level of immersion, without having to wear a helmet or glasses.
Video hosted by IA
In partnership with the Korean company Deepbrain.AI, France Télévision has taken an interest in the subject of human avatars. The company is proposing to make available human avatars capable of rendering text in spoken language.
To do this, a human is filmed and recorded for a few hours reading 500 sentences against a green background, by Deepbrain.AI cameras. These recordings are used to identify the voice applied to different words and sentence contexts, as well as filming facial emotions and hand gestures.
All this information is then processed by Deepbrain.AI's Artificial Intelligence algorithms (Machine learning and Deep learning) over a period of several weeks to create an ultra-realistic human avatar capable of rendering any text on video.
“Paris", an English-speaking avatar (in the video below), has been chosen on the Ai Studios platform to present - in English - the various innovations of this RG Lab.
This in no way prefigures the use of these human avatars by France Télévisions, whose position on the subject is very clear: they are not intended to replace journalists or presenters. However, it is essential for France Télévisions to keep abreast of these technological developments in order to understand the issues and master their uses.
