We met Olivier Palmieri, Director of XR Workshop & Game Director at Ubisoft Montreal, to discuss Virtual Reality (VR) and Augmented Reality (AR), including their latest developments and their current and future applications as learning tools. Here is the second interview of the series.
Olivier has been in the video game industry for over 20 years. He has worked on various projects including the Rayman series, Ghost Recon, Assassin’s Creed and Far Cry. He is the creator of Ubisoft’s first virtual reality game, Eagle Flight, which won a D.I.C.E. 2017 for its technical realization in immersive reality.
There is a wind of renewal blowing through the virtual reality industry: we see the number of headsets multiplying and technology giants trying to conquer the general public. But where is the VR in terms of comfort? We hear a lot about the “cyber-discomfort,” including nausea, which this technology causes in some people…
There are still many answers to be found, but some steps have been already taken. Over the past 4-5 years, there has been an increased awareness of the issue in the virtual reality community. Before, even though there were many companies working on VR, with many prototypes being released, the industry did not care about comfort. This technology was being “discovered” and, in fact, it was not necessarily known to cause discomfort, so, many products, not properly developed, were released.
Then the industry became aware of the problem, research was done, and solutions began to be applied. Some companies have tried to get around the problem, for example, by not adding movement into the VR experience and by integrating teleportation systems. There are many of them today. It consists of passing the user from one place to another by cutting the image through a fade to black, without showing the movement. It is a way of doing things that avoid the problem but does not attack it head-on.
We and other companies have tried to find solutions to create comfortable VR movement, for example, by adding a virtual nose to the users in their VR experience. The industry still has a long way to go, mainly in terms of the software component, but at least there have been some reactions related to comfort.
What needs to be improved on the software component side?
In virtual reality, two main components have a role to play: the hardware component – headsets, and the software component. The software component includes the performance aspect, which is also related to comfort. VR headsets require a high level of computing power. If an experience is too demanding, the machine will slow down, and it will cause discomfort, nausea, because it will not be possible to reach, for example, the 90 frames per second required. Today, different solutions are being considered. There are autonomous headsets that are starting to arrive that have this built-in computing power; therefore when you create a VR experience, you at least know for which headsets has to be developed, and you adapt the visuals, and the software computation needs for the device.
We can say that there is a problem since several different types of equipment are available on the market. However, the industry is beginning to streamline; the trend is towards autonomous, fixed and more, let’s say, “standardized” headsets. Having a standard will allow us to control better the performance of the games and experiences we will release. It’s going to take a few more years because some companies continue to create different equipment to stand out and try to take over the market. Therefore, there are still challenges. Maybe we will always have two systems, a bit like Apple and Android, like Mac and Windows. Now, there are more than two, but we are moving towards standardization.
What will be the next major step in VR?
One of the big steps forward for VR is the 5G. There are still discoveries to be made, but it would allow all the calculations – and therefore all the necessary performance – to be made in the cloud by very powerful machines from Google, Amazon, Microsoft, etc. In other words, server centers will do big calculations to achieve very realistic, very precise images that they will send back, by 5G, in real time, to the user’s headset.
This would solve the problem of latency between the time the user makes a head movement and the time he receives the new image: his head movement would be sent through 5G, calculated in the cloud, and when the new image is processed it would be returned by 5G in the headset in less than three milliseconds. It’s a very big challenge, it doesn’t work yet, but tests are being done in this direction.
What about the issue of comfort on the hardware side?
It’s pretty much settled. Ultimately, what manufacturers are looking to develop, the “future” VR and AR headset – because these are two applications of the same technology – is a pair of glasses that will be as light and non-intrusive as possible, like a pair of very thin vision glasses. It’s hard to say how long it will take to see this happen, but it could be in the next 5 to 10 years.
If we think of a slightly heavier pair of glasses, which include more equipment, then I think that in 5 years we will start to see interesting things. This is because in 2019 we will see the headsets of generation 2 coming out: they will already be lighter than those of generation 1 – which date from 2016 -, and they will not need headphones, because the sound projection will be completely autonomous.
Chances are that in generation 4 we’ll have smaller, lighter, cheaper headsets. The industry is moving in the right direction. Generation 2 will be good, and maybe for generation 3, new actors will join the scene.
What would the current VR comfort score, out of 10, be?
It’s pretty hard to give a grade. I would say that the industry as a whole is at 5 out of 10, I think it’s halfway there. Some research and progress have been made, but there is still a long way to go to reach 10 out of 10, to be able to say that all experiences are comfortable. However, I’m talking about the “global” industry. For my part, when I design experiences, when Ubisoft develops experiences, we really try to achieve an 8-9 out of 10 in comfort level. For Eagle Flight, that’s really what we did. The game came out two years ago, I saw thousands of people playing and according to the tests I was able to do, about 80% of people, male or female, are very comfortable with this game.
How much time can we spend in a VR headset before we feel uncomfortable?
It’s an interesting subject… In fact, there is an addiction phenomenon. The human brain can eventually get used to anything a little new by being exposed to it. Let’s take the example of a person who grew up in a culture where you don’t eat hot dishes and who finally tastes them. At first glance, the experience will not be pleasant, unlike that of someone else who comes from a culture where they eat spicy food, who will have a higher tolerance level because they have been partially desensitized by eating it.
With VR, exactly the same thing happens. The first few times, you are more sensitive to nausea, but the more you actually experience it, the more a habit is created: the brain understands that it is not a poison. As I explained in a conference, discomfort is a reaction of the human body trying to defend itself against what the brain thinks is a neurotoxin, a poison that must be quickly evacuated. By being exposed to VR, the brain’s reaction decreases and it may eventually become so accustomed to it that it will never cause nausea again.
That’s what happened to me. In my first experiences testing VR, I was getting nauseous. Then, by doing it all the time, it stopped making me sick. Today, I can even try VR experiences that no one should do, but that does not make me sick because I am used to them, and I have forced my brain and body to accept them.
We also saw it in our test teams, for example when we developed the game Eagle Flight. Testers who felt nauseous in the first few weeks became less sensitive after several months. Some have even become totally insensitive.
A final example: there are video games called “shooting games,” in which you can move around with the mouse and keyboard. When this type of game came out, like Doom, one of the first games of its kind, many players were sick just looking at a screen that had movement. Today, we are so used to seeing movement on a TV or mobile phone that this phenomenon of nausea is much less important. We have become accustomed to it, and we are much less sensitive to it. Finally, I think that with VR too, people will gradually get used to it and they will feel less and less nauseous.
Regarding the time you can spend in virtual reality, you should know that discomfort can also come from visual fatigue or the fact that you feel a little locked in a headset and want to get out. It varies from one person to another. I spend hours in a VR headset, I don’t have a problem, and I like it. There are people who spend two, three, four hours in a row, and who have no problem with that. It’s like any other activity…
You mentioned research that has been done on VR discomfort. Has neuroscience provided some answers?
To my knowledge, we have not yet done enough work on this plan. Then, I don’t know; maybe some companies have a whole specialized department that has already studied the subject a lot. Tests have been done, but I have not yet seen any spectacular studies or advances from companies that have used neuroscience to solve these problems. However, I firmly believe that the industry should get down to it.
Moreover, this is an initiative that I would like to pursue, especially in collaboration with other players in the Montreal ecosystem, that we study with the help of neuroscience more precisely what is happening in the users’ brains when they are in the experience, that we detect when they will be sick or why. Alternatively, that we use neuroscience for the more theoretical aspect: to do research and try to find solutions by studying how to create more comfortable experiences. This is a very interesting subject; it is an avenue that has not yet been explored and it should be. I would like to explore it!
Because we don’t yet have all the answers to the problem of VR discomfort?
Exactly. For example, to my knowledge, it has not been irrevocably established why women are more sensitive than men, in VR. This fact is somewhat noticed; many people in the industry have observed that women are more sensitive to nausea than men. I have my own theory on this.
And what is this theory?
In my opinion, since discomfort is a reaction to defend the body against what the brain believes is a danger, a poison, something that we must quickly get rid of, it made me think: women who can carry a child, the alert threshold of their bodies to trigger this rejection of a possible poison is undoubtedly more sensitive than that of men… It has not been established, we are still at a hypothesis level, but I think that these are avenues to be explored, especially in neuroscience!
