Reality, it’s one of those things that we all feel we mostly have a handle on. Most of the time, when someone asks you “Who are you?” you probably have a reasonable answer. You’re somebody’s friend, you’re someone who does a particular thing, you’re someone who has certain physical attributes, certain personality traits. You’re a big venn diagram of all of these things, and in the center of that diagram, is your sense of self, your sense of who you are.
When it comes to Virtual Reality, however, that becomes a different question with a vastly different answer – an answer you may not even know, yet. I’ve been spending quite a lot of time lately in a few different Social VR applications. Each one has a different approach to how you appear to others, and to yourself. Rec Room, for example, has a lot of customization options to allow you to change hair, accessories, and your shirt, and you appear to others as a fairly cartoonish head, hands and torso – so you can look how you want, as long as how you want to appear is not realistic at all. Your eyes and other facial features are 2D, drawn on.
BigScreen on the other hand, limits you further in some respects, giving you just a head and hands, though now you do have more realistic facial features – still in the stylized realm, but you feel a little less like you’re talking to a cartoon, and again, things like hairstyle, skin color, eye shape, accessories are all customizable.
Then there’s Altspace. In Altspace you’re pretty limited – there are a few different robot avatars, including one that’s basically a colored q-tip, a masculine robot, and a stylized female and male avatar. Customization here is quite limited – your only options are to change the color of your robot, or the color of your humanoid avatar. All the humans look basically the same, though, very little individuality is possible here.
Finally the other place I’ve been spending some time lately is High Fidelity. The default avatars here are pretty limited too – there’s a generic space alien default, and a couple of female and male avatars on the market place there, but one of the interesting things here is that you can also upload your own avatar. Of the avatars available, two of them are very realistic human scans, that move quite believably as the user talks. It’s easy to forget that the person you’re talking to doesn’t actually look like that in reality. One of the things that it’s possible for you to do, in High Fidelity though, is to upload a 3D scan of yourself, and to walk around in Virtual Reality as your own self. There’s also a separate company working on allowing you to play as your own 3D scanned person in a lot of different game experiences – including things like Skyrim. The company in question, Uraniom recently made a miniom of me – their name for your scanned avatar. The thing is, it’s both great, because the avatar really looks realistically like me, and also terrible, because it really looks realistically like me.
I’m not sure that I want to play as a realistic version of myself in virtual reality, because one of the appealing parts of VR is the ability to not be yourself. I also have an avatar in High Fidelity that’s a more stylized version of me based on a scan. I’m more comfortable with that, because it looks like me, but not too much. Other people I’ve talked to, though, don’t want to ever look like themselves in VR, but they’d much rather look like an avatar that they may have identified with for a really long time – it may not look like the Reality version of themselves, but it still represents, to them, who they are.
There are other things to consider, too, when deciding if you want to be yourself in VR or not. In the real world, you can’t choose your ethnicity – or at least, you can’t choose what your ethnicity appears to be to those around you. In VR, though, you can choose to avoid the negative connotations of being black, or being female, at least visually (verbally could be another thing entirely). If you can do so, do you? How much of your identity is tied up in your gender or skin color? How about if you’re an amputee, would you decide to make your avatar reflect that? Or would you rather have all four limbs if that’s a possibility for you in virtuality? I don’t have an answer for any of these questions, partly because I think that this is something that people will decide for themselves, based on the limitations of each system. I do think that your behavior in some way is governed by how you, and others, are represented. The more realistic the avatar, the more likely someone is to treat you exactly as if you are standing in front of them – the more generic you appear, the more stylized, the less likely it is that you will feel real to the other person. We’re hardwired biologically to recognize faces, to look someone else in the eyes and recognize that there is a person inside there. If I decide to be a kitten in VR, does that detract from how other people see me? If one day, my job involves attending meetings in VR, if I don’t look like me, is that a deal breaker? Will wearing your own skin one day be the same as those jobs where you must wear a uniform? What if I just want to be a slightly prettier, more appealing version of myself? If we can all be super attractive in VR, will we never return to reality, because your meat suit isn’t as appealing as your real life suit?
I don’t have the answer to any of these questions – but I do think that the skin we wear will determine how we are treated in VR, and so determining who we are, and how we as designers and developers allow people to represent themselves, will have ongoing implications for things like community management in the long term. When we allow people to answer the question “Who am I?” with a wide variety of options, it may be that we end up with a whole different virtual society that looks nothing like anything in existence right now. And given current events, maybe that’s a good thing.
So here’s the thing. I really really really like VR and AR. I love seeing all the new things people are making. I have a variety of headsets available to me – I have the Gear VR, I have Cardboard, I have Seebright, and I have the Vive, and if something is truly awesome, I can also get access to the Rift, or other headsets.
I follow a lot of media outlets about VR, so when I see stories that look interesting about a new experience, I want to go try out that experience. At the moment, with VR video content, especially, there are a huge number of places to go to find that content – just on the gear alone, I can use the Oculus video store, I can use Samsung Video (aka previously as Samsung Milk) and then I can use other apps too. All of these places seem to have different content. None of them are easily searchable.
If you’re going to write about VR experiences, then, please, tell us how to find them. If it’s available on the Gear, please say which app I should use to find it. If it’s only available through Youtube on cardboard, say so. If I can go use my Vive to access it? PLEASE tell me how, because as much as I love cardboard, there’s no way it compares to either the Gear or the Vive for quality. I’m always going to choose the highest quality I have available to me, so why, if you’re going to write about an experience, are you making it hard for me to go find it myself? It’s like if, on the early days of the web, you wrote about an awesome website you found, but never bothered to include the link.
That makes no sense, and neither does this.
Last night I got the chance to briefly try out the Daydream labs animation tool. As I started the demo, I mentioned that I was an animator, which I was told would both help and hinder me, because their prototype doesn’t work like any animation tool I might have tried before.
This was definitely true. The prototype I tried uses the HTC Vive. My right hand was the ‘animation’ tool, my left was the timeline. There was a box of items to play with – a cylinder, a dog, a plane (as in, flying plane, not geometric plane) an android droid. To start animating, you simply grab whichever item from the box that you’d like to animate, and then put it in the start position. Once you’re happy with where you want to start, you release it, and pick it up again, only now, the timeline is running, and you’re animating as you go – whatever path you move the object along, that’s the path that’s animated, shakes and wobbles and all. What’s interesting about this to me as an animator is that a lot of unconscious movement is built in – this works very similarly to a motion capture suit – you’re really recording the movement of your controller in space.
If you don’t like a particular moment of your animation, you can go back to there in the timeline, and re-record as much as you want, however from that point, the previous animation you’ve recorded will start wherever you stop – so if you end up with your toy in the air, but your previous animation had it jumping up and down on a table, now it will start jumping up and down in the air. The translations from the prior animation you didn’t record over will start from whatever your new zero point is – something that may be unintentional from the perspective of your user.
The tough thing to grasp initially as an animator was that I wasn’t setting keyframes and tweening between them, I was the tweening. Even when I grasped this, there were some challenges in making the animation I wanted to make. Since you have to hold down the trigger to pick up and animate your object, I felt sometimes like I was limited – I couldn’t turn my object around the way I wanted to smoothly, without sometimes having to stop and re-grab the object – e.g. if I wanted my plane to do a 360, well, my wrist isn’t going to be able to do that in one smooth motion, no matter how many times I try and re-record that section. I think for this to be a really useful toy or tool (it basically could go in either direction and be useful or fun for someone) it might make sense to let users record their animation once, and then go back and edit only specific attributes of the animation. For example, I could record my plane’s path without having to worry too much about it’s orientation all of the time, just trace out the loop I want in the air, and then go back, keep that information but just overwrite the rotation information, using two hands to let me smoothly rotate while the animation plays the translation. I think that would be useful – likewise, you could allow people to edit scale on the fly, maybe even using gizmos like those in Maya or other 3D animation tools to make it easier. Sure, for those people who just want to move a bunch of objects around in space, the existing toolset is fine, but I think even kids playing around would want a few more capabilities eventually.
The other thing that would be interesting to see is how to us a tool like this to approach animating rigged objects like a humanoid character. Maybe again, rather than trying to make this a mocap lite system, having the ability to set your base motion path, and then go back and move arms and legs how you want them to move, one at a time, would work. So rather than a ‘control’ for a specific body part, you just select the entire right arm, and use your two controllers intuitively – perhaps one is the ‘elbow’ and one is the ‘wrist’ so you’re moving the character around like a puppet or a doll.
Ultimately I think this kind of thing has a lot of potential to become the new way for animators to work – if you expand the tool set some, to allow things like slowing down the speed at which the timeline records, individual control over specific attributes, and individual control over separate aspects of one model in a way similar to animation layers, where each layer is additive, I think this would be much faster than traditional keyframe based animation. You can’t underestimate the intuitive nature of working with something in ‘reality’ and how much faster that would be, especially if you can use both hands as input for things like scale and rotation. I can also see this being a very kid-friendly creation toy, allowing kids to tell stories with a giant toybox, where you’re not limited to just how many toys you can move with your hands, and where you could potentially add things like particle effects – imagine blending this with Tiltbrush style effect brushes, for example.
I definitely enjoyed trying this out, and I hope to see this project move forward – though I’m not sure how many of the Daydream labs products will turn into real applications, I hope this is one of them. Regardless, I didn’t have quite as much fun as the guy who tried the demo after me – by the end of his time, I think he had animated somewhere in the region of 50 or 60 dogs bouncing around over the table in his scene, as he laughed and cackled gleefully. We can all hope for such reactions to our endeavors in VR.
I’ve done a reasonable amount of demoing VR to #Vrgins at different events, individually, etc. It’s always interesting to see the reactions of someone who’s a first time user, but yesterday I discovered some new and interesting information, as someone who thinks a lot about UX, and how we train our brains to find certain things intuitive or not.
I was helping http://svvr.com demo at the Maker Faire yesterday. Mostly the aim was not to demo a specific game, but rather a variety of great 5 minute experiences on the HTC Vive. We had a lot of people mostly trying Tilt Brush, Job Simulator and Space Pirate Trainer, with a few other things in there like The Lab.
We also had a fair number of children under 10 trying out VR for the first time – yes, I’m aware of the guidelines, but we were very careful about safety monitoring, and parents were present for the entirety of the demo experience, plus the children were in there for under 5 minutes.
The interesting thing I noticed though, with those under 10s, was a very specific learned behavior. When using Tilt Brush, your primary mode of action is the trigger on the controller – for which you use your index finger. You change menu by using the thumb touchpad on your non drawing hand to swipe around and see the different menu panels. Selecting a tool is done by pointing the controller at the tool you want, and again using the trigger on the other hand to select. Pretty intuitive for those of us who have grown up using a mouse.
Not so for the under 10’s. Every one of them had the same instinctive behavior – to use the thumb pad on the opposite hand as a button when they wanted to select a tool from the palette. No matter how many times I said trigger, showed them where it was, even helped them pull the trigger, they still kept trying to use the thumb track pad as their selection mechanism. This wasn’t the case when they were painting, however, then, they quickly got the trigger being the paint. My conclusion is that they’re just hardwired to use their thumbs to control things when there’s something available – because they’ve grown up playing with smart phones and tablets, where generally, you’re using your thumbs to do almost every action. We’ve literally raised a generation whose basic, instinctive, technological interaction model is different than our own. And that’s fascinating to me.
How to design Social VR so it isn’t awful.
There have been a fair number of people lately talking about potential problems that Social VR will or already does have. I think that this is the perfect time to start talking about this problem – but at the moment, nobody seems to be talking much about possible solutions. I want to break down what I see as potential problems for social VR, and how we go about addressing them using a combination of engineering solutions and social design solutions.
When you’re talking about Social VR, it’s important to recognize that there are a few different kinds of social VR – just as right now, there are plenty of ways to interact socially on the web that don’t solely exist of social media sites. The basic categories of interactions for social VR will be one on one interactions – you talking to a friend, partner or family member, commercial interactions like business meetings or education, solely social interactions which could include groups of friends or strangers, or mixed groups, and social gaming or social experience, where the social interactions are secondary to some other purpose. While all of these are subject to some of the same problems, not all those problems will be expressed equally, and should not be addressed equally. For example, business interactions are unlikely to experience the harassment problem, but could have other problems associated with miscommunication, or problematic body language – something likely to be experienced more frequently when interacting with people of vastly different cultures.
The biggest problem areas then, are likely to be solely social interactions, and social gaming/social experience – i.e. places where you cannot always guarantee knowing or being able to control all the participants in a particular space, and thus cannot predict or moderate their behavior. Right now, this is frequently a problem experienced both online and in the real world, especially by women or other minorities – the ‘comment section problem’ online, or street harassment in the real world. Solely social interactions are probably more likely to be a problem – because when there’s another activity to engage in, harassment is less likely (though verbal harassment is still extremely plausible, some of the other kinds are not)
Of course, one easy solution is to say “If you don’t like that, then don’t go there, or don’t read that website, or comment on that thing.” This isn’t a solution that I or any other member of those communities like hearing, because it’s not a solution, it’s giving in to the bullies and allowing them to dictate our experience of the world. This is even more important when it comes to VR, because I want to know that it’s safe for me to try new things, meet new people, and experience incredible things in VR without feeling uncomfortable, harassed, or in any way made to feel unwelcome. And that’s something I want for everyone out there – to feel safe in VR.
Now is really the make-or-break time for VR. We have a small window of time to show people that VR is something incredible that they want in their lives, or lose them forever, and it’s not just about the great experiences or the awesome tech, it’s also about those experiences that allow us to connect with human beings a world away. Empathy is going to be one of the biggest drivers of VR adoption once the wow-factor disappears (more on this in another article) so it’s important that we get all of this right, right now, not just for men, not just for white people, but for everyone.
So what are the problems we face?
Firstly, we face most if not all of the problems the internet currently has – repeated low level harassment and bullying, attempts to silence others by bullying, intimidation and social pressure, rape and death threats, obscene or offensive language in unexpected places, age appropriate safe spaces for kids, doxxing, public humiliation or outing, and ban trolling – where you and a team of people utilize reporting systems to harass unoffending users who you simply disagree with, but who isn’t actually breaking the rules. It’s important to consider these existing problems, and look at people who are solving them most effectively when you’re going to build social VR into whatever you’re developing, and I’ll talk about the most effective solutions for these problems in a bit.
Secondly, we also face problems which are unique to VR, such as the problem of personal space. If you’ve ever tried to walk through a wall or off a cliff in VR, you’ll most likely have experienced a small moment of either disturbance or fear, especially if the environment is very realistic. There’s just a mental hiccup, a shiver, before you reassure your lower brain functions that it’s ok, it’s not really real. When someone breaks your personal space boundaries in VR, it’s just as disturbing, if not more so. Having someone else’s face directly in your personal bubble is disconcerting, and uncomfortable, especially if they’re the one instigating it. With a wall or a cliff, you stop, and then reassure yourself. When someone breaks your bubble, you don’t get that moment to pause.
There’s also the problem of teleport stalking – in experiences where I could teleport away when someone bothered me, more than once they repeated the offending behavior, by teleporting to follow me and then doing the same thing. This starts to feel harassing after two or three teleports.
Being surrounded by a group of people – something that happens to me frequently if I’m wearing a female avatar in a mostly male space – can also feel just as intimidating as it does in real life, and can be difficult to get out of, even if you can teleport away, because again, the offenders can just follow you and do the same thing again.
Audio is also something important to consider in social VR – if my personal priority is that I want to hear the person I came to VR to talk to, but someone else keeps getting between us and taking over my audio priority level, this can be annoying and a very frustrating experience. I want to have the conversation I want to have, not the one you want me to be having. Heavy breathing and other disturbing ‘right behind your ear’ sounds and not being able to hide your gender when you talk are also audio problems that should be addressed.
Finally, there’s the gesture problem – one that will only get worse over time as devices and peripherals get better and better – but even now, someone can put their face in your crotch, jerk off with their leap or kinect enabled hands, touch you inappropriately with controllers, etc. All of these are things people have already experienced in social VR spaces – not hypotheticals.
Solving problems is hard!
The good news is that right now you’re reading this, so you’re already part way there to fixing this problem before it gets really bad and really entrenched in VR culture (I hope).
There are some things to consider before you begin trying to implement solutions. For one, you want to make social VR welcoming and safe for all users, without making it extremely restrictive – you don’t want to ban the use of hands just because of a few obnoxious users. You also don’t want to make it difficult for the user to deal with offenders, because when you are having a bad experience, the worst possible rider to that is for it to be difficult to report, block, or otherwise prevent this from happening to you again, because in that case, you’re far more likely to simply leave the VR space and never come back. Recording video and logging audio is also expensive in terms of server space, so how do you solve the documentation problem?
And then there’s the implementation problems – paid external moderation is expensive and doesn’t scale well, but if you make a design that gives users power over others, you can’t necessarily rely on them to use that power responsibly. If you use algorithms to do automatic moderation, you run into problems like those that facebook encounters – it’s already a very difficult problem with text, next to impossible with voice, completely impossible right now with gesture.
The other thing to consider is that permanent banning is difficult and a perennial problem already on the internet. Users may also share devices (especially high end HMDS) with other household members – should everyone be punished for the bad action of one?
Enough already! How do we fix this?
The following is a list of ideas that I’ve come up with that address potential problems in a way that ideally isn’t burdensome to the user. Feel free to use any of these methods, and please let me know if you implement them. I think there’s a potential here to actually change how people operate in reality too – to retrain them in appropriate social behavior that would extend from virtuality to reality. For the most part, these ideas are ones which are designed to make stranger filled social spaces comfortable for all users, where you are physically embodied in an avatar of some sort. This list is somewhat long, so I’m going to switch to bullet points from here out.
- Personal Bubble – Your personal bubble should extend as far as you want it to, but not be a physical collider for other users. Rather, anyone crossing your boundary line would simply become invisible and inaudible to you, and vice-versa. This works better than a physical collider because you cannot use your bubble to affect things like doorways or crowded spaces, nor could you use it to push other people’s avatars around. Comfortable VR requires that the user always have control over their own motion – so changing personal render settings works the best for this. It also enables you to have multiple people in a crowded space, and yet not feel claustrophobic.
- Endorsements – This is a relatively simple and elegant solution for a lot of the problems. An endorsement would allow you to set your own comfort levels for things like personal space and personal audio levels, and then put other people in the appropriate group. The default level 0 would include everyone who is a stranger to you. It would enforce your personal boundary line on all strangers, as well as setting their audio level at a default lower than people who are your friends – level 1. Level 1 users can also come closer to you and still be visible and audible. People who you specifically want to permanently ignore would be level -1, permanently invisible and inaudible to you. For special events (e.g. public speaking events) there could also be high level controls for moderators, e.g. + 5 for a speaker so everyone can hear them, and -5 for the audience if you want them to be silent.
- Ignore object that allows you to easily ignore a user. This could be an actual virtual object like a hammer or a baseball bat, or a fluffy bunny, that you throw at or towards the offender. They’re then booted from your visible environment, and if you’re using the endorsements, they’re set to a -1 level. For them, you simply disappear – this is important, so that they cannot molest or stalk your avatar and be seen by other users doing so. Again, this is something that happened to me.
- Reputation values – if many users have you in their +1 circle, you gain a defense against people who would attempt to get you banned. If many users -1 you, your behavior is monitored and you are potentially banned. Reputation systems are somewhat open to abuse, but given the realtime nature (and roughly limited nature) of interactions in VR, harder to abuse by crowds of people than current social media systems. They’re also pretty valuable in allowing users to set their own preferences outright – e.g. you could be able to set your own preferences for not seeing anyone with a negative rating, for example. Some people would still get around this by convincing others to +1 them, but overall this would solve quite a lot of problems.
- Verified identity. Facebook mostly succeeded because they insisted upon real life names, not usernames. To protect people from real world harassment or doxxing, have a two factor system – where the server knows your verified real world identity, but you still have a display name to other users.
- Robot Voices– using a filter to allow users to disguise their voice as heard by others. Ideally this would be a sort of ‘robot’ agender voice.
- Anonymous and/or honeypot rooms – for users who wish to act without any restrictions on their behavior (other than outright illegal behaviors). Free speech, free action, but for some rooms, all users use the robot voice, and all are anonymous. Attempting to ignore someone in a honeypot room would instead kick the user back to a regular room. Unavailable to minors.
- Avoid hyper-sexualized avatars completely. This is a problem with things like imvu – where every female avatar is basically half clothed, no matter how hard you try to properly attire them, and that leads to certain behaviors from some users. Avatars should be ‘normal people’, agender (e.g. robots) or creatures with no overt sexual characteristics. Hypersexualized avatars again should only appear in certain honeypot rooms – which are only available to people over the age of 18.
- Train users what appropriate behavior is – consider positive reinforcement via messages on loading screens about what is acceptable behavior in that space. When you break someone’s boundary, show a visible indication for you for example, a red screen flash.. When a user gets multiple -1 ratings, coach them on appropriate and specific behavior, including what their bad behavior is, if possible, before they can enter the space again. Use every possible tool in VR to emphasize this – e.g. when a user is due for a ‘coaching’ session before entering the VR space, use a virtual avatar to reenact their bad behavior, where they are the victim.
- Recording behavior – give users a way to record their own sessions easily, and share them. This takes the burden away from the provider having to have server space, and becomes a feature for recording fun things, as well as a way users can monitor their own sessions.
- Activities – give users a lot of different things to do with friends – generally people will be less likely to harass when there are activities of some kind, especially if they are non-competitive and more cooperative activities.
- Restricting the room usage of people who misbehave to honeypot rooms, or other specific restricted spaces.
- Allow users to set the appearance of negative reputation users – if you still want to see everyone so that you get a smooth social interaction, those negative users could have a specific highlight color, avatar item, hat, etc. (e.g. put everyone in an iron mask, scarlet letter A, etc) only visible to you, that would allow you to be more wary of someone with a bad reputation.
- Freeze frame – it can be hard to click on someone to ignore them in VR. A ‘freeze frame’ where the world temporarily stops for you, but you can still interact with the UI would be helpful. This would also be a great way to add a ‘Take a snapshot’ function for users at the same time.
- Gender balance – given verified identities, allow some rooms to be female identifying only, some male identifying only, some with 50/50 balance (with a +/- 1 to allow people to join and leave the room freely). This gives everyone options for how they choose to interact and what spaces they wish to frequent, without feeling restricted to women only spaces if they do not want to.
- Gesture limitation – allow you to choose not to see hands or controllers for others if you don’t want to, and detect inappropriate face/crotch interactions within certain radii. Detecting other inappropriate gestures is probably beyond where we are at this moment in time, but would be ideal in the future.
- Event/organizer level controls that let you choose the only audible people in a room, or temp freeze everyone in spot (could be based on space ownership, doesn’t need to specifically be moderator only)
If there are any more suggestions you have, feel free to let me know!