VR – Suzanimator

Social VR Solutions

How to design Social VR so it isn’t awful.

There have been a fair number of people lately talking about potential problems that Social VR will or already does have. I think that this is the perfect time to start talking about this problem – but at the moment, nobody seems to be talking much about possible solutions. I want to break down what I see as potential problems for social VR, and how we go about addressing them using a combination of engineering solutions and social design solutions.

When you’re talking about Social VR, it’s important to recognize that there are a few different kinds of social VR – just as right now, there are plenty of ways to interact socially on the web that don’t solely exist of social media sites. The basic categories of interactions for social VR will be one on one interactions – you talking to a friend, partner or family member, commercial interactions like business meetings or education, solely social interactions which could include groups of friends or strangers, or mixed groups, and social gaming or social experience, where the social interactions are secondary to some other purpose. While all of these are subject to some of the same problems, not all those problems will be expressed equally, and should not be addressed equally. For example, business interactions are unlikely to experience the harassment problem, but could have other problems associated with miscommunication, or problematic body language – something likely to be experienced more frequently when interacting with people of vastly different cultures.

The biggest problem areas then, are likely to be solely social interactions, and social gaming/social experience – i.e. places where you cannot always guarantee knowing or being able to control all the participants in a particular space, and thus cannot predict or moderate their behavior. Right now, this is frequently a problem experienced both online and in the real world, especially by women or other minorities – the ‘comment section problem’ online, or street harassment in the real world. Solely social interactions are probably more likely to be a problem – because when there’s another activity to engage in, harassment is less likely (though verbal harassment is still extremely plausible, some of the other kinds are not)

Of course, one easy solution is to say “If you don’t like that, then don’t go there, or don’t read that website, or comment on that thing.” This isn’t a solution that I or any other member of those communities like hearing, because it’s not a solution, it’s giving in to the bullies and allowing them to dictate our experience of the world. This is even more important when it comes to VR, because I want to know that it’s safe for me to try new things, meet new people, and experience incredible things in VR without feeling uncomfortable, harassed, or in any way made to feel unwelcome. And that’s something I want for everyone out there – to feel safe in VR.

Now is really the make-or-break time for VR. We have a small window of time to show people that VR is something incredible that they want in their lives, or lose them forever, and it’s not just about the great experiences or the awesome tech, it’s also about those experiences that allow us to connect with human beings a world away. Empathy is going to be one of the biggest drivers of VR adoption once the wow-factor disappears (more on this in another article) so it’s important that we get all of this right, right now, not just for men, not just for white people, but for everyone.

So what are the problems we face?

Firstly, we face most if not all of the problems the internet currently has – repeated low level harassment and bullying, attempts to silence others by bullying, intimidation and social pressure, rape and death threats, obscene or offensive language in unexpected places, age appropriate safe spaces for kids, doxxing, public humiliation or outing, and ban trolling – where you and a team of people utilize reporting systems to harass unoffending users who you simply disagree with, but who isn’t actually breaking the rules. It’s important to consider these existing problems, and look at people who are solving them most effectively when you’re going to build social VR into whatever you’re developing, and I’ll talk about the most effective solutions for these problems in a bit.

Secondly, we also face problems which are unique to VR, such as the problem of personal space. If you’ve ever tried to walk through a wall or off a cliff in VR, you’ll most likely have experienced a small moment of either disturbance or fear, especially if the environment is very realistic. There’s just a mental hiccup, a shiver, before you reassure your lower brain functions that it’s ok, it’s not really real. When someone breaks your personal space boundaries in VR, it’s just as disturbing, if not more so. Having someone else’s face directly in your personal bubble is disconcerting, and uncomfortable, especially if they’re the one instigating it. With a wall or a cliff, you stop, and then reassure yourself. When someone breaks your bubble, you don’t get that moment to pause.

There’s also the problem of teleport stalking – in experiences where I could teleport away when someone bothered me, more than once they repeated the offending behavior, by teleporting to follow me and then doing the same thing. This starts to feel harassing after two or three teleports.

Being surrounded by a group of people – something that happens to me frequently if I’m wearing a female avatar in a mostly male space – can also feel just as intimidating as it does in real life, and can be difficult to get out of, even if you can teleport away, because again, the offenders can just follow you and do the same thing again.

Audio is also something important to consider in social VR – if my personal priority is that I want to hear the person I came to VR to talk to, but someone else keeps getting between us and taking over my audio priority level, this can be annoying and a very frustrating experience. I want to have the conversation I want to have, not the one you want me to be having. Heavy breathing and other disturbing ‘right behind your ear’ sounds and not being able to hide your gender when you talk are also audio problems that should be addressed.

Finally, there’s the gesture problem – one that will only get worse over time as devices and peripherals get better and better – but even now, someone can put their face in your crotch, jerk off with their leap or kinect enabled hands, touch you inappropriately with controllers, etc. All of these are things people have already experienced in social VR spaces – not hypotheticals.

Solving problems is hard!

The good news is that right now you’re reading this, so you’re already part way there to fixing this problem before it gets really bad and really entrenched in VR culture (I hope).

There are some things to consider before you begin trying to implement solutions. For one, you want to make social VR welcoming and safe for all users, without making it extremely restrictive – you don’t want to ban the use of hands just because of a few obnoxious users. You also don’t want to make it difficult for the user to deal with offenders, because when you are having a bad experience, the worst possible rider to that is for it to be difficult to report, block, or otherwise prevent this from happening to you again, because in that case, you’re far more likely to simply leave the VR space and never come back. Recording video and logging audio is also expensive in terms of server space, so how do you solve the documentation problem?

And then there’s the implementation problems – paid external moderation is expensive and doesn’t scale well, but if you make a design that gives users power over others, you can’t necessarily rely on them to use that power responsibly. If you use algorithms to do automatic moderation, you run into problems like those that facebook encounters – it’s already a very difficult problem with text, next to impossible with voice, completely impossible right now with gesture.

The other thing to consider is that permanent banning is difficult and a perennial problem already on the internet. Users may also share devices (especially high end HMDS) with other household members – should everyone be punished for the bad action of one?

Enough already! How do we fix this?

The following is a list of ideas that I’ve come up with that address potential problems in a way that ideally isn’t burdensome to the user. Feel free to use any of these methods, and please let me know if you implement them. I think there’s a potential here to actually change how people operate in reality too – to retrain them in appropriate social behavior that would extend from virtuality to reality. For the most part, these ideas are ones which are designed to make stranger filled social spaces comfortable for all users, where you are physically embodied in an avatar of some sort. This list is somewhat long, so I’m going to switch to bullet points from here out.

Personal Bubble – Your personal bubble should extend as far as you want it to, but not be a physical collider for other users. Rather, anyone crossing your boundary line would simply become invisible and inaudible to you, and vice-versa. This works better than a physical collider because you cannot use your bubble to affect things like doorways or crowded spaces, nor could you use it to push other people’s avatars around. Comfortable VR requires that the user always have control over their own motion – so changing personal render settings works the best for this. It also enables you to have multiple people in a crowded space, and yet not feel claustrophobic.
Endorsements – This is a relatively simple and elegant solution for a lot of the problems. An endorsement would allow you to set your own comfort levels for things like personal space and personal audio levels, and then put other people in the appropriate group. The default level 0 would include everyone who is a stranger to you. It would enforce your personal boundary line on all strangers, as well as setting their audio level at a default lower than people who are your friends – level 1. Level 1 users can also come closer to you and still be visible and audible. People who you specifically want to permanently ignore would be level -1, permanently invisible and inaudible to you. For special events (e.g. public speaking events) there could also be high level controls for moderators, e.g. + 5 for a speaker so everyone can hear them, and -5 for the audience if you want them to be silent.
Ignore object that allows you to easily ignore a user. This could be an actual virtual object like a hammer or a baseball bat, or a fluffy bunny, that you throw at or towards the offender. They’re then booted from your visible environment, and if you’re using the endorsements, they’re set to a -1 level. For them, you simply disappear – this is important, so that they cannot molest or stalk your avatar and be seen by other users doing so. Again, this is something that happened to me.
Reputation values – if many users have you in their +1 circle, you gain a defense against people who would attempt to get you banned. If many users -1 you, your behavior is monitored and you are potentially banned. Reputation systems are somewhat open to abuse, but given the realtime nature (and roughly limited nature) of interactions in VR, harder to abuse by crowds of people than current social media systems. They’re also pretty valuable in allowing users to set their own preferences outright – e.g. you could be able to set your own preferences for not seeing anyone with a negative rating, for example. Some people would still get around this by convincing others to +1 them, but overall this would solve quite a lot of problems.
Verified identity. Facebook mostly succeeded because they insisted upon real life names, not usernames. To protect people from real world harassment or doxxing, have a two factor system – where the server knows your verified real world identity, but you still have a display name to other users.
Robot Voices– using a filter to allow users to disguise their voice as heard by others. Ideally this would be a sort of ‘robot’ agender voice.
Anonymous and/or honeypot rooms – for users who wish to act without any restrictions on their behavior (other than outright illegal behaviors). Free speech, free action, but for some rooms, all users use the robot voice, and all are anonymous. Attempting to ignore someone in a honeypot room would instead kick the user back to a regular room. Unavailable to minors.
Avoid hyper-sexualized avatars completely. This is a problem with things like imvu – where every female avatar is basically half clothed, no matter how hard you try to properly attire them, and that leads to certain behaviors from some users. Avatars should be ‘normal people’, agender (e.g. robots) or creatures with no overt sexual characteristics. Hypersexualized avatars again should only appear in certain honeypot rooms – which are only available to people over the age of 18.
Train users what appropriate behavior is – consider positive reinforcement via messages on loading screens about what is acceptable behavior in that space. When you break someone’s boundary, show a visible indication for you for example, a red screen flash.. When a user gets multiple -1 ratings, coach them on appropriate and specific behavior, including what their bad behavior is, if possible, before they can enter the space again. Use every possible tool in VR to emphasize this – e.g. when a user is due for a ‘coaching’ session before entering the VR space, use a virtual avatar to reenact their bad behavior, where they are the victim.
Recording behavior – give users a way to record their own sessions easily, and share them. This takes the burden away from the provider having to have server space, and becomes a feature for recording fun things, as well as a way users can monitor their own sessions.
Activities – give users a lot of different things to do with friends – generally people will be less likely to harass when there are activities of some kind, especially if they are non-competitive and more cooperative activities.
Restricting the room usage of people who misbehave to honeypot rooms, or other specific restricted spaces.
Allow users to set the appearance of negative reputation users – if you still want to see everyone so that you get a smooth social interaction, those negative users could have a specific highlight color, avatar item, hat, etc. (e.g. put everyone in an iron mask, scarlet letter A, etc) only visible to you, that would allow you to be more wary of someone with a bad reputation.
Freeze frame – it can be hard to click on someone to ignore them in VR. A ‘freeze frame’ where the world temporarily stops for you, but you can still interact with the UI would be helpful. This would also be a great way to add a ‘Take a snapshot’ function for users at the same time.
Gender balance – given verified identities, allow some rooms to be female identifying only, some male identifying only, some with 50/50 balance (with a +/- 1 to allow people to join and leave the room freely). This gives everyone options for how they choose to interact and what spaces they wish to frequent, without feeling restricted to women only spaces if they do not want to.
Gesture limitation – allow you to choose not to see hands or controllers for others if you don’t want to, and detect inappropriate face/crotch interactions within certain radii. Detecting other inappropriate gestures is probably beyond where we are at this moment in time, but would be ideal in the future.
Event/organizer level controls that let you choose the only audible people in a room, or temp freeze everyone in spot (could be based on space ownership, doesn’t need to specifically be moderator only)

If there are any more suggestions you have, feel free to let me know!

3D asset creation – a very basic primer

I’m going to do my best here to break down some terms, ideas and principles behind 3D asset creation, but this is by no means a comprehensive guide – rather I’m just going to talk about the different concepts and important ideas, so that the terminology and workflow will be easier to understand. I’m going to be using Maya in all the images, but these ideas stretch across all 3D modeling software (although CAD based software operates a little differently.) I’m also going to focus on things that are important for VR and game dev specifically, rather than high-end animation. I will bold vocabulary words the first time I use them in the text.

Modeling and Meshes

Creating any 3D asset always starts here – once you have your concept idea (maybe some sketches, or just an idea in your head, or a real world item you’re copying) the first thing you are going to do is create a polygon mesh (there are other ways to create meshes that have different strengths and weaknesses, but in general, polygons are the way you’re going to want to go for VR/game dev)

The simplest example of a polygon primitive is the plane – but the most useful is probably the simple cube. Think of a polygon as being a face of any object – in 3D modeling, every object is made up of faces (at least usually – again, there are other categories that I am not going to go into within this tutorial.) A primitive is one of a variety of basic ‘starting point’ shapes that modeling programs include. Others would be a plane, a sphere, a torus, a cylinder, etc. So in a cube, there are six polygons. We can also refer to an object created from polygons as a mesh, or polygonal mesh.

From here it’s possible to create basically every object that you can think of, with a few modeling tools.

The components of the mesh are pretty simple – each face of the cube is a polygonal face. Each green line you can see above is an edge, and the corners where those edges meet are vertices.

All of those things can be moved around, scaled, rotated, etc, to create new shapes. It’s also possible to add divisions to a primitive, to give you more to work with – e.g.

Another way to add complexity is to use an extrude tool. What this does is allow you to select a face or an edge, and pull a whole set of faces and edges out of it – e.g.

In this case, I selected three faces, and extruded them out. I could also have scaled, moved or rotated them – but now, where there was just one face, there are four more faces. There are a lot more modeling tools available to you depending on the software package, and I encourage you to experiment, but this is one easy way to build things – extruding, and then manipulating edges, faces and vertices to get the shape you want.

Polygon best practices

Bear in mind when modeling for games or for VR/AR, what you’re doing involves real time rendering – the hardware you’re using has to evaluate and display everything you create in real time. When it comes to polygons, that means using as few as you can get away with – if you have a cube with 6 faces on screen, that’s a lot cheaper than having a cube with 600 faces. Obviously building an entire scene solely from 6 sided cubes might not be what you want, but it’s worth thinking about what your polygon budget is for any given scene – benchmarks for VR are usually something like 20,000 to 100,000 polygons for a scene. So use as few as you can.

Other things to be aware of – a game engine like unity turns every polygon into a triangle, and your life will be much simpler if you avoid the use of polygons with more than 4 sides. An n-gon with 6 or 7 sides might seem more elegant sometimes, but will actually make your life harder in a few different ways. Stay simple, use quads and tris.

Materials, shaders, textures and normals oh my!

Whatever renderer you’re using, it has to be able to decide what every pixel on the screen looks like. It does this doing lots of complex math based on lighting and a variety of other things. Right now, your model is a collection of faces that are a hollow shell. Each face has an associated normal. This is an imaginary ray that points out from every face, and defines which is the front or back side of the face (most faces in 3D modeling are one sided- which means that they’re invisible from the wrong side.) Normals also govern how imaginary light rays hit the object, so editing them using a normal map lets you add complexity to your model that doesn’t require additional polygons. A normal map is a 2D image that wraps around your 3D object. The colors on the normal map don’t translate to colors on your object, they translate to edits to the normals.

Vertex normals displayed above – the small green lines radiating from each vertex.

In the above image, I selected two edges and “softened” the normal angle – you can see how that’s changed how we perceive the edge, even though the geometry of the cube hasn’t changed at all. A normal map is a more complex way of doing this – more on that later.

A Material is something you assign to a mesh that governs how that mesh responds to light. Materials can have various qualities –reflectivity, specularity, transparency, etc, but basically a material is what your object is ‘made of.’ E.g. if I want something to be metal, metals have certain properties that are different from something I want to appear plastic, or wood, or skin. Examples of simple materials are things like lambert, blinn, phong. Bear in mind, a material is not color information, it is how something responds to light.

Here’s the same shape as before, with a more shiny blinn material applied – the previous examples were all lambert.

A Texture is where your color information comes in. A texture can be a simple color, or it can be something generated procedurally, or a 2D file mapped to a 3D object. Here are examples of each of those.

Simple color information (note, we still have the ‘blinn’ material here, which is why this is slightly shiny)

A simple procedural noise texture – this is something generated by the software based on parameters you pick. Notice how the pattern is oddly stretched in a couple of places – more on this later.

Here’s a file based texture – where a 2D image is mapped onto my 3D mesh. Notice again how there are places where the text is stretched oddly.

Final definition for this section – a shader is the part of your 3D software that takes the material, mesh, texture, lighting, camera and position information, and uses that to figure out what color every pixel should be. Shaders can do a lot of other things, and you can write them to do almost anything you want, but mostly this is something I wouldn’t worry about until later.

UV maps and more on textures.

Remember in the previous section how those textures I applied were a little wonky in places? That’s because I hadn’t properly UV unwrapped my object. A UV map is what decides how your 2D image translates to your 3D object. Think of it like peeling an orange – if you peel an orange, and lay the pieces out flat, you end up with a 2D image that describes your 3D object.

Here’s what the UV map from my last image looks like right now.

You can see that there isn’t really any allowance made for my projections from the cube – what is here is just a cube map – and maybe some of the data I wanted on object isn’t displayed at all – like the faces of my event speakers.

How a UV map works is it assigns every vertex on the object to a corresponding UV. What that means is that each face then knows what area of the 2D texture image it should be displaying – and the software will stretch that image accordingly.

If I remap my cube properly, here’s what it looks like

You can see now that I have my base cube, and I’ve separated out the projected pieces into their own maps – that’s why there are holds in the base cube, because that’s where those maps attach. I’m not going to go into the different ways you can unwrap objects, because it’s a fairly complex topic, and there are a lot of different ways to unwrap something, but in general, you don’t want to stretch your textures, and you want as few seams (i.e. separate pieces) as possible. It’s also possible to layer UVs over one another, if you want to use the same piece of a texture multiple times.

Here’s what the remapped cube looks like now – no more odd stretching.

Final note on file textures – they should ALWAYS be square, and always a power of two number of pixels per side – e.g. 256 x 256, or 1024 x 1024.

(also side note, if I was actually using this as a production piece, obviously I’d be taking more care with the image I used, instead of using a promo image for our recent event)

Normal Maps, Bump Maps, Displacement Maps

All three of the above are ways of using a 2D image to modify the appearance of a 3D mesh. A normal map can be seen below. In this case, the image on the left is what we’re trying to represent, the image in the center is the map, and the right shows what that looks like applied to a 2D plane – it appears to have depth and height values, even though it is a single plane.

A bump map does something similar, but uses a greyscale image to calculate ‘height from surface’ values. A bump map is very useful for doing what it says in the name – making a surface appear bumpy or rough. The thing to note with a bump map is that it doesn’t affect the edges of an object – so an extreme bump mapped plane seen from the side will just look like a plane – no bump information

A displacement map is similar to a bump map, but does seem to affect the calculated geometry of an object – ideal for adding complexity, but not usually supported in game engines. Most game engines support normal mapping as the way to add depth information to polygonal objects.

There are other types of map too, that govern things like transparency or specularity, but those are beyond the scope of this post.

Rigging!

So now we have a lovely cube, with a material and texture. If your asset isn’t intended to be have moving pieces, at this point you’re done – you’ve built your table, or chair, or book. Any animation you might do on it probably will just move the entire object. If this is not the case, however, if what you have is a character or a car, or a robot, you’re probably going to want it to be able to move.

Here’s SpaceCow. SpaceCow animates – her head and legs and body and udders all move. And that’s because I built a rig for her – a skeleton and set of controls that move that skeleton around, and that define how the skeleton moves the mesh. Rigging is a vast, deep and complex subject, so I am not going to go too far into it right now, I’ll just show you what a rig looks like and explain very briefly how that works.

In this side shot, you can see white triangles and circles which show the joints that make up SpaceCow’s skeleton. Every part of her that I want to be able to control has a joint associated with it, and those joints are attached together in a hierarchy that governs which ones move when other joints move.

In order to animate SpaceCow, I want to be able to control and key those joints – assign specific positions at specific times or keyframes.

So I build a control structure for the joints that consist of simple curves that I can move around easily.

If I hide the joints, that structure looks like this

The white lines here are the control curves – each one lets me move around different parts of the skeleton. The very large line around the whole cow lets me move the entire cow, too. There are other parts of rigging that define how the mesh attaches to the joints, but that isn’t important now. If you want to learn rigging, I highly recommend Jason Schleifer’s Animator Friendly Rigging, but there are a lot of other great resources out there.

Animation

Once you have a rig in place, you can move on to animation. Animating is done by assigning keys to specific frames along a timeline. That means that for any key, I can set the properties of position, rotation, (and sometimes scaling).

In the above image, I have selected the curve that governs SpaceCow’s head. The timeline at the bottom of the image shows all the keys I’ve set for her head movement – each red line represents a key that I set. The position between each key is determined by curves that interpolate smoothly from one to the next – so if my x rotation starts at 0 at frame 1, and ends at 90 at frame 100, at frame 50 it will be around the 45 degree mark. Again, this topic is more complex than I have time to go into, but this is the basics of how this works.

Conclusion

Thanks for wading through this – I know this ended up being a long document, partly because 3D asset creation is a complicated subject. Hopefully you should at least understand the basic workflow (the topics appear in the order of operation) and how everything fits together, if not how to do each specific thing. Please let me know if you are confused by any of this, or if any information is inaccurate in any way.

Thanks for reading!

Suzanne