Note from Torch: Designers of all skill levels interested augmented reality and immersive design should register for our January 24th workshop: Interaction Design & Prototyping in Augmented Reality. Register here
Thinking in Z
If you have experience designing for screens, you are probably familiar with “X” and “Y” as 2D coordinates. These coordinates represent a distance (usually in pixels) relative to some origin point. Typically this origin point is the upper left corner of an image, window, or some other rectangular container. X is the distance left or right and Y is the up or down distance relative to the origin. For example if there is a digital image that is 12 pixels wide by 12 pixels tall, then the pixel at “6, 6” is near the center of that image.
What if you could move that single pixel forward into the screen or backward out of the screen? You would need a third axis in addition to X and Y which is commonly known as “Z”. A pixel location of “6, 6, 10” means it’s still within the image when looking at it straight on but is 10 pixels into the screen away from the viewer. “6, 6, -10” means it would be hovering out of the screen towards the viewer.
Unfortunately working in 3D doesn’t just mean adding a Z coordinate to element positions. Recall that a position like “6, 6, -10” is all relative to that upper left corner origin of the image which is “0, 0, 0.” The assumption here that keeps this simple is the origin can only move along the X, Y, and Z axes. But what if the image (and that origin point) can also rotate along any of those axes? If you tilted the image 90º backwards, it is now laying flat and facing upwards towards the sky. Our magic 3D pixel that was 10 pixels in front of the other pixels towards the viewer is now hovering 10 pixels above them.
As you can see, something simple in 2D can get complex quickly in a 3D context. The good news is 3D tools for creatives are getting easier to use each day and allow you to work visually instead thinking too much about the math involved. As 3D creation tools like our own become more accessible, we thought it was useful to review some terms and concepts you may encounter.
Key Mobile AR Terms Designers Should Know
Even with the complexity reduced, there are still some key terms you should be familiar with as you start to work in 3D. We have a short list of them here to help you get started.
Translation
Translation is a fancy way of saying the position of an object in 3D space. Usually this is expressed as three numbers (in meters, typically) that represent the distances right, up, and forward. Usually these are called “x”, “y”, and “z”, respectively. Beware: not every industry or tool agrees on which axes these labels represent. These numbers are relative to a point called “origin” or “anchor” that represents 0, 0, 0.
Rotation
Rotation is similar to Translation in that it represents the three dimensions of values. These three values are best thought of like an airplane with yaw, pitch, and roll. Just like Translation, these are sometimes expressed with “x”, “y”, and “z” labels. The values are usually in radians or degrees.
Mesh
If you think back to our pixel example, we were really just talking about a very specific point in the 3D coordinate space. Three points can be connected to make a triangle. A mesh is a collection of those triangles used to define surfaces and shapes. A point in 3D is also known as a vertex.
Materials
Materials give something a colorful, detailed appearance in 3D. At a very high level, this is where 2D images are applied to meshes (often these are called Textures). There may also be other parameters described in a material that can affect the final appearance. Think of these like image filters that are common in 2D tools.
Model
A model is a collection of meshes and materials used to represent a visual object. These are what you can download from Google Poly or Sketchfab. Even to get a 2D image to display in 3D space, it needs to be made into a simple model (a polygon mesh to create a flat plane and a material to render the image on the plane). Models can contain animations.
Lighting
Virtual lights work similar to lights in the real world. Lights affect the final appearance of models and their material appearance in a scene. Ambient light is the overall lighting of the 3D scene described with an intensity and color. Directional lights are like ambient lighting but they have a direction property which affects shadows and highlighting. Point lights radiate light in all directions from a single point which means they generally have properties of a position, intensity, and color. A spotlight has a location like point light but also has additional properties to define the cone of light coming from it.
Occlusion
The hiding of virtual objects behind other virtual objects, or behind real things is known as occlusion. To accomplish this in any arbitrary location or environment, the mobile device must create meshes that represent the physical world. One of the companies working on solving this for mobile AR is 6D.AI and they have an excellent primer on occlusion in mobile AR.
Physics
Virtual objects generally need to behave like their real-world counterparts in order to preserve a sense of immersion. This is known as physics. A virtual brick, for example, should have different virtual physical properties from a virtual ball, in terms of weight, solid appearance and interaction with the environment. A brick falling onto a table might jolt upon impact, while a ball will bounce several times. To achieve this, physical attributes (such as bounce damping ratio) are applied to each virtual object using a physics engine, which is a software library containing physics-related functions. The tighter the coupling between the real and virtual layers in AR, the greater the need for an underlying physics engine.
Spatial Audio
Spatial audio is the sonic experience in AR where the audio source has a specific position in the virtual space. Utilizing the positional tracking of the viewing device, the audio engine will pan and attenuate the audio which gives a very strong sense of location relative to the viewing device. For example, if a virtual puppy is yapping directly in front of you in the AR app, you would hear it clearly in both ears. But if you move your head or device left, the audio perspective would shift along with the visual, and the puppy’s bark would be heard mostly in your right ear.
These fundamentals should serve you well as you begin your exploration into creating experiences for 3D. Of course nothing beats actually working in 3D. Start by downloading Torch AR today – you'll be prototyping in minutes!
Thanks to Alysha Naples.
