Product Designers Need 3D Tools That Work With Existing 2D Workflows
Why aren’t there more UX prototyping tools for me to choose from?
— No one, never.
A recurring theme among mobile product designers that I have talked to as Design Director at Torch, where we are building a spatial prototyping app, is that the last thing anyone needs is yet another tool to learn.
Designers are not frustrated with the tools they use frequently — quite the opposite. Designers uniformly describe their favorite tools as making them more efficient and more creative, both of which are vital since they are increasingly asked to do more in less time.
But when the topic of new tools comes up, they express a feeling of “tool burnout.” The high cognitive load of having so many tools to switch between, the time commitment and learning curve to become proficient in new tools should they pass muster for the design task at hand, and the struggle to fit all the different tools into a clean workflow doesn’t exactly excite them.
In our survey of designer tools and workflows conducted earlier this year, we found most use multiple asset creation tools (Sketch, Adobe) but focus on a single prototyping tool like Figma or InVision. This makes sense. Prototyping tools have defined workflows that include not just creation, but sharing of prototypes and the ability to gather feedback. In choosing prototyping tools, designers list interactivity, collaboration, and seamless integration with asset creation apps as a chief priority. Complexity kills creativity, as one designer told us in an interview.
None of this even contemplates the introduction of 3D tools for mobile AR design and prototyping.
Once designers dip their toes into the pool of 3D design tools, they find a learning curve that is not just steep, but nearly asymptotic. The addition of the Z axis rendered on a 2D screen, with all the associated complexity this brings to representation and manipulation of assets, is a big factor in that initial struggle to gain competency. But more than the broad conceptual challenges that need to be overcome, the UI of 3D tools is really quite alien to traditional product designers.
Tools like Blender and Sketchup, not to mention game engines like Unity and Unreal, evolved to meet the needs of an entirely different user group — architects, engineers, animators, and game designers mostly. A quick survey of the UI of super-powerful 3D model creation tool Blender shows top-level menu items like Physics and Render and an workspace in which objects are manipulated using three-axis gizmos that are likely quite alien to a mobile designer.
Where does one start?
I have put together a list of factors to consider when you are ready to start prototyping a 3D project and want to minimize the cost of learning a new tool.
- 3D tools, especially ones for mobile AR should look and behave like existing 2D tools — Tools should adopt, where possible, the familiar UI and UX conventions of 2D tools. Given all the tool fatigue, this really should be non-negotiable. And anyone building tools that doesn’t do this is missing the point. In the next three years experts predict the number AR-capable devices to number in the billions while worldwide community of mobile product designers already comfortable designing on mobile devices numbers in the millions. Mobile AR tools should, to start, look and feel like existing tools as much as possible.
- 3D design and prototyping tools should fit into existing workflows — again, this really should be non-negotiable. Most mobile design and prototyping apps have APIs and well-worn usage patterns and workflows. The former makes it inexcusable not to adapt to the latter. If you have to do too much switching between tools and your assets don’t sync seamlessly between the various tools, cost goes up and tool fatigue begins to mount. Designers deserve better.
- Support for all three layers of a mobile AR app — What do I mean by “layers of a mobile AR app?” Consider that mobile AR apps are a hybrid of traditional 2D mobile apps and immersive apps traditionally experienced through headsets or glasses. A mobile AR app has three distinct and interrelated layers: the 2D UI elements that are locked to the screen (i.e. also known as “on glass”), the 3D and 2D elements arranged in three dimensional space, and the images captured by the camera. Each can interact with the other. Any tool that doesn’t either allow the designer to build and test all UI for three layers or integrate seamlessly with tools that do some of the above is not complete.
- Support for 2D assets in 3D — the idea that 2D assets have no role to play in 3D space is a common misconception about 3D in general and mobile AR specifically. In fact, our own experience with 3D at Torch is that 2D assets are essential. They are far less expensive to produce, easier to iterate, and convey a familiarity that users will welcome as they navigate new immersive experiences. 2D menus may move off the screen and into immersive space, but they aren’t going away. Mobile AR design tools need to make it dead easy to incorporate 2D files and interactive art boards, treating them as a native format and ideally syncing directly from APIs.
- Support for in-camera design: Any tool for mobile AR designers must support in-camera design and prototyping. The camera and the environment it captures together form the very foundation of mobile AR. As a designer, you not only need to verify your UI works overlaid across all sorts of real-world environments, you will need to incorporate interactive triggers like gaze (at or away from an object), proximity, computer vision and image tracking, and occlusion into your designs. Similar to the on-device mirroring feature for many prototyping tools, any 3D design tool that does not at least incorporate the camera’s contents is not a mobile AR design tool. The very best option is designing on the device itself.
- Support for complex information architectures — 2D tools let users design and prototype apps with branching flows and interactions that reflect the complexity of real-world applications. Mobile AR apps should absolutely do the same. For every screen state you design in a program like Figma or InVision, there may be one or more 3D scenes. This also means interactions — 3D prototyping tools especially must allow users to set interactions that trigger object state changes and navigation between scenes.
The bottom line: if you are a mobile product designer, you already have 90% of the skills needed to make an amazing mobile AR prototype. The tools for mobile AR need to adapt to your workflows as much as possible, without compromising what makes spatial computing so amazing and emotionally engaging. It is an exciting prospect to consider that millions of designers are about to bring their expertise in UX to a new medium, and in the process, to unlock the enormous potential of 3D for the first time.
— Tera
