Mapping Concepts in Apple Vision Pro

Some question why there is not more knowledge work done in XR. We contend that it is partly because getting your own knowledge into the headset is not straightforward and the knowledge in traditional digital formats is not appropriate for spatial knowledge work, as it’s linear & framed. This is why Author allows you to write notes on your Mac and turn them into Defined Concepts for the Apple Vision Pro at a click.

On Mac

  • In the bottom left toolbar, choose Ask AI, then Define Concepts

    • You will then be presented with the concepts the AI process discovered, in the Define Concepts dialog for you to choose to ‘Edit’, ‘Skip’ or ‘Save and Continue’, until you have gone through all the concepts found in your notes.

  • Put on Apple Vision Pro and open Author

    • Then open the same document, which will appear just like on your Mac, including a toolbar at the bottom. Pinch ‘Map’ to see the Defined Concepts which then appear all around the document.

In Vision Pro

  • Use the visionOS interactions you are used to, to move, select and open your Defined Concepts, in addition to the commands on the Toolbar and more Advanced Gestures, as described below.

 

Interactions

Core Gestures

  • To select a node, look at it and pinch.

  • Top open, look at it and double pinch.

  • To move a node, look at it, pinch-and hold and move, then let go. Note that this does not select or deselect

  • To edit a node, look at it and long-pinch. Note that the dialog to edit it appears on the toolbar, not on the node

‍ ‍

Toolbar Options

Depending on whether you have selected one node, several nodes or no nodes, the options in the toolbar will be different, including:

  • Focus hides any node not selected or connected to. This is very powerful to reduce visual complexity.

  • Snap moves any selected node back to their plane of origin. This gives you a useful plane to work with when you want to.

  • Hide allows you to hide selected nodes. You can choose to ‘Reveal’ them again later.

  • Like lets you ‘Like’ a node as a favorite, which you can use for selections later.

  • Select allows you to select by category, such as ‘People’.

  • Show allow you to deduce what to see, also by category.

You also have [D] in the toolbar to ‘Deselect’ any nodes quickly and [A] to view all nodes.

When you Like and Hide, these options are available to you also when you go to Author macOS.

‍ ‍

Advanced Gestures (experimental)

  • Same selects all the other nodes which are the same category as what you have selected, for example, all ‘Persons’ or ‘Places’. Zoom out by holding both hands together, pinch and then move them apart while still pinching to select all of the same category of nodes

  • Deselect zoom in by holding both hands apart, pinch and then move them together while still pinching

  • Focus Pinch and flick your wrist down

  • Snap Pinch and flick your wrist up

  • Alignment:

    • Align vertical gesture - ‘two hands, pinch and drag in a circular motion clockwise around Z axis’

    • Align vertical reverse gesture - ‘two hands, pinch and drag in a circular motion anti-clockwise around Z axis’

    • Align Horizontal gesture - ‘two hands, pinch and drag in a circular motion clockwise around Y axis’

    • Align Horizontal reverse gesture - ‘two hands, pinch and drag in a circular motion anti-clockwise around Y axis’

    • Align Depth gesture - ‘two hands, pinch and drag in a circular motion clockwise around X axis’

    • Align Depth reverse gesture - ‘two hands, pinch and drag in a circular motion anti-clockwise around X axis’

‍ ‍

Context

  • Context is an aspect we are experimenting with, allowing you to define concepts as a Context to your current work. This aspect of interactions in XR is ongoing and experimental.

‍ ‍

Toolbar

  • The ‘Write’ view of your document does not disappear when you pinch ‘Map’

  • A second Toolbar for the Map view appears on top of the standard Toolbar, giving you further controls

  • If you want to concentrate on the Map, you need to reduce the size of the document, which you can do by looking at the bottom right of the document, pinching and re-sizing, as is visionOS standard interaction. Unfortunately this cannot currently be done automatically, at least yet. We are looking at ways to make this cleaner

Author Features