Mapping Concepts in the Vision Pro

Author lets you write notes on your Mac and transform them into spatial knowledge structures on Apple Vision Pro, turning linear documents into spatial thinking environments.

The aim is ‘literally’ to expand our minds. The word ‘understand’ in old English understandan meant “to be in the midst of,"“ to be surrounded by. You could also say that know something is not to possess it but to be in relationship with it.A spatial map gives you knowledge as a landscape to inhabit. When you move through the Map in Author — reaching toward a node, drawing out its connections, repositioning yourself relative to what you find — you are not browsing information, you are under-standing. Interaction is not a convenience layer on top of understanding. At the cellular level, this interaction mirrors how understanding is physically made in our brains. What you see here and what you experience in Author visionOS is only the beginning, we are only scratching the surface of working with spatial knowledge. Join us on this journey.

Writing is thinking. This is our exploration into spatialized thinking.

the research : Spatial Thinking

 

From Notes to Nodes

  • Author macOS

    • Open the document with your notes.

    • Click ‘Ask AI’ in the toolbar and choose ‘Define Concepts’.

    • You will then be presented with the concepts the AI process discovered, in the Define Concepts dialog for you to choose to ‘Skip’, ‘Save & Next’ or ‘Save All’. (You can of course also create the Defined Concepts manually)

  • Author visionOS

    • Open the document, which will appear just like on your Mac, including the toolbar at the bottom.

    • You can read and edit your text as you would on your Mac.

    • Pinch ‘Map’ to see your Defined Concepts.

    • In addition to moving your concepts right, left, up & down, as you can on your Mac, you can now also pull them towards you and push them away.

    • Your notes have been turned into knowledge objects with which you can now build spatial knowledge structures.

Defined Concepts as Nodes in Space

  • Use the visionOS interactions you are used to, to move, select and open your Defined Concepts. You can also use the commands on the Toolbar and more Advanced Gestures, as described below, to get more out of the spatial experience.

Interactions

Core Gestures

  • To Select a node, look at it and pinch.

  • To Open a node to see more information, look at it and double pinch.

  • To Move a node, look at it, pinch-and hold and move, then let go. Note that this does not select or deselect.

  • To Edit a node, look at it and long-hold-pinch. Note that the dialog to edit it appears on the toolbar, not on the node.

  • To Focus by hiding all nodes not selected or connected to, select node(s) and pinch both hands close together and move them apart, while still pinching. To un-Focus repeat gesture. (experimental)

  • To Deselect pinch-zoom in to center using both hands (the opposite of Focus).

‍ ‍

Toolbar Options

Depending on whether you have selected one node, several nodes or no nodes, the options in the toolbar will be different.

Always Available:

  • Focus hides any node not selected or connected to. This is very powerful to reduce visual complexity. If you select a node which was previously only connected/pointed to, any nodes it points to will then also become visible.

  • Views where you can choose to save or load layouts.

  • [D] to ‘Deselect’ all.

  • Select allows you to select by category, such as ‘People’ or ‘Events’.

  • Show allows you to reduce what to see, to only include what you select, also by category.

  • [A] to show ‘All’ nodes.

  • + to create a new node.

With One Node Selected you also get:

  • Snap moves any selected node back to their plane of origin. This gives you a useful plane to work with when you want to.

  • Hide allows you to hide selected nodes. You can choose to ‘Reveal’ them again later.

  • Like lets you ‘Like’ a node as a favorite, which you can use for selections later.

  • Edit to edit the contents of the node.

  • < to deselect the current node(s) and select any it links to instead of currently selected node.

  • = to select all other nodes of the same category.

Select Several Nodes and you get:

  • Layout where you can choose to configure the selected nodes spatially alphabetically or by time.

If you have Hidden or Liked any nodes you will also have the following:

  • Hidden to un-Hide.

  • Liked to choose to see only Liked, only Not liked, select liked or only Not Liked nodes.

When you Like and Hide, these options are available to you in Author macOS.

‍ ‍

Settings

  • You can specify themes, font size (for Write view), and more in Settings, top right.

Notes on ‘Context’

  • The notion of ‘Context’ is an aspect we are experimenting with, allowing you to define concepts as a Context to your current work.

  • In Author macOS, when you ‘Copy Defined Concepts’ and then go to paste them in a new document you have the option to ‘Paste Defined Concepts as Context’.

  • This is useful for curating a set of Defined Concepts over time which act as your context for new concepts. In visionOS you can choose to Select and Show where nodes marked as ‘Context’ are ignored, allowing you to have them in known positions in space and not worry about them being moved by accident.

‍ ‍

Notes on the ‘Toolbar’

  • The ‘Write’ view of your document does not disappear when you pinch ‘Map’.

  • A second Toolbar for the Map view appears on top of the standard Toolbar, giving you further controls.

  • If you want to concentrate on the Map, you need to reduce the size of the document, which you can do by looking at the bottom right of the document, pinching and re-sizing, as is visionOS standard interaction. Unfortunately this cannot currently be done automatically, at least yet. We are looking at ways to make this cleaner.

Further Notes

The audio for the video introductions was made using AI (using suno.com). This is part of experimenting with music as another medium to think about text, something we have been using for summaries of our Open Office Hours at the Future Text Lab, which you are very welcome to listen to and also, should you like, to join in, any given Monday: futuretextlab.info.

Mapping Knowledge

The goal with this project is to augment how you map your knowledge in order to better understand it, so we are developing functionality to support this. Below is a 2 min video showing you another perspective on how this works:

Further Work

What we are working on further is making the interactions even smoother and even more fluid. We are also working on making it more useful to see what is presented by the nodes and what they can show when opened. We are working on how to best move the nodes, how to see them connect, how to decide what should be context and what should be contents in terms selections and what to show and hide. There is much left to do. For more information about the thinking behind this work, have a look at our research on Spatial Thinking.

Author Features