east to west.

Pic: Thanos Pal | Unsplash

tactile design language for indoor-outdoor pedestrian navigation.

"Like Google maps, but better." That's what a research participant (aka friend) said after trying it on :)
Grad thesis, CMU Design for Interactions, 2018. Thank you, Molly Steenson and John Zimmerman.
Let's talk about why this is my favorite work.

Be sure to scroll all the way ↓ to see some fun experiments.

👉 Watch the video if you're in a hurry. Got time? Download thesis.
🙏 Dhanyavād for your voice and performance, Ashlesha Dhotey.

Abstract

People navigate through indoor and outdoor spaces all the time. These environments are rich with visual and audio information, and noise. This poses a challenge to someone trying to navigate in a new environment. While GPS services like Google maps help, they demand visual attention in an already visually stimulating environment, other than keeping one hand busy. Audio-based turn-by-turn navigation, while useful in certain scenarios, temporarily mutes the outside world for a pedestrian. In addition, these services do not work very well for people with visual and hearing impairments.

tac.tic is a tactile design language for indoor-outdoor pedestrian navigation. It consists of a vibrotactile interface that navigates a user through complex environments, both indoors and outdoors, by drawing patterns on the forearm. Apart from communicating directions that help navigate people, the design language aims to communicate the complexity of indoor environments, such as going left vs. going up the stairs to the left vs. going down the stairs to the left. Through a process of iterative prototyping and testing with people, the result is a preliminary language for navigating pedestrians within the built environment. It also paves the way for designers to design experiences beyond the visual.

Sketching with hardware

Despite being an established research field, designers lack the knowledge of designing tactile interfaces due to the lack of experience working with the medium. This has led to an absence of design patterns and norms to guide interested designers and turned into a chicken vs. egg problem. Additionally, multiple works of research have noted that the vocabulary around the subject is limited, making it hard to verbalize and communicate our experiences from the tactile sense.

The nature of this subject and its reliance on the sense of touch makes it extremely difficult to explore without being hands-on. Keeping this in mind, ‘Make to Learn, Learn to Make’, is the ideal to learning adopted for this study, from Camille Mousetti’s work on Simple Haptics. Following a practice of sketching in hardware and building various prototypes allow for a wider range of tactile experiences.

I had no experience working with tactile interfaces or in electronics prior to starting this study. It was a cyclic process of making from what I know and learning what I want to make. Throughout the year, prototypes were built (most times semi-working), tested with people and changes made based on the observations; moving from design to making to testing and back to design within the same day on many occasions.

I'm skipping the process and jumping straight to the design language here, but if you're on a comfy chair, download thesis.

⚡️ From messy desks to mild shocks, it was all worth it!

The design language

tac.tic is a design language for a tactile sleeve that navigates pedestrians through complex indoor-outdoor environments. It uses a 9-motor 3 X 3 square grid to communicate different navigational instructions on the forearm, in the form of a felt pattern.

The language can produce more than one hundred patterns, identifiable without experiencing a steep learning curve. Each pattern consists of three parts - Direction, Feature and Level change. Direction consists of the 8 basic cardinal directions which tells people which direction to walk in. These are left, right, straight, behind and the four diagonals.

Eg: ‘Go left’ can be communicated using only the direction. Feature communicates the architectural element like stairs, escalator, ramp and elevator, and is always combined with the level change which communicates whether to go up or down. The direction, feature and the level change come together to communicate complex directions.

Direction → Left, Right, Straight, Behind, Diagonals
Feature → Stairs, Escalator, Ramp, Elevator
Level → Up, Down

Direction (Left) = Go left
Direction (Left) + Feature (Stairs) + Level (Up) = Go up the stairs, to the left

Get attention

During evaluation, it was observed that the first time people felt the pattern, it only got their attention. They identified it the second and third time they felt it. In this regard, most patterns (except on track, stop and destination reached) are preceded by a vibration. The motor at the center of the grid vibrates for 1 second, followed by the pattern. This proved helpful to get people’s attention and then communicate the direction.

Get attention,
left

The vibration takes attention and the direction follows. The basic directions (Left, Right, Straight, Behind & Diagonals) are communicated by drawing a line on the arm, in the respective direction.

Diagonal,
towards the right

The pattern for diagonals was not identifiable during the prototyping and evaluation sessions. The new pattern takes advantage of the sensory funneling illusion which helps create a virtual vibration at a particular location even though no vibrotactile motors are present in that location. This helps the pattern draw a curved line towards the diagonal direction. This iteration of the pattern can be distinguished better from the patterns for ‘left’ and ‘right’.

Complex directions

The language communicates more complex directions through the sequential expression of the direction, feature and level change. While the basic directions are communicated by drawing a continuous tactile line on the arm, the feature and level change are communicated by varying the timing in between vibrations, direction of movement, duration and intensity of vibrations. Going up is communicated by increasing intensity and duration of successive vibrations in the respective direction, while going down is communicated by decreasing intensity and duration of successive vibrations. This is explained further using multiple examples.

Factors manipulated
1. Direction of movement
2. Time in between vibrations
3. Duration of vibration
4. Intensity of vibration

Up the stairs,
to the right

‘Up the stairs, to the right’ is composed of Right (pattern moves from left to right) + Stairs (successive vibrations are not continuous, they are activated one by one) + Up (the duration and intensity of each vibration increases).

Down the stairs,
straight

‘Down the stairs, straight’ is composed of Straight (pattern moves from bottom to top) + Stairs (successive vibrations are not continuous, they are activated one by one) + Down (the duration and intensity of each vibration decreases).

Going up → Increasing intensity and duration of vibrations
Going down → Decreasing intensity and duration of vibrations

Ramps

Ramps form a different pattern on the skin. It is a combination of an individual long vibration that defines the level change and a line drawn in the respective direction. If the individual long vibration precedes the line drawn, it denotes going down. If the individual long vibration follows the line drawn, it denotes going up. The rule stays consistent across patterns - higher intensity and duration of vibration indicates a higher ground and lower intensity and duration of vibration indicates a lower ground.

Up the ramp,
to the left

‘Up the ramp, to the left’ is composed of Left (pattern moves from right to left) + Ramp (a combination of an individual longer vibration and a line drawn on the skin) + Up (the individual long vibration is felt at the end of the pattern).

Down the ramp,
to the right

‘Down the ramp, to the right’ is composed of Right (pattern moves from left to right) + Ramp (a combination of an individual longer vibration and a line drawn on the skin) + Down (the individual long vibration is felt at the beginning of the pattern).

Elevators

Elevators are communicated by increasing and decreasing intensity of vibrations, as well as the movement of the pattern along the arm. Elevators are the only patterns where each pattern in the sequence (directions are a series of 3 patterns as discussed previously) moves along the arm.

Up/down the elevator

‘Up the elevator’ is communicated by gradually increasing the intensity of vibrations as well as the pattern moves up along the arm. ‘Down the elevator’ is communicated by gradually decreasing the intensity of vibrations as well as the pattern moves down along the arm. The current iteration does not communicate the destination floor.

During testing, I found out that in most cases it is not necessary to communicate whether the one should go up the stairs, escalator, ramp or elevator as long as the communication for going up is clear. However, the individual patterns were still maintained as part of the language so it can be used in scenarios that need the clarity. It also shows how people are able to identify tens of patterns and distinguish between them over a short time.

The evaluation revealed a critical need communicate the next two directions at the same time in certain scenarios, such as when there is not enough time to communicate the second pattern after the user has taken the first turn. To address this, I developed patterns that could communicate two turns at once. There is a pause in between the two turns to indicate that they are two distinct actions for the user to take.

Left,
then right

‘Left, then right’ is a combination of the pattern for ‘left’ and the pattern for ‘right’.

Right,
then up the stairs to the right

‘Right, then up the stairs to the right’ is a combination of the pattern for ’right’ and the pattern for ‘Up the stairs to the right’.

U-turn

A pattern for ‘U-turn’ was designed for scenarios that involved turning 180 around and walking in the direction behind the user. This is a combination of straight, left or right depending on the side of the turn, and behind. There is no pause in between the patterns to indicate that it is a single direction and not three different directions.

Apart from these, three patterns designed that are important in the context of navigation were ‘On track’, ‘Stop’ and ‘Destination reached’.

On track

‘On track’ is composed of successive vibrations of low intensity, of a single motor, to indicate to the user that they are on the right path. This proved to be a hugely liked pattern during the study. This will be communicated every 25 meters when the user is expected to walk over 50 meters on the same path. Also, this is used when there is only one path ahead. In the case where there are multiple path possibilities, ‘straight’ is used to indicate moving forward.

Stop

‘Stop’ is composed of 5 simultaneous vibrations of highest intensity for a longer duration. This tells the user to stop. The most common use case for this pattern is when the user goes in the wrong path.

Destination reached

‘Destination reached’, another favorite of the study participants attempts to replicate a ‘happy dance’ using vibrations. Each of the 9 motors vibrate one after the other in a random order.

Looking back

Indoor navigation is dependent on improvement in the field of localization. While a lot of work is currently being done towards improving the accuracy, the design will not work successfully at the current state of the technology. The design itself was heavily based on the hardware that I could build with my amateur experience with electronics. The inclusion of an expert to the project may take it a in different direction. Learning to distinguish between and identify the patterns involves a learning curve, albeit not very steep. While the interface does not demand the user’s visual attention, it still requires their cognitive attention in identifying the pattern and mapping it to the environment. Also, the language was designed for the built environment and will not work in organic environments (Eg: While hiking in the mountains) in its current state.

a platform for silent, non-audiovisual communication; a physical manifestation of digital information.

Other work experiments

to cross, or not to cross

A system for autonomous vehicles to communicate with pedestrians, whether the car is stopping/has stopped for them to cross or not. Team of 4.

ditto

...creating something delightful that makes people be in the moment in their daily lives and reflective of what they are projecting to the world through their facial expressions. Team of 4.