Emilie Zhang's Portfolio
Frisson
Paper accepted @ leading conference in HCI: the 2022 ACM International Conference on Multimodal Interaction
MIT Media Lab, Prof. Pattie Maes; tested device that triggers "aesthetic chills" or goosebumps
Team & My Role
UX Researcher and UI/UX Designer in a team of engineers at the MIT Media Lab, Fluid Interfaces
Authors of Publication
Frisson: Leveraging Metasomatic Interactions for Generating Aesthetic Chills
-
Emilie Zhang, Wellesley College
-
Abhinandan Jain, MIT Media Lab;
-
Felix Schoeller, MIT Media Lab;
-
Pattie Maes, MIT Media Lah
Timeline
February - August 2022
Tools
proto.io
Photoshop
There is a physical feeling of shivers down your spine or tingling on the nape of your neck and the back of your arms. That feeling is called aesthetic chills, or frisson. We built a device meant to trigger frisson. We have shown that our device can 1) reliably induce chills in participants at experimenter-chosen moments and 2) can recreate the downstream cognitive effects of chills, including increases in pleasure and empathy. This is a working, tested device, and we've already published results from studies. Learn more...
My Role:
-
Redesigned a working UI that will allow users to choose their preferred settings on the device.
-
Modularized the functions so that time, intensity, and location can be independent of each other.
-
Author section on UI and presentation for the academic paper accepted to one of the leading conferences in Human-Computer-Interaction, the 2022 ACM International Conference on Multimodal Interaction
The Overview
Opportunities to evoke emotional experiences and modulate cognitive processes hold great importance in studying relations between emotion, cognition and behaviour as well as building technologies for maintaining positive mental health. We present our device, Frisson, a metasomatic interface built to elicit the sensations underlying the embodied emotion of aesthetic chills (i.e., goosebumps, psychogenic shivers).
In contrast to other emotions, aesthetic chills provide unique characteristics to be useful for the study of psychophysiology of emotions. 1) The ’chills down the spine’ or ’shiver’ physiological response is easily detectable in both conscious percepts and physiological sensors; 2) they are spatially isolated with other emotions localized in the facial area (such as smile, sad, anger) because it is in the back/spinal region. 3) Aesthetic chills are regarded as universal emotions, found in almost all cultures with almost the same somatic marker.
The contributions of our work are as follows:
-
We introduce the concept of metasomatic interaction as a step towards designing effective emotion prostheses technologies.
-
We emphasize the notion of priors to determine what patterns of artificial sensations would invoke targeted emotional experience.
-
We propose the design and development of a Frisson wearable device and its user interface for controlling psychogenic shivers and their downstream effects on pleasure and engagement.
-
Preliminary evaluation of the technology in a user study (N = 14) suggests the effectiveness of emotion prostheses when the patterns of artificial sensations overlap with those of reported priors (i.e., somatic tuning is essential for effective emotion prostheses).
The Device
Figure 1: Frisson wearable prototype with three point and five point thermal feedback configuration and embedded vibrotactile motors. The block diagram shows the main components on the control PCB. To save space one peltier and one motor driver is shown.
To stimulate the area for aesthetic chills, we developed multiple wearable prototypes capable of delivering different patterns of thermal (and vibrotactile) feedback. The devices consist of three and five peltier elements as in Figure 1. The five point device becomes a super set of the three point device. The peltier locations were chosen to be covering upper back to lower back. While the three point device is ergonomic and easy to install, the five peltier device allows for multiple contact points and patterns of sensations.
The final device delivered thermal feedback in a manner closely resembling the internal chill, a traversing cold temperature across the back in a period of 3 seconds. Each peltier element is activated for a 1 second duration in series from top to bottom. An iOS mobile application was developed to activate the device, deliver chills and collect data in a time synchronous with the media stimulus.
My Contributions:
1. User Interface
While the engineered device already had numerous publications, without an interface, no one--save the engineers--would be able to operate it. That's where I came in. When I entered the team in 2022, the interface I was given had many problems--the most important one being a lack of modularity. Firstly, a 2D slider was being used to represent each node. That meant we could only adjust one thing on that slider: activation time OR intensity level. But to fully utilize all aspects of Frisson, we needed customization on three levels:
1. customize when each node was activated (activation time)
2. customize the intensity of each node when activated (intensity level between nodes)
3. customize changes in intensity levels for each node, within the times they are active (intensity level within nodes)
Over 6 months of mock-ups, meetings, and iterative design, I was able to bring the interface...
...from this...
Modularity: 2D
Customizable features:
-
which node is activated at which time slot ✓
Cons:
-
cannot set the intensity of each node ❌
-
cannot alter intensity over time ❌
...to this
Modularity: 3D
Customizable features:
-
which node is activated at which time slot ✓
-
setting the intensity of each node ✓
-
altering intensity over time ✓
2. Publication at 2022 ACM ICMI
All this culminated in our publication, Frisson: Leveraging Metasomatic Interactions for Generating Aesthetic Chills, that was accepted to one of the leading conferences in Human-Computer-Interaction, the 2022 ACM International Conference on Multimodal Interaction
The Process
The hardest problem was devising a modality that could handle all three features described previously (activation time, intensity levels between nodes, intensity levels within nodes). Taking inspiration from video and music editing software, I devised the first edition for Frisson's interface below. The first screen demonstrates what happens when you hover over the track representing Node A. Screen two demonstrates what happens when you click on the track, initializing a default intensity level. The third screen demonstrates what happens when you click on the intensity line of one bubble . It creates a white circle, which you can then drag to change intensities within the node. With this format, all three modalities could be successfully contained with one elegant representation.
The overall color scheme and foundational template is inspired by the music UI of Dmitriy Kharaberyush.
Iterating on Device Masks
Iterating and Incorporating
proto.io
Final Design
Home Screen
In the home screen, users will be able to choose between the 3 peltier model and the 5 peltier model. A button is included for future designs.
Apart from the control panel that would allow users to choose a song/video to set the chills to, there is a button to switch between the thermal and vibration modes, intensity levels for each node, and timestamps in seconds
3 peltier model
Thermal mode (3 nodes)
Vibration mode (1 node)
Thermal nodes use the alphabet, and vibration nodes use numbers.
In this example, node A is activated first, with intensity changing between 50, 75, 30. Node B's activation overlaps with Node A and its intensity slowly increases to 60. Node C activates last, with intensity dipping between 85 and 50. Node 1 on vibration mode has similar intensity changes to Node A.
Ex. Sequence
Thermal mode (3 nodes)
Vibration mode (1 node)
This is an example screen of when the timeline has been extended to 20 seconds. Users will be able to zoom in and out by pinching with two fingers.
Ex. Zoomed in
This is the default screen for the 5 peltier model. The first screen is the thermal mode, represented by the alphabet. The second screen is the vibration mode, represented by numbers.
5 peltier model
Thermal mode (3 nodes)
Vibration mode (1 node)
Different from the 3 peltier model, the node and track that the user is on is highlighted in yellow. This is because the nodes on the 5 peltier model are not clearly vertical
Ex. Sequence
Thermal mode (3 nodes)
Vibration mode (1 node)
This is an example screen of when the timeline has been extended to 25 seconds for the 5 peltier model. Users will be able to zoom in and out by pinching with two fingers.
Ex. Zoomed in
Final Design Screens
