top of page
Music Tactilization
Wearable, Tactile Design, UX Research
PROJECT SUMMARY
This is my master's project that aims to investigate tactile user experience of listening to music. Based on the subjects' emotional responses in Emery Schubert's study, my project is one step further to bridge 2 tactile variables, vibration intensity and location, with 2 emotional variables, arousal and valence.
ROLE
Programmer, Maker, Researcher
TOOLS/METHODS
Pilot Study: Literature Review
Hardware: Arduino
UI/Data Collection: Processing
Data Analysis/Visualization: Matlab, Pearson Correlation, T-test
DELIVERABLE
MOTIVATION
WHERE IS TACTILE
Now we interact with digital products mainly through visual and auditory channels. Thus we have decent guidelines, frameworks, principles for good visual and acoustic design.
We are missing tactile interface in communication and information consumption.
WHY TACTILE
Intuitive when visual and audio perception channels are under overload condition.
Immersive in media consumption.
WHAT MAKES DESIRABLE TACTILE EXPERIENCE
You might not know. I don't know either. Then I have this project:)
INSPIRATION
Emotion is a promising dimension when defining good user experience. So I researched on 2 fields :
AFFECTIVE HAPTICS
Haptics has been proved to influence users' emotion. But relevant projects are too varied in terms of :
-
user scenarios (navigation, entertainment, notification, etc)
-
platforms (gloves, jacket, mobile phone, etc)
MUSICOLOGY
Measurement and time series analysis of emotion in music, Schubert (1999)
Music also influences emotion. In MEVD (Music Emotion Variation Detection) projects, subjects' emotional responses to music were measured and proved to be valid to be applied to other people. But none of them tried to map music features to tactile patterns.
Hence, I wanted to consolidate tactile design and go one step further, bridging music with tactile patterns, if any, through aroused emotions.
PROPOSED DESIGN
Based on my research findings above and many others, I identified three key components for this project :
HARDWARE PLATFORM
A vest to convey tactile effects to subjects.
EXPERIMENT DESIGN
An experiment needs to be conducted to quantify the relationship between emotional responses and tactile variables.
DIGITAL INTERFACE
A desktop application to listen to selected music, to manipulate tactile effects and to collect data with.
Two research questions need to be answered :
CORRELATION
What is the correlation, if any, between vibration intensity and emotional responses ?
SIGNIFICANCE
What is the significance, if any, of the relationship between vibration spots and emotional responses ?
PROJECT ARCHITECTURE
Emery Schubert's work
My work
ITERATION AND IMPLEMENTATION
UI DESIGN
I started sketching on paper to come up different designs, aiming to help subjects to complete the experiment.
THE VIBRATION VEST
Not everyone can fit in a down jacket. Some slim participants in my first test were unable to feel the vibration. In my later prototypes, I changed the garment to a high visibility vest.
THE EXPERIMENT DESIGN
Throughout the repetitive discussions with my team, Aoxing Zhang (data analyst) and Michael Nitsche (advisor), we kept on refining the experiment flow that could be both subject-friendly and well structured to yield valuable data.
FINAL DESIGN & IMPLEMENTATION
The UI walkthrough (developed in Processing 3)
The finalized vibration vest
12 subjects (6 males/femels) participated in the 30-minute experiment
A sample data sheet
RESULTS & DISCUSSIONS
The data shows no correlation between vibration intensity and emotional responses. Below are the scatter plots of 12 subjects' mean of vibration intensity. (when they were asked to listen to 4 selected music pieces, Adagio, Morning, Slavonic Dance and Pizzicato Polka)
The result happens maybe because :
-
The music pieces didn’t affect participants’ emotions during the experiment. In order to accomplish the experiment, however, subjects chose to "tactilize" music features in the music, such as the pitch and tempo.
-
Another explanation could be, for the same vibration pattern, different people might hold different meanings.
The data also shows the significance of the relationship between different vibration spots and subjects' emotional responses varies.
The significance of the relationship between different vibration locations and emotional arousal varies
The significance of the relationship between different vibration locations and emotional valence varies
FUTURE WORK
Based on some problems in this project, I list some directions worth considering for iteration :
-
Give subjects the freedom to choose their own music pieces so that they might find themselves more emotionally related to the music
-
Use qualitative methods for the mapping process instead of quantitative ones (e.g. Pre-define some tactile "effects" and ask subjects to choose the most relevant ones based on some given descriptions)
-
Design a more intuitive interface to eliminate the latency of subject's responses (e.g. A mobile app or a gesture control app)
bottom of page