SYNCOPIX
UX Copywriting | Visual Design | Illustrations/Animations
The music you love, at your literal fingertips.
Meet SyncoPix, your new best friend from Logitech, crafted just for Spotify! This little wonder is like a magic wand for sound, blending machine learning with Arduino to let you dive into your music like never before. Get ready to explore, create, and groove with SyncoPix—where your sound gets its groove on!
Overview
The challenge? Create a cutting-edge intelligent camera that lets you control your Spotify app through various gestures. The project involved crafting a prototype that merges Machine Learning magic from Teachable Machine with the innovative tech of Arduino.
The result? A sleek, interactive camera that transforms how you experience and manage your music—just by moving your hands!
Mikah Chavez
Emma Reige
Raul Rodriguez
Diego Rojas
— members
— role
UX Copywriter
Visual Design
Illustrator
— duration
1 month
UX RESEARCH
Spotify challenged our design team to create a smart camera that lets users control their Spotify app with gestures. To bring this idea to life, we blended Teachable Machine, JavaScript, and Arduino into a sleek prototype.
We trained our machine learning model by capturing and analyzing gesture images, allowing it to recognize and respond with precision. The result? Spotylight, a camera that gives instant feedback through vibrant colors, letting you know your music commands are a go. All of this is delivered with Spotify’s signature style and flair, ensuring a seamless and engaging experience.
Background
Develop a High-Fidelity Prototype: Create a detailed, functional model of the intelligent camera, SyncoPix, that showcases its gesture control capabilities and interaction with the Spotify app.
Design Packaging and Manual: Develop user-friendly packaging and a comprehensive manual that effectively communicates the camera’s features and setup instructions.
Conduct User Tests: Perform thorough user testing to gather feedback on the camera’s performance, ease of use, and overall user experience.
Prepare Final Presentation: Compile and present the completed project, including the prototype, packaging, user test results, and key insights, to demonstrate the value and functionality of SyncoPix.
Goals
How can we create an intelligent camera that enables gesture-based control of the Spotify app, ensuring accurate recognition and immediate, intuitive feedback, while reflecting Spotify's unique style?
problem statement
Research into Teachable Machine and Arduino: Investigate how Teachable Machine and Arduino work together to build a gesture recognition system. This includes understanding their capabilities, integration methods, and how they can be utilized to create a responsive and accurate gesture-based control mechanism.
Research into Signs and Gestures: Explore different gestures and signs to ensure the camera accurately recognizes and responds to a wide range of user inputs. This includes understanding common gestures and their cultural contexts to enhance user experience.
Teaching the Model: Learn how to develop and train a machine-learning model to recognize and interpret various gestures. This involves gathering and preprocessing data, training the model, and refining its accuracy to ensure reliable gesture recognition.
Exploring User Interaction and Feedback: Examine best practices for providing visual feedback to users, ensuring that the camera’s responses are clear, intuitive, and aligned with Spotify’s user experience standards.
Prototyping and Testing: Develop and iterate on a high-fidelity prototype, followed by comprehensive user testing to gather insights and refine the design based on real-world use and feedback.
Methodologies
Communication through signs
While researching and exploring signs for training the model, we delved into American Sign Language (ASL). We discovered that integrating ASL was a brilliant way to use established signs for controlling SyncoPix, rather than inventing arbitrary symbols. This approach not only ensured that interactions were intuitive and meaningful but also significantly enhanced accessibility.
By incorporating ASL, we created an opportunity to include a broader range of users, particularly the deaf community. This inclusion means that users can control SyncoPix using a language they are already familiar with, rather than relying on a voice-based interface. It aligns with Spotify's commitment to accessibility and user-centric design, making the technology more approachable and inclusive.
With ASL, we embraced the idea of making controls accessible to everyone, ensuring that SyncoPix isn't just a gadget but a tool that resonates with and serves diverse user needs. This thoughtful integration helps maintain Spotify's signature style of being both innovative and inclusive.
Given the tight timeline, our initial goal was to focus on four essential signs: volume up, I like this song, volume down, and next song (based on the order of the photograph). However, we encountered challenges with the ASL symbol for "I like," which proved difficult for our model to recognize accurately. As a result, we had to modify this sign to improve recognition.
Looking ahead, we plan to expand our gesture set beyond these four basic commands. Future iterations will introduce additional signs, allowing users to have more comprehensive control over their Spotify experience without being limited to just these initial gestures.
product Design
In developing SyncoPix, our team combined Arduino and Teachable Machine to create an intelligent camera for gesture-based control of Spotify. We focused on four core gestures: next/previous song, volume down, I like this song, and volume up. Each team member took over 100 photos of these signs to train the model, ensuring it could accurately recognize and respond to user inputs. Despite encountering challenges with the ASL symbol for "I like," which proved difficult for the model to recognize, we successfully modified this gesture for improved accuracy.
The integration of Arduino with Teachable Machine allowed us to build a responsive prototype. Arduino facilitated the connection between the camera and gesture recognition system, while Teachable Machine provided the machine learning capabilities to interpret the signs. This setup enabled a smooth user experience, and with future iterations, we plan to introduce additional gestures, further enhancing SyncoPix's functionality and aligning with Spotify's commitment to innovation and accessibility.
Teachable machine
For SyncoPix's visual language, we chose to incorporate simple line-art illustrations paired with Spotify's vibrant color palette and distinctive design elements. By drawing inspiration from Spotify Wrapped campaigns and Spotify ads, we maintained the platform’s fun and unique personality throughout the design. The clean, minimalist line art keeps the interface intuitive and approachable, while the eye-catching colors inject a lively and dynamic feel.
To further amplify Spotify's playful tone, we included engaging animations that bring the visual language to life. These animations not only capture users' attention but also provide smooth, interactive feedback during gesture-based control. For example, the camera's response to different gestures is highlighted with animated color changes and fluid transitions, enhancing the overall user experience. This combination of static illustrations and dynamic animations ensures SyncoPix embodies Spotify's vibrant and entertaining brand identity, creating a cohesive and enjoyable interaction for users.
Visual language
copywriting
For SyncoPix, we crafted a cohesive and engaging copywriting strategy across all mediums, including advertising, presentation materials, and package design. Our approach involved using playful and conversational language that reflects Spotify's unique personality and keeps the tone consistent with its fun and vibrant brand identity.
In advertising, we highlighted SyncoPix's innovative features with catchy, upbeat phrases that capture attention and convey excitement. For the presentation, we focused on clear, dynamic copy that not only explains the technology but also resonates with Spotify's enthusiastic style. The package design was complemented with inviting, friendly text that makes the product feel approachable and accessible. By maintaining this lively and engaging tone throughout all touchpoints, we ensured that SyncoPix stands out and aligns perfectly with Spotify’s signature brand voice, delivering a unified and compelling message.
Usability testing
We conducted comprehensive usability testing, both in person and online, to evaluate how effectively our model recognized and responded to user gestures. Test participants were asked to recreate four specific signs in front of the camera, using Teachable Machine to determine how well the model had been trained. For online users, we shared a detailed manual that outlined the system’s functionality and provided clear instructions on the signs they needed to perform. This ensured that remote participants could engage with the product just as easily as those testing in person, reinforcing our commitment to accessibility.
Throughout the testing process, we received encouraging feedback, with the model successfully interpreting the majority of gestures. However, we did encounter challenges with the "I like this song" sign, as the model initially struggled to recognize the ASL gesture accurately. To address this, we made slight modifications to the gesture and provided the model with additional photos, further refining its ability to interpret the sign correctly. By incorporating ASL into our gesture set, we not only made the system more intuitive but also took steps to enhance accessibility, particularly for the deaf community. The inclusion of ASL allowed us to create a product that resonated with a wider audience, making SyncoPix more inclusive while maintaining an engaging user experience.
Usability testing Results
Overall Positive Performance: The model successfully recognized the majority of gestures during both in-person and online testing.
Easy and Intuitive Signs: Users found the gestures easy and intuitive, appreciating that the signs were based on a real language (ASL), which made the experience more meaningful.
Manual Support for Remote Users: Online participants easily followed instructions with the provided manual, ensuring smooth usability across different environments.
Surprising Responsiveness: In-person testers were often impressed by how well the camera and Spotylight responded to their gestures, providing immediate feedback.
Challenge with "I Like This Song" Gesture: The model initially struggled with recognizing the ASL sign, prompting adjustments to the gesture and additional training images.
Improved Accessibility: By incorporating ASL gestures, we enhanced accessibility for the deaf community, making the system more inclusive and user-friendly.
Key takeaways
The biggest challenges the team faced were navigating JavaScript and working within a tight time constraint. During the coding phase, when we were programming the camera to function as intended and connect with the model, several issues arose due to our limited knowledge of JavaScript. Fortunately, with the help of colleagues skilled in coding and support from advisors, we were able to troubleshoot and resolve the technical problems.
In terms of time constraints, we had to stay acutely aware of deadlines, ensuring we tackled every deliverable without compromising quality. Despite these hurdles, the project provided valuable lessons in time management and adaptability. We also gained experience in researching lesser-known fields and learned the importance of leveraging our networks for guidance when venturing into areas outside our expertise.
challenges
Before this project, my experience with physical product design and coding languages was limited, so diving into SyncoPix offered an exciting challenge. Not only did I get to learn new skills in hardware integration and model training, but I also developed a deeper understanding of the technical intricacies involved in product design. Working hands-on with Arduino and machine learning opened up a whole new creative and analytical side of design for me.
Though the research phase wasn’t as in-depth as other projects—since much of the foundational research was already provided—I was able to leverage my problem-solving and critical thinking skills to push the project forward. My ability to adapt quickly and navigate new tools ensured that we stayed on track despite the technical and time constraints.
In terms of visual design and copywriting, the challenge of capturing Spotify’s unique voice and tone was a true test of my creative flexibility. It required a balance between staying true to their fun and vibrant personality while ensuring the message didn’t feel exaggerated. I relied on my experience in branding, UX writing, and design cohesion to maintain consistency across all touchpoints, from the product itself to packaging and presentations. This project honed my skills in brand mimicry, creative writing, user-centered design, and developing visual languages that resonate with users—all while having a ton of fun crafting something fresh and innovative!