Lee-ding the Beat is a Dance Dance Revolution-inspired AR experience exhibited at the LKY: The Experience in Singapore. The experience transforms the iconic poses of Lee Kuan Yew into dance moves enabling the user to step into history in a fun and engaging way. The project was executed by Serial Communication a Singapore-based creative agency. I worked as a creative technologist on the project to design and execute some key components of the experience. The experience uses HTML/CSS and JavaScript with p5.js along with ML5 for pose detection. I primarily worked on integrating pose detection with an AR garland that drops on the users when they successfully do a pose that is prompted on the screen.
The biggest challenges in this project were hands-free free user-experience and performance. The team went through several iterations of the onboarding process to ensure that the users could understand the mechanics of the game within a short span of time without having to swipe through or read lots of instructions. This being a public installation, we also had to consider edge cases like users leaving or entering mid-game, more users than the game is designed for, etc. The other main challenge was to keep the game performant while delivering delightful interactions.
