Hey there! I’m Xinyue ︎︎
An interactive designer & creative technologist based in NY → READ MORE

Creative Tech ++ 
Art & Design ++


Flappy Melody


Collaborated with Monica Wen
December, 2018


Flappy Melody is an interactive game installation. The idea was inspired by Flappy Bird; users control the bird to avoid the coming barriers. In Flappy Melody, we use FerroFluid as the moving ‘bird’, its position changes by different tones, and the barrier would rotate and change at the same time, followed by the voice.

Players would use their voice, sing a melody to control the FerroFluid away from musical notes as barriers. ( *Ideally, once users hit the barrier, FerroFluid would drop and barriers go back to origin to start again.)


Interacting with the game would be a studying process – at first users will try multiple times to get the patterns of the game, and once they succeed in several moves, they will find the song we programmed as success is ‘Merry Christmas’ and finish the game.


  • p5js: sound library, to detect the frequency from the voice input
  • Serial Communication: sending pitch data to Arduino
  • 2 Belt & Pulley system to control the two motions
  • *Electromagnets to drop or attract FerroFluid for game setting (not added the project yet)

Each time users make a voice (with a different tone), p5 will send a message to Arduino to control the motions. A smaller stepper motor with 2 pulleys would rotate a specific angle and drive the barrier’s rotation; A bigger motor with 3 pulleys, and a magnet attached to the belt, would turn an angle corresponding with the change of the tone, the magnet on the belt would have a linear movement to control the ferrofluid by magnetic force. (*By counting the rotation counts, compared it with the turning angle of the bigger motor to determine whether players hit the barriers or not, and control the switch of the electromagnet.)

*Click here redirect to my blog for the detailed documentation.


Index    Next→