Based on the "Journey to the West" puppetry play, two virtual puppets were designed for the characters Monkey King and Princess Iron Fan. Two physical versions of the puppets were made by hand and 3D printing. Participants follow along the story Journey to the West and are able to take part in designing puppet movements, dubbing voices, and performing the opening monologue. Participants can experience the differences between four different ways of operating the puppets: the traditional physical method, VR with hand gestures, VR with gloves, and AR with full body movements. Guests are able to experience a whole new representation of traditional puppetry through the integration of technology and arts.
The name of the exhibition itself has two layers of meaning. First, the audience can witness the unseen hands of the masters behind the glove puppets. But also, they can also experience the unseen essence and spirit behind glove puppetry. The exhibition team was able to capture the spirit, the movement, the flexibility of the masters’ hands and represent it with new technology.
To record their movements, the puppet masters wore sensory gloves developed by our team. The attitude and flexure sensors within the gloves were used to capture the movements of the puppet masters. By comparing and analyzing the data collected, the information is transformed and visualized to showcase the dynamic beauty of the strength, flexibility, and motion trajectory of the hands behind the glove puppets.
Aiming to preserve the cultural context of puppetry as best as we could, our team attempted to incorporate into this project the authentic emotional experiences of the puppeteers. Though we did not have the opportunity to participate in the glory days of glove puppetry, through documentaries, oral history, and performances, we were reintroduced to this amazing art form. Through this creation, we hope to showcase forms and emotions in a way that has not been presented in the past. So that the public can truly appreciate the culture and spirit encompassed within.
Glove puppetry is a well-known Taiwanese folk art. Spanning across multiple mediums, from outdoor puppet shows to TV programs and movies. The post-production techniques of film and sound have brought amazing changes to the development of puppetry. Today, with the advancement of the integration of art and technology, we can expect to see alternative styles of puppetry.
After analyzing the subtle movements of puppetry, the production team attempted to reproduce these movements through VR technology, transforming physical puppet gestures into virtual puppet performances. With a VR headset, the audience can manipulate virtual puppets with their own gestures, experiencing the beauty through pure physical movement without the constraints of a script or plot.
In addition, guests are transported into an immersive environment with interactive backgrounds, sounds, and lighting effects.
Glove puppetry is an important intangible cultural heritage of Taiwan, yet it is gradually declining in today's society. Our team conducted a series of preliminary research and interviews to promote the cultural significance of puppetry. As a result, we redesigned the puppets while preserving their significant traditional features, and used modern designs to create two virtual puppets, Monkey King and Princess Iron Fan. Using digital technology to create puppets that merge tradition with modernity. The audience can interact with the two puppets with this new technology that aims to reduce user frustration and enhance interest among the public. In response to people's curiosity regarding the puppet master's gestures expressed in previous surveys, we preserved the beauty of the gestures and reinterpreted puppetry with new technology, creating an innovative artwork.