The Silent Carnival
Interactive Architectural Projection Mapping | 2025
Created for the 250th anniversary of Goethe’s arrival in Weimar, this seven-minute projection on the Weimar City Castle stages Goethe’s Faust.
The narrative intertwines fiction and history: in Faust, Gretchen kills her child yet is met with compassion, while in real life Goethe voted for the execution of a woman condemned for the very same crime. By confronting the gap between Goethe’s literary empathy and his historical judgment, the piece culminates with a QR code inviting the audience to choose Gretchen’s fate themselves.
Credit
Creative Direction, Art Direction, Music Direction, AI Integration: Lois He
Web App Development, Media Server Integration, TouchDesigner and Unreal Engine VFX: Vio Zhu
AI Video Generation: Lois He, Vio Zhu
Trailer
Video Documentation
Technical Details
To enable real-time interactivity, we developed a cloud-hosted web application where each audience member can scan a QR code to vote for the ending of the projection. When the QR code is displayed at the conclusion of the narrative, the system allows 30 seconds for audience input. After this period, we use a custom Python plugin for the MXWendler media server to send a request to the web application’s server side and retrieve the aggregated votes. The script then clears the results for the current round and triggers the corresponding ending sequence in MXWendler based on the majority choice.
Since MXWendler was the organizer’s chosen media server, we quickly learned how to work with it and extended its functionality by writing this plugin, enabling the projection to achieve its interactive, audience-driven nature.
Animation Production
The animation was developed through a hybrid AI–human workflow. We began by generating concept art and storyboards to establish the overall style and narrative direction. From there, we created key frames to define the aesthetic language and guide the animation approach. Individual animation segments were then produced using AI-based video generation methods.
To refine and integrate these outputs, we applied post-production visual effects with interactive media and real-time rendering software, enhancing the AI-generated material with dynamic motion and atmosphere. Finally, professional editing tools were used to smooth transitions and shape the animation into a spatial composition—aligning the visual flow with the architecture’s form and creating a seamless dialogue between projection and building.
Voting Results Summary
We implemented a detailed logging system in our web application to track user behavior and participation across three festival days. Over three festival days, the projection mapping was shown 44 times with 371 audience participants. Nearly half voted to save Gretchen, while others chose to side with the devil or leave the ending to fate. About 10% of voters required German-language support, with the rest comfortable in English. Participation rose significantly over the weekend, with total votes on days two and three increasing by 30% compared to day one.
