Performative Media
Performative Media.pdf，作者 tian dongWEEK 1-Welcome to Performative Media
This week, the teacher began by focusing on the core concepts of the course:
1. Core Definition: As an artistic practice, its core is to design spaces and environments, creating interactive experiences that integrate humans, moving spaces, and primitive elements. The core difference lies in transforming the audience from "passive observers" to "active participants."
2. Three Key Elements: The course must be structured around an "input-processing-output" logic. Input relies on sensors (such as microphones and motion sensors), processing relies on code and programming logic, and output is presented as visual, audio, or dynamic interactive effects.
The teacher introduced the course software tools to download this semester.
The focus will be on TouchDesigner, a node-based real-time programming tool that supports sensor data integration and can be used to create visual and audio content and interactive effects. It is a core tool for creating interactive installations and performances in the industry.
The teacher explained the course assignments:
- Assignment 1: Proposal Revision
- Assignment 2: Prototype Layout
- Final Project: Performative Media Interaction
- e-Portfolio
WEEK 2—Creative Coding and Generative Art
The teacher reviewed last week's content:
Last week, we learned about the concept of performative media and the principles of input-process-output interaction and explored the interplay between people, space, sound, and images.
Creative Coding and Generative Art Concepts
- Creative Coding: Using programming to create art, not just solve functional problems.
- Generative Art: Setting rules and algorithms to allow a system to automatically generate images or sounds.
- Core elements include:
- Rules and Algorithms
-Randomness and Noise
- Repetition and Iteration
-Emphasis on the "balance between order and chaos": too much randomness leads to chaos, while too many rules lead to rigidity.
The teacher played several artist videos to illustrate the thinking and process of generative art.
In class, the teacher guided us to download and install TouchDesigner (the main tool of the course).
And took us to practice using the software in class.
Practice process
Video after practice saved
The teacher explained the content of Assignment 1 in class：
We need a team of four to complete this, with a deadline of October 17, 2025. Teams are required to select two international/regional/local media artists or groups (each artist should research two representative works, for a total of four works). The presentation should be completed using a 15-20 page PPT (exported as PDF). The presentation must include three sections: artist background (2-3 pages per artist), case study (4 pages per work, covering six dimensions: background and purpose, technical tools, IPO model mapping, audience role experience, aesthetic atmosphere analysis, and effectiveness evaluation), and reflection and application (3-4 pages). The presentation must also include relevant visual materials and references from at least three credible sources (APA/MLA format). The presentation will ultimately be evaluated based on six dimensions, including research depth, case analysis, and reflection on aesthetic experience, according to the corresponding scoring criteria.
Week 3: TOPs Deep Dive in TouchDesigner
This week's Media Arts class focused on further studying TouchDesigner's visual generation and compositing logic. The teacher first reviewed the basic concepts of TOP (Texture Operators), CHOP (Channel Operators), and SOP (Surface Operators) from last week, and further explained how to use them to achieve image overlays and dynamic visual effects.
The teacher reviewed the concept of "generative systems," discussed in the previous class, which involves generating variable visual results through the repeated execution of rules and algorithms.
• Randomness: Completely unpredictable numerical variations.
• Noise: Structured randomness with a certain degree of continuity and controllability.
In creation, the balance between randomness and order can create a visual sense of "liveliness," a key characteristic of generative art.
TouchDesigner Hands-on Exercise: Composite & Feedback
1. Composite Operations
Learn how to overlay multiple images (such as text, shapes, and backgrounds) on top of each other.
Understand how different compositing modes (Over, Multiply, Add, etc.) affect visual hierarchy
In actual operation, we use Constant, Circle, Transform, Composite and other nodes to create the effect of layer overlay.
Final product video
Document
2. Feedback
By re-inputting the output into the system, a continuously changing dynamic visual effect can be achieved.
This method can be used to create "smearing," "afterimages," or a sense of fluidity.
The instructor demonstrated how to adjust parameters and resolution to achieve subtle changes in the image within each frame.
Final product video
Document
Finally, the teacher reminded us that next week we would be conducting group presentations (analyzing two artists and two works each). Each group was required to prepare a 15-20 page slide deck and a 10-minute presentation. Individuals were also required to complete written reflections and applications.
Week 4: Assignment 1 pre and Working Towards Our Final Project
This week we gave presentations for each group on Assignment 1.
After the presentations, the teacher gave a brief introduction to Lecture 2.
The teacher mainly talked about an important transition—from the "concept stage" to the "practice and execution stage." In the past few weeks, we have been mainly doing concept research and experimental ideas (SP concept), but now we have to actually "make works" and enter a more practical stage.
The instructor mentioned that the past few weeks have helped us understand how to think about the concept, context, and form of our work.
Starting this week, the focus will shift to hands-on practice and execution.
At the same time, we will learn how to make more nuanced choices in our designs, such as materiality, lighting, and interactive methods.
The instructor encouraged us to try more experimental tools, such as TouchDesigner, and explained that further in-depth instruction will follow, allowing us to learn how to integrate visuals, sound, and interactivity into our work.
📌 Homework and Next Steps
The teacher mentioned that the next assignment is:
"Concept Proposal + Prototype."
The goals of this assignment are:
• Present a complete and clear design idea;
• Express the concept through experiments and small-scale models;
• Lay the foundation for the final project direction.
In other words, this assignment is the first attempt to move from "idea" to "product."
The teacher emphasized:
• Don't worry too much about whether your work is perfect; what's more important is to discover problems through experimentation;
• Keep a record and reflect on your practice, making every attempt a part of your research;
• A work isn't just about the "result"; what's more important is the "process" and the "depth of your thinking."
Week 5 Class Reflection—Perform & Interactive Audio Systems
This class primarily helped us transition from the research phase to the creative phase. The instructor explained that we now needed to begin conceptualizing and experimenting with our final interactive installation project. Having previously studied artist examples and theories, we now needed to translate these concepts into our own work.
Our task was to create interactive experiences through sound, sensors, and visual feedback, ultimately creating an engaging installation experience. The instructor emphasized that perfection wasn't necessary, but it had to "express the idea"—this was the most important goal during the conceptual prototyping phase.
TouchDesigner Technical Learning Content
This week's practical focus is TouchDesigner's CHOP (Channel Operator) and TOP (Texture Operator). The instructor guided us through the basics, gradually building an interactive visual system.
1. CHOP Basics (Green Nodes)
• CHOPs are used to process data and sensor input, such as the mouse, microphone, and keyboard.
• We learned operators such as Constant, Math, Select, Logic, Trigger, and Audio In.
• We understood the data flow: input → transformation → output.
2. TOP Basics (Blue Nodes)
• They are used to generate and blend images, such as Constant TOP, Circle TOP, and Over TOP.
• We learned how to control TOP parameters through CHOPs, such as moving a circle with the mouse position and controlling its size with the volume.
Practical Exercises
In class, the teacher walked us through two small systems:
System 1: Mouse-Interactive Circle
Move the mouse to control the circle's position, and click to change its transparency. Use Select and Map to adjust the input range.
System 2: Visualizing Sound Responses
Using Audio In CHOP, we convert sound waveforms into numerical values and then map them to the radius or position of a circle, achieving a sound-driven visual response.
Extended Application: Sound and Keyboard Interaction
The teacher demonstrated how to use Keyboard In CHOP to switch between different color visual outputs. Each keystroke can trigger a different color or image change. Multi-state control is achieved through Logic CHOP and Switch TOP.
This helped us understand how to combine multiple inputs (sound, motion, keyboard) to create more complex interactive logic.
In the second half of the class, the teacher demonstrated more advanced examples using Audio Spectrum CHOP and Trigger CHOP:
• Audio Spectrum can analyze frequency and rhythm to create visuals that move with the music.
• Trigger can trigger visual events based on volume thresholds, such as a drum beat lighting up or a shape flashing.
The teacher also introduced TouchDesigner's Palette (preset tool library), including the "Audio Analysis" module, which directly analyzes the rhythm, energy, and beat of music, making it ideal for performance and interactive lighting design.
Next Week's Plans
- Continue developing interactive concepts and visual prototypes.
- Begin thinking about the structure and theme of the final work.
- The instructor encouraged us to experiment with combining audio, visual, and sensor input to create a work that responds to audience behavior in real time.
Document
Week 6: Embodied Interactivity using Webcam in TouchDesigner
Through case studies and theory, the teacher helped us redefine the boundaries of "interaction"—no longer limited to traditional input devices like mice and keyboards; the body itself is the best tool for interaction.
The teacher showed us a particularly interesting case: an "interactive shadow installation" located in a city center square. Inspired by the perceptual techniques of Dutch painters, this installation uses a camera to capture ground shadows 20 times per second. When a pedestrian's shadow "reveals" a pre-set portrait pattern, the hotspot turns white and triggers a brief black screen, after which a new portrait appears in the new location. What impressed me most was its "naturalness"—pedestrians didn't need to learn any rules; they would instinctively adjust their bodies and shadows, even striking various creative poses, achieving truly "barrier-free interaction."
The teacher also emphasized that good interactive design should meet three key points:
1. Intuition: Users should be able to participate instinctively without complex learning.
2. Theme Relevance: For example, this installation uses "shadows" to echo the art theme of "perception and observation," ensuring the interaction and concept are not disconnected.
3. Surprise: The appearance of a black screen or a new portrait can continuously attract users, making the interaction less monotonous.
Practical Technical Training: Implementing Visual Interactive Effects
The teacher posted a link on mytimes about the steps we would be taking with Touch Designer in this lesson. Each step was explained very clearly. He first asked us to try it ourselves, and he would help us answer any questions we had.
The teacher then guided us through breaking down the implementation logic of "dynamic visual feedback," the core of which is "frame comparison using cache," with very clear steps:
1. Frame Cache (Cash): The program stores the most recent 32 frames (or 128 frames, adjustable), essentially "saving historical frames";
2. Frame Selection and Comparison: Select two frames from the cache (e.g., the current frame and frames from 6 frames ago) and calculate their differences. This step is crucial for "capturing dynamic changes";
3. Threshold Adjustment: Set a value to filter out background noise in the image, retaining only obvious dynamic edges (such as hand movements or body movements);
4. Effect Optimization: Add blur, displacement, saturation adjustments, etc., to make the visual feedback smoother and avoid a harsh feeling.
The teacher also encouraged us to do extended experiments, such as adding "audio response" to this visual effect—using the audio analysis tools learned last week to make the image change with the rhythm of the sound.
If anyone is struggling to keep up, the teacher said they would upload a complete demo file, and by adjusting the parameters, you can gradually get started.
Exercise Document
Key points of the assignment: Connection between Assignment 2 and the final project
The instructor clearly outlined the three main components of Assignment 2 (Prototype and Concept Proposal), which are directly related to the final project. Everyone should pay close attention to them in advance:
1. 10-Minute Presentation (Submitted by November 21st)
• Don't dwell on details; focus on the "core concept" and "key interactive methods," such as what problem your device aims to solve and how users can participate.
• Be clear, allowing the audience to quickly grasp your design logic. Don't strive for a "perfect presentation," but ensure your ideas are feasible.
2. Supporting Documentation
• Clearly explain the technical details (what sensors were used, how the interaction was implemented), and the team's division of labor (who was responsible for the creative idea, and who was responsible for the technical aspects).
• Include visual materials: flowcharts, technical sketches, and reference case images are all acceptable to help the audience understand your solution more intuitively.
• Anticipate "potential problems": For example, what if the camera's recognition is inaccurate in strong light? Clearly explaining the problems and solutions will earn you extra points.
3. Prototype (Proof of Concept)
• Key point: Perfection is not required; the goal is simply to "prove the idea is feasible!" For example, if you want to create a "gesture-controlled light" device, you don't need to build a complete installation. Just implementing the core interaction of "waving your hand → light changing color" through programming is sufficient.
• The instructor strongly encourages "trial and error": even if the prototype has bugs, as long as it demonstrates your experimental direction (such as trying new sensors or new visual effects), it's more valuable than "creating a perfect prototype without any highlights."
Additionally, the instructor offered a small suggestion: find at least five interactive examples from different fields for reference (such as art installations or interactive products). If you're unsure where to look, you can ask the instructor for a recommendation list.
Week 7: Integrating MediaPipe for Extra Interactivity
This week's course focused on the combined application of TouchDesigner and MediaPipe. The instructor guided us through an in-depth study of how to achieve interactive video control using body tracking and hand tracking.
At the beginning of the course, the instructor reviewed last week's concepts of body interaction systems:
A feedback loop is formed between the artwork and the viewer;
The viewer's body movements become the input, and the artistic response is the output;
Interactive instructions should be intuitive, natural, and easy to understand;
All interactions must support the artistic theme and expressive purpose, and cannot be merely complex technical effects.
Subsequently, the instructor introduced how to use the MediaPipe plugin in TouchDesigner and guided us step-by-step in building an interactive screen that tracks gestures. Key learning content included:
- The components of MediaPipe nodes: such as face tracking, hand tracking, and pose tracking.
- How to use output data: how to use hand coordinates (X/Y) to control the position and size of screen elements.
- Nodes such as CHOP, TOP, Math, and Limit are used to smooth motion, adjust numerical ranges, and control mapping.
- The addition of feedback and noise effects makes the images more fluid and artistic.
- Finally, the instructor demonstrated how to control the size of the "brush" using hand gestures, allowing the gestures to create ripples and color changes on the screen like a brush.
This lesson helped us gain a deeper understanding of the logic behind interactive system design and learn how to completely transform data flow into visual effects.
Practice finished product demonstration video
🧩 Project 2 and Final Project Explanation
Our instructor reminded us that the focus of Project 2 is on "demonstrating the concept and key interactive logic," without needing overly complex physical construction.
The demonstration video should simply showcase the interactive prototype.
The documentation will explain the concept, feel, and technical logic.
This project will directly help us prepare for the **Final Exhibition**.
The instructor also mentioned that the final exhibition is expected to be held in Week 15, possibly at GMBB, where we will actually showcase our work, combining software and physical space elements.
He suggested that we begin experimenting and testing the interactive logic in the coming weeks and finalize the concept and equipment requirements by early December.
Week 8: Designing an Effective Proposal & Prototype
This week's class marked the beginning of a crucial phase in project development. The instructor repeatedly emphasized that while our concept was taking shape, the most important thing now was to "concretize, make it actionable, and make it demonstrable." The entire lesson revolved around three core elements: concept refinement, interactive design structure, and space/materials and prototyping.
In the first half of the lesson, the instructor started by addressing the overall project direction, reminding us to return to the fundamental questions: What is the core concept of the work? What do we want the audience to feel from this work? Any action the audience takes during the interaction must be related to the concept; it can't be as simple as "pressing it to light up." Interaction should be an extension of emotions, experiences, and the story, and both visuals and actions must "express" the theme, rather than simply providing a rigid explanation in words. The instructor stressed that a mature work shouldn't require lengthy explanations, but rather allow users to understand it naturally through visuals, triggering methods, and spatial atmosphere.
Next, the instructor guided us to re-examine the structure of interactive design: Input → Process → Output → Meaning. Whether it's movement, gestures, images, touch, or multi-person participation, everything should be clearly recorded and drawn into an interactive flowchart. The teacher repeatedly reminded us that good interaction should be "intuitive and a little surprising," encouraging the audience to explore, not just a simple button-triggered relationship.
When discussing space and structure, the teacher pointed out that the physical space of the installation is not just a place to put equipment, but an extension of the concept. Projections, light and shadow, materials, and shapes should all speak the same language as the concept. For example, if the atmosphere of the work is soft and healing, then it's impossible to use sharp, industrial-textured materials; if projection is the primary medium, then fabrics, gauze, and reflective materials must be tested in advance. The teacher also gave many examples of how to create actual installations, including using PVC pipes, discarded objects, second-hand materials, fabrics, and plastic sheets, emphasizing that the prototype stage doesn't require perfection, but it must have clear functionality and a stable structure.
The second half of the class focused on planning the prototype development. The teacher asked us to clearly define priorities for next week's presentation: "Core interactions must be completed, secondary interactions can be reduced or postponed, and the art style can be gradually added later." Each group needed to write down their most important interactive goals, such as: head movement triggering visual changes, light affecting patterns, and positional changes causing interface transitions. He hoped to see a "usable, functional" prototype next week, not just a conceptual model.
During his rounds, the instructor offered numerous suggestions regarding our work. These included issues like shadow occlusion, projection angles, camera recognition areas, the need for Kinect, and how the prototype space should be isolated. He reminded us that discovering problems during testing was perfectly normal; the key was being able to propose solutions, such as changing the projection position, adding occlusion cloth, adjusting proportions, or clearly defining the interactive areas. He also emphasized writing a good "story and meaning" for the work, like a description on an exhibition wall, allowing viewers to understand the work's emotions, intentions, and the experiential value behind the interaction.
Overall, the biggest takeaway from this class was realizing that the focus of the prototyping stage is not "perfect visuals," but rather "clear, functional, and conceptually expressive core interactions." The work needs to integrate concept, space, materials, behavior, and the audience experience, and we are currently at a crucial turning point in piecing all these pieces together. Over the next week, we will strengthen the interactive process, revise the prototype logic, and tell the concept story more completely and coherently, based on the teacher's guidance.
Week 9: Assignment 2 - Presentation and introduction of the Final Assignment
Our group gave a presentation in class, and the teacher gave us some feedback.
He acknowledged that our technical direction is strong and that we already have a clear understanding of the interaction flow and touch-based logic. The prototype successfully demonstrates the main interactions, showing that our foundation is solid. He also appreciated our awareness of space, lighting, and materials, especially considering that we are working within a small installation area. However, he reminded us that projection visuals may be weakened by bright lighting, so we need to carefully decide whether to prioritize projection or shift toward lighting-based visual expression. He suggested exploring DMX lighting, explaining that even small and affordable fixtures can create layered and atmospheric effects. Moving forward, he encouraged us to enhance our visual presentation, further evaluate the relationship between light and space, and refine our installation sketches. Overall, he believes our direction is right and that with stronger visual strategy and lighting design, the project will become even more compelling.
After the presentation, we discussed the final design and the effects we wanted to achieve in Touch Designer.
There are approximately five effects in Touch Designer:
1. Blizzard
2. Blizzard weakening (controllable by moving both hands up and down)
3. Snow slowly melting into water (can be controlled by extending and lowering your hand)
4. Spring gradually arriving (controllable by opening and closing your hand)
5. Spring's revival (controllable by different hand gestures)
The teacher then explained the equipment usage and spatial planning requirements for the upcoming final assignment. The teacher also provided numerous specific suggestions regarding the narrative and spatial experience of the presentation, drawing on past student work.
1. Exhibition Space and Usage Arrangements
The instructor explained that this exhibition will utilize the white-walled space and tower-like structure within the same area of GFED.
In previous years, the exhibition would last approximately 7–10 days, but due to time constraints, this year it will only be on display for 3 days. During the exhibition, students will take turns guarding the artworks, explaining the concepts, and ensuring equipment safety. This is a crucial part of experiencing the real industry process.
2. Equipment Usage and Laptop Issues
All interactive content requires a laptop for playback. Therefore, each group needs to decide whose laptop will be used for the presentation or borrow one from the school. However, borrowing a laptop requires someone to be present and supervise the equipment throughout the presentation.
Furthermore, the instructor repeatedly emphasized the importance of pre-conference testing to avoid issues such as sensor malfunctions or playback failures.
3. Professional Requirements for the Presentation
The instructor showcased many outstanding works from previous years, emphasizing several points:
Physical elements are essential; it cannot rely solely on the screen.
The work needs to possess narrative, visual logic, and an experiential connection to space.
Simple sketches drawn up on the last night are unacceptable; it must undergo genuine thought, design, and testing.
The instructor repeatedly reminded:
"The final presentation is your first portfolio card in the industry."
Through the teacher's explanation, we gained some new initial ideas. Regarding the end-of-term assignment, we drew a simple sketch and showed it to the teacher.
We proposed a system consisting of three screens, ground reflective materials, and dynamic lighting:
The central screen: Presents more detailed dynamic winter visuals (e.g., snowflake particles, blizzards, etc.).
The side screens: Use more abstract, softer colors and prism/mirror-like refraction to create atmosphere.
Ground reflective materials: Place low-profile reflective surfaces to refract the image into the space, enhancing immersion.
Spatial lighting: Consider using subtle, low-brightness LEDs or DMX lights to create a sense of flowing light on the ground.
The goal is to present a semi-enclosed "winter emptiness" experience, allowing viewers to feel the "absent season" within a calm and gentle visual environment.
Teacher's feedback on our ideas
The teacher shared that our overall direction is promising, especially the idea of using layered visuals, reflective materials, and seasonal emotions, but emphasized that the project needs to move beyond technical sketches and become a fully imagined spatial experience. He encouraged us not to abandon our lighting ideas too quickly, explaining that subtle LED or DMX lighting on the floor could give our installation a more unique and immersive atmosphere, as long as it stays within our technical ability. At the same time, he stressed that our next sketches should not look like technical diagrams but instead clearly communicate the experience and mood we want visitors to feel—something that immediately makes people understand the “winter world” we are trying to create. The teacher also reminded us that a successful installation must include real physical components, not just screens; the reflective surfaces, spatial arrangement, and interactive elements must be designed intentionally and integrated into the narrative. He emphasized the importance of early testing, especially for reflection, lighting, and any sensors, so that we avoid issues like visuals not triggering or lights not working during the exhibition. He asked us to prepare a complete materials and equipment list, think about carefully cables and power, and produce clearer spatial layout diagrams. He hopes that by next week we will have sketches or simple 3D visuals that truly “sell the idea” and allow others to visualize the atmosphere we are aiming for. Overall, his feedback encourages us to refine our concept into something more visually convincing, technically grounded, and experientially strong.
The teacher posted two things on Times, which I will complete by next Friday.
Comments
Post a Comment