Performative Media : Weekly Learning

 Performative Media

Weekly Learning

WEEK 1:

This was my first experience with performative design. The teacher explained what it is and showed us many video examples. The music video for "Love" particularly impressed me. The use of robotic arms and mirrors made the music video even more engaging and engaging. I think that's the charm of performative design. The teacher then asked us to think of a story, film, artwork, game, or performance that has inspired us. The most popular topic in our group was a shooting game. At the end of the class, the teacher quickly outlined our main tasks for the semester.

WEEK 2:

This week, the teacher reviewed last week's content, including the concepts of performing media and the principles of input-process-output interaction. 

The main courses covered:

1. Creative Coding (using programming for artistic/interactive creative expression) and Generative Art (generating artwork based on rules/algorithms, rather than direct drawing).

2. Key Logic: Art is created through a balance of "rules + randomness." Repeating rules (iteration) can produce complex results (emergence), essentially a human-computer collaboration.

3. Tools: Download TouchDesigner and then practice visual animation.

After the main course, we learned how to use Touch Designer and, under the teacher's guidance and through exploration, completed my first visual animation.

Process:

Final product：

Finally, the teacher explained Assignment 1. We need to find two performing artists and analyze two works of each artist, a total of four.

WEEK 3:

1. Review of Core Concepts

• Generative Art: Relies on rules, algorithms, and processing workflows to generate output via systems. Key elements include randomness/noise and iteration (repeating rules to create new forms), with examples like fractals, random walks, and particle systems.

• TOP vs. CHOP Operators: Clarified their functional and visual differences 

• TOP (Texture Operators): Pixel-level visual processing tools (e.g., noise textures, image composition) with green icons.

• CHOP (Channel Operators): Used for numerical and signal processing (e.g., LFO waveform generation) with bubble-marked icons.

2. Key Tools & Hands-on Practice

• Application of TOP Operators:

• Learned core TOP tools such as Composite, Multiply, and Transform, mastering the logic for layered image compositing, resolution adaptation, and position/size adjustment.
• Hands-on Case: Sequentially composited banana images, text, and circular patterns using Composite; fine-tuned positions (e.g., X/Y axis translation) with Transform; merged black backgrounds with images via Multiply to create a complete visual effect.

• Parameter & Detail Control:
• Adjusted the layer order in Composite (to avoid blocking key content) and the numerical precision in Transform (e.g., 0.01 increment for fine-tuning); resolved image stretching by adapting resolution via "Refit".

WEEK 4:

This week is the week for presentation of Assigment 1. Through my classmates’ sharing, I learned about many artists and artworks that I had never known before, which deeply attracted me. After our sharing, the teacher’s feedback said that we did a good job and the analysis was very thorough, and we really expressed the emotions of the works, but it would be better if we could be more confident.

WEEK 5:

1. Key Operations of the Three Core Operators

• TOPs (Purple): Use operators like Circle and Over for visual generation (e.g., circles, layer overlay), and learned to overlay textures via feedback loops.

• CHOPs (Green): Use Mouse to obtain mouse coordinates, Map to remap value ranges, Lag to smooth values, and Math for numerical operations.

• Control Tools: Use the keyboard (keys 1/2/3) to switch colors, and Logic and Count to realize functions like circle disappearance on click and loop counting.

2. Two Interactive Projects

• Control the circle's position with the mouse and switch its color via the keyboard to achieve input-driven visual changes.

• Extract audio data to make the circle scale with music rhythm and trigger position switching (left/center/right) based on sound.

Class Exercise Field:

WEEK 6:

Today's lesson focused entirely on the advanced practical application of Touch Designer and the core requirements of the second assignment. It included detailed hands-on experiments and clarified the implementation direction of subsequent tasks, following a "practical exploration + assignment breakdown" rhythm.

In the practical session, we delved deeper along the "input-processing-output" logic, exploring more advanced techniques than before. First, the instructor guided us to convert motion into usable data: using the Analysis Top tool to read screen pixels, selecting the "Average" mode resulted in higher values ​​(range 0-81) for more intense motion. However, this range couldn't directly control the noise amplitude. The instructor taught us to use the "Map Top" operator to adjust it to 0.5-20.51. After connecting the lines, the screen noise actually changed with the magnitude of my movements, which was very intuitive.

Next, we added audio responses to enrich the visuals. We opened the Audio Analysis tool, focusing on extracting "low-frequency" and "high-frequency" values, as they are most sensitive to changes in sound. I connected the low-frequency values ​​to the image offset and the high-frequency values ​​to the flash trigger. I also added a "smoothing" module to avoid image clutter. The result was a clear image offset when the sound was loud, and a white light triggered by high-frequency sounds. Although there were some overexposures due to incorrect value adjustments, I found the right parameters after a few tries.

We also experimented with layer overlay techniques: using the "Select Top" and "Threshold Top" tools to extract the image edges, then using the "Composite Top" tool to overlay the original image, and adding a blue constant top layer to enhance the sense of depth. The teacher said, "Don't try to get it right the first time; try and fail." I adjusted the threshold from 0.3 to 0.5, and the edges were indeed clearer.

Class Exercise Field:

Regarding the assignment, the teacher clarified that the core of the second assignment was "to translate the idea into a demonstrable solution," divided into three parts:

1. Concept Presentation: Create a simple slideshow, without piling on details. Clearly state "what interaction you want to create," "the source of inspiration," and "the feeling you want to convey."

2. Supporting Documents: Provide detailed documentation, including audience analysis, a thorough explanation of the inspiration (clearly outlining the points of reference), team division of labor, and, if the inspiration has changed, an explanation of the reasons for the shift.

3. Prototype Creation: Don't strive for perfection; focus on creating the core interaction. Emphasize documenting the experimental process (e.g., tool adjustments, parameter modifications). Trying new tools will be a plus.

Finally, we had a 15-minute group discussion, where each person shared their initial ideas. The instructor reminded us that "the interaction should match the theme; don't make it complicated for the sake of complexity" and suggested completing the prototype within three weeks, with opportunities to contact him anytime with questions.

WEEK 7:

This was the seventh week's TouchDesigner hands-on class. I mainly learned advanced uses of the MediaPipe plugin from the instructor and completed a hand-controlled interactive visual project.

The instructor first reviewed last week's body interaction design principles—interactions should be natural, intuitive, and relevant to the creative theme, avoiding unnecessary complexity. They also mentioned that gamification can make interactions more engaging. Next, they focused on the MediaPipe plugin, which offers features like face, hand, full-body, and object tracking. The instructor reminded us to enable features only as needed to avoid excessive performance consumption.

During the hands-on session, I followed the instructor step-by-step to load the plugin and connect the camera, keeping only the hand-tracking function enabled. We first used the midpoint coordinates of our hands to control the position of a circle on the screen, then adjusted the circle's size using fingertip distance. We added Lag Chop for smoother movement and used Range to adjust the numerical range and Limit to avoid negative values. We then further optimized the design, using Displace Top for image displacement, combining it with Noise Top to create fluid dynamics, adding color overlays, and finally changing the solid circle into a more refined circular outline.

The teacher gave us 30 minutes to practice on our own, trying to use tracking data from our second hand to control other dimensions of vision, such as brightness and color, in preparation for subsequent assignments and the final project. Finally, the teacher mentioned that they would explain the details of the second assignment and the final project later and also provide feedback on our creative thinking.

Class Exercise Field:

The teacher clarified that the exhibition ends in week 13 and the course concludes in week 14, emphasizing the importance of the second assignment and promising to answer questions after class. My team and I shared our idea of ​​"creating an interactive snow scene for Malaysians," and the teacher affirmed the creativity, suggesting adding physical props and deepening the interaction (such as pushing snow pellets with one's body). Regarding the need to detect the distance of the audience, the teacher recommended using a depth sensor and also provided an alternative: using MediaPipe to detect the human skeleton, focusing on achieving the core effects on the software side first.

WEEK 8:

This week’s class felt like a turning point, pulling our project out of the conceptual phase and into something that needs to actually exist. The instructor kept reminding us that the priority now is to make the work executable, functional, and experienceable. He guided us back to the core questions—what the concept truly wants to express and what emotions the audience should feel—emphasizing that interaction shouldn’t be a mechanical trigger but an extension of mood and narrative. A good interactive piece, he said, should communicate through intuition rather than explanation.

We then refined our interaction structure, making every input and output traceable and meaningful, whether it involves movement, light, or spatial change. The idea that interaction should be “intuitive with a touch of surprise” made me realize that discovery matters more than complexity. When discussing space and materials, the instructor stressed that the installation environment is part of the work itself. Even in the prototype stage, simple materials—PVC pipes, fabrics, second-hand items—are enough as long as they clearly express the concept.

The second half of the session focused on planning next week’s prototype. Core interactions must be completed, while visuals can come later; the instructor wants to see something that actually works, not just a conceptual sketch. During his rounds, we uncovered several issues—projection angles, shadow interference, recognition areas—but he reminded us that encountering problems is normal, and finding solutions is what moves the work forward.

The main takeaway this week is that a prototype doesn’t need to be perfect—it needs to function and convey the concept. Moving forward, we’ll refine the interaction logic, adjust the structure, and strengthen the narrative clarity as the project slowly shifts from an idea into a real, experiential piece.

WEEK 9:

Today, we presented the latest prototype of our interactive installation project. The instructor provided detailed feedback on technology, spatial considerations, and next steps.

Technology & Interaction

• The instructor praised our understanding of the interaction flow and touch module logic. The prototype clearly demonstrates core interactions and system responses.
• The project has a solid foundation and can further enhance its visual presentation.

Space & Lighting

• The handling of lighting and materials in the small space was recognized as effective, directly contributing to user immersion.
• We have preliminarily identified potential placement locations for the installation, which is the right direction.

Key Issue: Projection vs. Light Sources

• Projection may be weakened or obscured by surrounding light.
• Suggestions:
• Adjust light placement or reduce ambient light to ensure projection visibility;
• Or use lighting as the main visual medium, shifting the concept toward “lighting choreography.”

Technical Extension: DMX Lighting Control

• DMX allows for rich, controllable, and layered lighting effects.
• Small fixtures are sufficient; high-cost equipment is not required.
• If lighting becomes the primary visual focus, DMX can significantly enhance the atmosphere.

Next Steps

1. Clarify the core visual output method (projection or lighting)
2. Reassess the relationship between space and lighting to ensure visual and interaction harmony
3. Refine installation design sketches for the next review
4. Experiment with DMX lighting on a small scale

Overall Evaluation

The instructor concluded that the project has a solid foundation, feasible interaction logic, and clear creative intent, with thorough spatial considerations. Optimizing visual and lighting strategies will further enhance the installation’s overall impact.

After that, we held a group discussion based on the teacher's feedback.

The teacher then explained how to use the equipment required for the final assignment and highlighted the importance of space planning for the presentation. He also referenced past student projects to give practical advice on presentation narrative and spatial display, helping us understand what to focus on for the final showcase.

1. Exhibition Space and Usage Arrangements

The lecturer explained that this year’s exhibition will take place within the same GFED area, using both the white-wall sections and the tower-like structure.

In previous years, exhibitions typically lasted 7–10 days, but due to time constraints, this year’s display period will be limited to three days. During the exhibition, students will take turns supervising the artworks, explaining their concepts to visitors, and ensuring the safety of the equipment. The lecturer emphasized that this responsibility mimics real industry practice and is therefore an important part of the learning experience.

2. Equipment Requirements and Laptop Management

All interactive components must be run using a laptop. Each group must decide whose laptop will be used for the showcase—or, alternatively, borrow one from the school.

However, if a borrowed laptop is used, someone must remain present throughout the entire presentation to monitor and safeguard the equipment.

The lecturer repeatedly stressed the importance of pre-show testing, reminding us that technical issues such as sensor malfunctions or playback failures must be identified and resolved beforehand.

3. Professional Standards for the Presentation

The lecturer presented several strong examples from past years and highlighted a few key expectations:

• Physical elements are essential—the work cannot rely solely on screens.

• The installation must have narrative clarity, visual logic, and a meaningful spatial experience.

• Last-minute sketches or rushed concepts are unacceptable; the design must reflect real thought, planning, and testing.

He emphasized again and again:

“Your final showcase is effectively the first page of your professional portfolio.”

This set a very clear standard for the level of refinement expected.

4. Our Initial Direction for the Final Project

After hearing the lecturer’s explanation, we began developing new ideas for our final project. We sketched a simple concept and shared it with him for early feedback.

Our proposed setup includes three screens, ground reflection materials, and dynamic lighting:

• Center Screen: Displays detailed winter visuals—snow particles, storm-like motion, and subtle atmospheric shifts.

• Side Screens: Use softer, more abstract colors with prismatic or mirrored refraction to create ambience.

• Reflective Ground Surface: A low-key reflective material is placed on the floor to refract visuals into the surrounding space, enhancing immersion.

• Spatial Lighting: Soft, low-intensity LEDs or DMX lighting create a subtle flow of light across the floor.

The goal is to evoke a semi-enclosed sense of “winter emptiness,” allowing the audience to feel the quiet, gentle atmosphere of a space where the presence of the season is suggested through absence.

Teacher feedback

The teacher acknowledged the potential of our concept, particularly the layered visuals and reflective effects, but emphasized several key points: the installation must include well-defined physical components rather than relying solely on screens; lighting and DMX effects can add unique depth; concept sketches and diagrams should clearly convey atmosphere and narrative; and all technical elements—reflection, sensors, and lighting—must be thoroughly tested before the exhibition. Additionally, next week we need a complete equipment and material list, spatial layout, and visualizations to prepare for fabrication.

WEEK 10:

This week’s class marked a key turning point as we officially entered the final stretch before the public exhibition. The instructor clarified expectations, outlined technical requirements, and walked us through the exhibition timeline. Our group also received specific feedback that will guide our next steps.

01｜From Concept to Execution

We are now past the brainstorming stage. Starting this week, our focus shifts to building an actual, functioning installation. This includes:

• Refining the concept

• Planning interaction and technical logic

• Beginning physical construction and material tests

• Preparing for on-site installation

Each upcoming week must show visible, tangible progress.

02｜Exhibition Timeline Overview

December 12 (Thu) — Preparation

First access to the exhibition space for measurements, layout planning, and equipment placement.

December 13–15 (Fri–Sun) — Exhibition Days

Three days of public exhibition.

Each student will take a 1.5-hour duty shift.

The instructor will be present throughout to assist with technical issues and audience interaction.

December 26 (Thu) — Final Presentation

We will present our concept, technical development, building process, exhibition outcomes, and reflections.

03｜In-Class Task: Writing the Art Description

One major task this week was writing a 3–6 sentence Art Description—the type of text displayed in galleries. It must clearly communicate:

• What the piece is about

• What the audience will experience

• How the interaction works

• The emotional or thematic intention

The instructor will use this description to check whether our concept and visuals align.

We have updated the documentation.

After that, we showed the teacher the current project progress, and we received feedback.

TouchDesigner

Sketch

TouchDesigner Visuals

The instructor was impressed by our particle interaction visuals, describing them as “cool” and “visually strong.”

However, he also mentioned that:

“It doesn’t look like snow yet.”

This means our visuals are not fully aligned with the intended narrative:

blizzard → gentle snowfall → melting → blooming

We need to adjust either:

• the visual style, or

• the conceptual narrative

to ensure the interaction matches the storyline.

Art Description Must Be Very Clear

The Art Description must be extremely concise and precise. After reading it, the instructor should immediately understand:

• the theme

• the emotional progression

• the interaction model

He will compare the description with our visuals to check for alignment.

Feedback on Physical Structure

Our spiral structure and hanging elements were seen as promising.

He encouraged us to continue exploring materials such as:

• CDs

• metal or aluminum wire

• lightweight ornaments

• foam materials

At the same time, we must consider:

• safety

• stability

• weight distribution

• durability during the exhibition

He also requested more detailed structural sketches.

Scent Component (Spring Aroma)

The inclusion of a spring-like scent surprised him in a positive way.

He agreed it could enrich the experience but reminded us to consider:

• the safety of the scent mechanism

• whether the scent should react to interaction

• how close the audience will be to the scent source

Overall Direction

The instructor thinks our combination of structure + interactive visuals + seasonal transformation is strong.

But he emphasized that all components must come together as one cohesive experience.

We then refined the sketch and tried to express it using 3D modeling.

Our group presented the updated version of our interactive installation, and the teacher responded positively overall. He also gave several key suggestions that are crucial for our next development steps. This feedback helped us understand how to move forward more clearly.

Installation Size: Controlled Footprint, Flexible Height

The teacher emphasized two main points regarding the physical dimensions:

• The footprint should ideally stay within 1m × 1m to ensure easy transportation to the GMDB exhibition space.

• The height can be flexible, and building it close to human height is completely fine.

This means we can still achieve an immersive scale as long as the structure remains portable.

Effective Placement of the Interaction Zone

The teacher supported our decision to place the interaction area at the front of the installation. He noted that this layout makes it intuitive for visitors to know where to stand, whether we use projection or a physical screen.

Screens / Projection Need More Dimensionality

Currently, the projection surface appears too flat. The teacher suggested:

• Giving the screen or projection surface more three-dimensional form, or

• Extending the visuals onto parts of the structure.

This would create more depth in the visual experience and integrate the digital content more naturally with the physical installation.

Material Selection Still Needs Clarity

Our materials are not yet finalized. The teacher reminded us to begin considering factors such as:

• Weight and stability

• Transparency or opacity

• Surface texture and reflectivity

• How each material will affect projection and interaction

Material choices impact not only the construction process but also the aesthetics and user experience, making this an important focus for the next phase.

Overall Evaluation: Strong Direction, High Potential

The teacher believes we are moving in the right direction. The visual concept is engaging, and the interaction aligns well with the spatial layout. With further refinement in materials, projection integration, and spatial depth, the installation will become even stronger and more complete.

After that, we further determined the required materials and tools.

WEEK 11:

This week we discussed the materials and ideas we had already purchased with the teacher and updated the sketches.

WEEK 12: