System Design: Guan Xinyu (Ben)
Key Words:
“Multi-agent”, “Gamification”, “Creative Understanding”, “Role Immersion in Performance”, “Theater Rehearsal”
Venues:
Hong Kong Research Grants Council Theme-based Research Scheme, General Research Fund
Abstract:
Rehearsing a play requires considerable effort and time. Actors often struggle to immerse themselves in the script world and explore their characters without directorial guidance. To provide context for supporting actor rehearsal processes, we worked with a director to design an interactive environment that uses LLM-based agents to represent information in a theater script, enabling actors to experience narrative logic, character perspectives, and social contexts beyond the script. We studied five professional actors using the system during rehearsal and performance. Findings reveal that agent-based rehearsals enable actors to prototype different perspectives in contrast to traditional rehearsals that focus on only their own roles. In reaction to LLM-agents often going off script, actors faced these challenges by creatively exploring the boundaries of the narratives, despite problems of information overload. This work explores the trade-offs between process-based and context-based representation of characters for practical applications in performance.
Details:
In this game-theater project, 11 autonomous agents are designed using generative AI, each representing a character from script Ye Mengxiong Executes the Surrendered. These agents are imbued with distinct personality traits, dialogue styles, and backstories that align with their roles in the original theatrical narrative (see Fig below). The generative AI ensures that each character maintains a coherent and contextually appropriate behavior, enhancing immersion and role adaptation for players.
The Lines as Lens system was designed to transform a static theatrical script into an interactive, agent-driven rehearsal environment. As illustrated in Figure below, the architecture follows three stages: Initialization, Simulation, and Reframe.
Initialization. We collaborated with the play’s director to extract core elements from the original script, including narrative summaries at both global and scene levels, character biographies, relationship maps, and contextual descriptions. These materials were encoded into the prompts that initialize each character agent. The agent design combines static attributes—such as canonical script lines, personality profiles, and initial prompts—with dynamic attributes, which evolve during runtime through user interaction. Dynamic attributes include memory states, evolving inter-character relations, and adjustments to action rules, reflecting how characters may reinterpret their motives in light of new events. To support engagement, the system also provides action hints that scaffold users’ input dialogues.
Simulation. The narrative unfolds as a series of interconnected plots, each comprising multiple scenes populated by relevant agents. Within each scene, agents autonomously execute scripted dialogue sequences (Round A). When the user intervenes, they can disrupt or extend the dialogue flow (Round B), while the system preserves the canonical entry point of the initiating character. Agents’ actions are categorized as interactive (e.g., interrogating another character) or non-interactive (e.g., self-reflection or commentary). These interactions update personal memory traces, reflection logs, and the evolving social network. In addition, a simulated “social media” channel enables characters to generate off-script posts that expose implicit thoughts or perspectives. Posts often trigger responses from related characters, creating indirect exchanges that are stored as social memory and reflected in dynamic relation networks.
Reframe. After each round, the system generates concise summaries that capture the trajectory of the interaction. Dialogue logs, agent posts, and user interventions are compiled into round-level records, which serve both as rehearsal aids and as inputs for generating extended side-narratives. While the main plot remains intact, these additional threads enrich the interpretive space by offering alternative perspectives and speculative storylines. This design ensures that the system continuously reframes the canonical script into a more layered, exploratory rehearsal experience.