Overview

Hello everyone! As I progress through this academic term, I'm thrilled to share the progress on several projects that sit at the intersection of gaming, artificial intelligence, and human-computer interaction. Each initiative represents a piece of a larger puzzle, exploring how we can enhance virtual experiences through innovative applications of AI and thoughtful design principles. Let me take you through this exciting journey of discovery and development.

Healthcare Innovation Through Reinforcement Learning

Working with three colleagues, I'm currently engaged in a fascinating project that applies reinforcement learning (RL)—a type of AI that learns through trial and error—to healthcare decisions. We're building upon Microsoft's research into what they call "medical dead-end states": critical points in patient treatment where certain decisions could lead to irreversible outcomes.

This project will focus initially on sepsis cases, where early intervention is crucial. Think of it like a GPS system that not only shows you where you are but warns you about roads ahead that might lead to dead ends. Our goal is to create a RL model, inspired by the paper’s implementation, that helps doctors identify these high-risk states and make informed treatment decisions.

While this project is not directly related to my gaming research, it strengthens my understanding of reinforcement learning techniques, which may later contribute to optimizing AI decision-making in virtual environments.

Enhancing Usability: The Shift from 2D to 3D in Simulation Games

My journey into my human-computer interaction (HCI) project began with a simple observation while playing Shop Empire 2, a mall management simulation game. The game, while engaging, presents several usability challenges that I believe could be addressed through thoughtful 3D design.

Current Challenges in the 2D Version

The existing game faces several key usability issues that affect player experience:

Interface Clutter

The current user interface (UI) spreads across the screen in what I've found to be a somewhat chaotic manner. This isn't just about aesthetics; it directly impacts how players interact with the game. Since everything exists in a single 2D plane, the interface elements compete for limited screen space. It's like trying to arrange puzzle pieces when there's no way to layer them; everything competes for the same space.

Visual Congestion

As the mall gets busier, Non-Player Character (NPC) sprites begin to overlap. This creates what I call a "visual stack" problem. Imagine trying to watch multiple people through a window where everyone appears to be standing in the exact same spot! Speech bubbles and emotion indicators, which are crucial for understanding customer needs, become particularly problematic as they layer on top of each other.

My 3D Prototype Solution

My 3D prototype aims to address these challenges by utilizing depth and space in more intuitive ways. Think of it as moving from watching a puppet show to observing a real mall; the added dimension creates natural separation and organization.

Measuring Impact Through Careful Testing

To ensure these changes actually improve player experience, I'm developing a comprehensive testing approach:

Within-Subject Testing Sessions

Each player will experience both versions of the game, allowing them to make direct comparisons. Think of it like trying on both an old and new pair of shoes—you can immediately feel the difference.

Think-Aloud Protocol

Players will verbalize their thoughts as they play, giving insights into their decision-making process and frustrations. This is similar to having someone narrate their experience as they navigate a new building.

Performance Metrics

I’ll also try to track various in-game measurements such as time to complete certain tasks and number of clicks to provide and compare performance metrics when playing each version of the game.

Breathing Life into Virtual Characters: Generative Agents using LLMUnity

One of my most exciting ventures this term involves exploring how we can make game characters feel more alive and believable. This project builds upon fascinating research described in the Generative Agents: Interactive Simulacra of Human Behavior paper, which introduces a novel way of thinking about NPC behavior.

A Stepped Approach to Complex AI

Instead of tackling everything at once, I'm structuring the project into phased steps to gradually build complexity. Here's how I'm approaching it:

Integration with Language Models

I’m starting with the integration of large language models (LLMs)—sophisticated AI systems that can understand and generate human-like text. I'm using a library called LLMUnity, which should allow me to integrate these capabilities directly into the game engine.

Short-Term Memory Implementation

The next step is to add a lightweight version of the memory architecture. Think of it as giving NPCs the ability to remember recent events and react to them, similar to how we maintain awareness of our immediate surroundings and recent interactions.

Long-Term Memory Implementation

The final step is to implement the full version of the memory architecture proposed in the paper. This could potentially pave the way for the implementation of memory-enabled NPCs in commercial games, leading to more immersive and dynamic interactive experiences for players.

Tackling Challenges

This project presents several interesting hurdles both technically and ethically that I'm actively working to overcome:

Model Selection and Cost Management

One of the biggest challenges is finding the right balance between capability and cost. The original research used expensive commercial models, but I'm exploring open-source alternatives that could make this technology more accessible.

Ethical impact

One of the main concerns when using LLMs is their potential behavioral impact on humans. The paper highlights some of these issues, such as racial bias and the possibility of individuals forming deep attachments to AI. Although my current project won’t tackle these challenges directly, they remain crucial considerations for anyone working with AI.

Another key concern with generative AI is where to draw the line between AI-generated content, such as dialogue, and the work of a narrative designer. I strongly believe that generative AI should serve to aid and enhance human creativity, not replace it. It will be fascinating to see how these discussions evolve as AI research continues to shape the industry.

Master's Research: Enhancing Background NPC Behavior for Greater Realism

All these projects feed into my master’s research, where I’m working to transform how we think about background NPCs in open-world games. These are the characters that make up the "living" part of our virtual worlds, yet they're often relegated to performing simple, repetitive actions.

The Background NPC Challenge

Traditional approaches to background characters often rely on what can be seen as static behavior patterns:

Fixed Scheduling

Current NPCs typically follow unchanging daily routines. Imagine a shopkeeper who performs exactly the same actions at exactly the same times, day after day. It's efficient from a programming perspective but breaks the illusion of a living world.

Limited Persistence

Many games use what's known as a "spawn-despawn system," where NPCs essentially cease to exist when not in the player's view. While this saves computational resources, it can break immersion if a player decides to track or follow specific characters.

Building a New Framework

My research aims to create a more dynamic system where characters can:

• Generate realistic schedules that adapt to circumstances

• Maintain persistence even when not in the player's view

• React meaningfully to changes in their environment

To accomplish this, I’m looking forward to implementing these systems:

Scheduling and Planning

I'm developing a system that combines a sort of goal-oriented action planning (a way for NPCs to figure out how to achieve their objectives) with the creative capabilities of language models. Think of it like having each NPC maintain their own dynamic to-do list that adapts to circumstances. This will be heavily inspired by the Generative Agents project.

Action Translation System

A crucial component of my framework is what I call the "natural-to-mechanical translation layer." This system will take the high-level behaviors generated by the language model (like "get coffee because I’m tired") and convert them into specific game actions (walking to the coffee machine, operating it, drinking the coffee) that are actually carried out by the NPCs. This will definitely be limited by the number of actions and animations that are present in the simulation or game where I test the framework. However, it could be a great first step.

Technical challenges

These are some of the technical challenges that I’m aware will be present during my master’s research:

Performance Considerations

One of my key focuses is ensuring these more complex behaviors don't overwhelm system resources. It's like choreographing a complex dance. We need all the dancers to move naturally and independently, but we also need to make sure the stage can handle everyone performing at once.

The Translation Challenge

Perhaps the most interesting technical puzzle I'm facing is how to convert the natural language outputs from these AI models into actual game actions. Imagine if someone described what they wanted to do, and we needed to translate that into specific button presses in a game controller. The system that I want to implement has a similar challenge, but at a much more complex scale. My solution involves creating a mapping system where AI-generated outputs are constrained to predefined in-game actions.

Beyond Gaming

While I’m focusing the implementation of my research in gaming, the exciting part of this project is how it could extend to other areas.

Virtual Reality and Training

By focusing on background NPCs, I'm actually addressing a crucial need in virtual reality training environments. Think about medical training simulations or emergency response scenarios. The background characters need to behave realistically to create authentic learning experiences. My research could help create more believable environments for professional training across various fields.

Digital Twins and Simulation

The techniques I'm developing for scheduling and planning NPC actions could be valuable for simulating human behavior in digital twin environments. This could help urban planners understand how people might use new spaces, or help businesses optimize their operations by simulating customer behavior.

Future Virtual Social Spaces

As we move toward more immersive virtual social spaces (aka "the metaverse"), the need for believable background characters becomes crucial. Not every avatar in these spaces will be controlled by a real person, but they all need to contribute to the sense of a living, breathing virtual world.

See you soon!

As these projects progress toward their spring completion dates (except my master’s research), I'm excited about their potential impact on both gaming and broader virtual experiences. Each challenge solved brings us one step closer to creating more believable, engaging virtual worlds. This research journey represents more than technical advancement—it’s about creating meaningful experiences that enhance how we interact with digital worlds. Whether in games, simulations, or virtual environments, these innovations aim to make our digital interactions more natural, engaging, and purposeful. Microsoft’s research team just published a paper titled World and Human Action Models Towards Gameplay Ideation in the journal Nature (you can also check out a more general blog post here). This publication further reinforces my belief that research in this area will only become increasingly impactful for the industry.

I can’t wait to share more about these projects once they’re completed. Meanwhile, stay tuned for my next blog post, where I'll be sharing insights from my reading journey so far in the year. I’d love to hear your thoughts! Which of these projects do you find most interesting? Let’s discuss in the comments!