ArchTA
Research Assistant, Team Lead, Generative AI Engineer & Researcher
Jun 2025 – Jun 2026
This project plans to train a general multi-modal model for architecture, localize it for HKU, and build AI tools for formative assessment and gamified learning.LLM，AWS，React
How can long-term course archives be transformed into auditable and reusable formative feedback?
How can we model tacit knowledge and “expert taste” in architecture in ways that remain interpretable and controllable?
Under what tasks and conditions can an AI teaching assistant reliably improve teacher orchestration and student iteration?
For advanced undergraduates, which gamification mechanisms are effective in practice?
Can AI-derived progress signals increase the effectiveness and fairness of gamified interventions?
Identify research gaps and design opportunities through cross-domain literature reviews in gamification, AI training, and learning analytics.
Train domain-specific, multimodal expert models by combining public resources with institution-held datasets in a privacy-conscious way.
Build a maintainable, extensible, offline-first architecture that supports easy A/B variant switching (e.g., with/without gamification).
Coordinate cross-lab collaborations, aligning roles, timelines, and shared evaluation protocols.
Run long-term user studies, including participant recruitment, consent, instrumentation, and longitudinal data collection.
The project will first curate existing architectural cases from the web and convert them into multi-turn Q&A tasks for AI training, with the goal of developing a multimodal expert model in architecture. The model will be trained across multiple dimensions—image understanding, integrated reasoning tied to design intent, and expert-level analysis, critique, and reflection on case studies—to cultivate a refined architectural “taste.”Next, we will draw on tens of thousands of student projects collected by the architecture school over the past two decades, select representative samples, and localize the expert model to critique student work. This will help students understand the current level of their projects and identify concrete avenues for improvement.We also plan to introduce gamified design elements (such as points and dashboards) to quantify each student’s positioning and progress along different personalized paths in architecture, thereby encouraging consistent learning. By incorporating RAG (Retrieval-Augmented Generation), the system will surface targeted reference cases to support students’ day-to-day study.Finally, leveraging the expert model’s interpretive capabilities, we aim to evolve from a basic architectural Q&A chatbot into a context-aware teaching assistant that can understand multimodal student work and learning states, and provide guidance accordingly.As the project lead and primary developer, I will curate the architectural training corpus, train and localize the expert model, and co-develop a scalable, maintainable product in collaboration with a senior engineer at IBM. I will also partner with researchers from PolyU and HKU to conduct user studies and experiments. We plan to publish two to three papers spanning education, HCI, and AI.
Curios
Research Assistant, Full-stack developer
Sep 2024 – May 2025
HKU’s metaverse concept-related immersive learning platform.React, Firebase
Implemented instrumentation hooks to capture user interaction data and stored it securely in the cloud.
Gained hands-on R3F (React Three Fiber) development experience, combined with Firebase and database-backed web development.
Curios offers 5 selectable classroom scenes where students can customize their appearance and control a 3D avatar to greet others in a virtual environment. They can also make calls and send private or group messages.Students collaborate on group assignment whiteboards, editing content and adding 3D decorations. Professors, visitors, and students can like and comment on assignments.The project includes flexible controls for smooth facilitation—for example, quick spatial teleportation and convenient instructor tools to modify group setups: swapping group assignment panels, adding members, and adjusting management permissions.A rich set of hooks captures in‑scene interaction data (such as editing durations and dwell time). This data is stored in a cloud database and powers features like in‑platform leaderboards, as well as research use.To ensure a seamless user experience, the codebase is substantial: at least 100 scripts with 500+ lines each, spanning frontend and backend deployment, user authentication and authorization, researcher-facing control pages, end‑to‑end system logic from R3F (React Three Fiber) scene rendering to UI interactions, and cloud functions that interface with the database and use caching to accelerate loading.I was the sole maintainer and developer on the project. This was my first transition from Unity to web development, and I resolved 150+ issues across the stack. Additional contributions include upgrading rendering across all five scenes to a more efficient system, optimizing the existing UI, and implementing group broadcasting and communication features. In a previously sparsely documented codebase, I added hundreds of comments, compiled an index of code entry points for major features, and wrote a concise handover guide to help the next developer continue the work.
Digital Reality Painter
Nvidia Joint Lab STEM Intern, Technical Lead and concept designer
Oct 2024 – Oct 2024
VR painting tool integrating AIGC methods for 3D, 2D, and projection-based art creation.Unity (Oculus), ComfyUI
What is the impact of AI-augmented interaction on interaction fluency and perceived experience in creative design?
How does collaborating with AI reshape reasoning patterns, attention allocation, and exploration behaviors?
How do participants negotiate agency and trust with AI suggestions and scaffolding over time?
Hong Kong International Licensing Show, 2025
PolyVerse - Culture and Art Tech Exhibition 2025
Paper under submission (IEEE VR 2026)
Rapidly developed functional demos and integrated them into the project framework.
Gained development experience on next-gen VR headsets, including hand-gesture interaction and in-headset UI control.
Drew insights from prior VR painting literature to identify research gaps.
Integrated a ComfyUI-Unity generative pipeline into the VR project.
The project integrates AI across three layers of VR painting: 3D spatial strokes generate 3D models, 2D surface strokes generate 2D decals, and lighting strokes drive an AI-powered projector that casts dynamic lighting. All generative features are driven by ComfyUI workflows.The system includes a suite of editing tools, such as grouping selected objects for unified scaling and translation, duplication and deletion, and a magnet-based deformation tool (similar to a “liquify” effect in digital painting).The project supports both hand-tracking gestures and controllers.Development took just one week. A senior from PolyU validated the core logic for 3D strokes and magnet deformation, while I completed the remaining development, including system refinement and all AI workflow implementation.
Beauty of Canton Porcelain
Nvidia Joint Lab STEM Intern, Generative AI Engineer & Unity Developer
Jul 2024 – Jul 2024
The project utilizes Lora to learn intangible cultural heritage patterns and integrates AI into collaborative art creation.Unity, ComfyUI, GPT-API
How can content generated by AI maintain the traditional characteristics and authenticity of Intangible Cultural Heritage (meet the expectations of users)?
How to balance the automation of GenAI with the creative engagement of novices?
Journal of Human Computer Interaction (second author)
Featured on television channels such as Channels News Asia
PolyVerse - Culture and Art Tech Exhibition 2025
Optimized training data and trained LoRA models.
Took over and refactored research project codebases.
Designed and validated a ComfyUI-Unity generation pipeline.
The project began with a senior from PolyU gathering image samples of Guangcai porcelain and validating the basic interaction interface.Based on these samples, I re-cropped and optimized the dataset and trained a LoRA model suitable for creative production.I then refactored the interaction system from the ground up to improve performance and deliver smoother mouse-and-keyboard control, and I developed GPT–ComfyUI workflow integrations. I also added editing features for inserting decals with scaling and rotation.
GeneyMAP
Part-time Research Assistant, Sole developer
Feb 2024 – Jul 2024
Tool for designing GenAI-empowered user journey mapping.Unity, GPT-API
How can GenAI be systematically integrated into journey mapping without disrupting established designer workflows, given that current tools are manual and struggle with large, complex data?
The authors co-designed GeneyMAP with eight GenAI functions and a four-step human-in-the-loop process that automates analysis, mapping, synthesis, and ideation. Studies showed improved efficiency and output quality, but raised concerns about transparency, trust, and control.
CHI 2025 Late-Breaking Work (co-first author)
Built human-AI co-design prototypes using the GPT API.
Implemented rich Unity UI interactions based on Figma design specs.
Followed the full HCI workflow from interviews to development to user studies.
As one of the earliest projects at the PolyU Design School to integrate the GPT-API, I first explored the capability boundaries of GPT-4 for creating User Journey Maps. I proposed a table-based editing workflow and an AI submission/retrieval mechanism, and validated a system where stable JSON structures enable both granular and global table adjustments via AI.Over the next two iterations, working from a colleague’s Figma designs at PolyU, I implemented complex, precise UI interactions in Unity (though a web stack would have been more suitable). Features included AI-suggested or custom table dimensions, collapsible/expandable sections, adding and removing rows and columns with reflow, rapid AI corrections for individual cells, and AI-generated reference imagery for inspiration.
GPT-World
Master’s thesis project, Supervisor: Prof. Henry Duh
Jan 2024 – May 2024
An AI-driven DnD game featuring text-based role-playing, with JSON-based object generation, multi-agent AI integration, quantified game mechanics, and automated data retrieval via LLM.Unity (DIY Framework, FEEL), GPT-API
Does formalizing game assets as quantitative variables significantly reduce logical inconsistencies and uncontrollable behaviors in LLM-based interactive narrative games?
Can a readable–writable JSON asset system reduce inconsistencies in narrative state information ?
Does a modular multi-agent architecture improve performance and playability across different functional modules of the system?
Can the combined approach support long-term, scalable AI-native game operation?
Using asset quantification, JSON read/write archives, and a modular multi-agent setup—versus pure text LLMs—reduces logic errors, state drift, and runaway behavior, and improves maintainability and gameplay depth.
Current evidence is from engineering runs and iterations, with positive qualitative feedback. To prove statistical gains, we still need controlled user studies and offline metrics.
Recognized differences between academic and engineering problem framing and methods.
Gained experience in academic writing.
Explored the limits of GPT-API creativity and stability, enabling flexible AI-integrated development.
Independently iterated through multiple design-and-build cycles for AI games.
This is the project I’m most proud of and passionate about. Players start by describing the world they want to explore and defining their own character. The system then automatically generates class templates needed for the game (for example, Place and Character). Each class comes with well-defined attributes; a Character might have dimensions like Magic, Strength, Wisdom, Values, and Secrets. The system also creates Events and instantiates entities under each class, such as concrete locations and NPCs.Compared to AI Dungeon (aidungeon.com), the gameplay is more structured and engaging. The system proactively introduces challenges, and player choices have quantifiable, systemic effects on entities. For instance, successfully cultivating land increases a location’s Prosperity, rather than just advancing a narrative beat. It’s more than a storytelling simulator—it’s a dynamic, stateful game world.The original goal was to build a fully AI-generated and AI-driven homage to Taiko Risshiden V. Over five iterations, I developed a stable JSON-based content pipeline. The system evolved from directly generating entities to first producing flexible class templates, enabling a modular, extensible architecture. A multi-agent setup now handles different subsystems, progressively increasing flexibility. As the project evolved from GPT-3.5 to GPT-4o, the game became notably more adaptable and fun.
The rhythm of life
Master’s Assignment
Oct 2023 - Nov 2023
An art installation aims to express vibrant vitality through the rhythm of sound and visuals.Visitors can use various methods to rotate or shake the water bottles in their hands, and our device will provide auditory and visual feedback based on their interactions.Ableton & Max, Microcontroller
Cɑs1n0
Bachelor’s thesis project
Oct 2022 – Apr 2023
This project focuses on the four elements of "Interaction," "Metaverse," "Fun," and "AI" and is dedicated to helping players improve their adaptability and observation skills in virtual environments while learning the basics of classical cryptography.Unity (DIY Framework, Inworld AI)
In Metaverse/VR settings where criminals may hide information with scene-based cues and ciphers, how can police adapt to effectively detect and decode encrypted clues and receive targeted training?
Developed a Unity-based VR training/puzzle game system that integrates six classic cryptography challenges, a scene “view highlight” aid, a graffiti-style note-taking Paint System, radial VR keyboards, achievement and autosave systems, AI chat assistance, and AI-generated music, to enhance officers’ observation and decoding skills in virtual environments.
GridSystem
Tech Intern - Unity, Supervisor: Zhang Liming (Tech Director)
Jul 2022 – Sep 2022
This Demo project is my work as an intern in Unity, aiming to verify the feasibility and effect of grid system construction.
By rewriting the plug-in -- Grid Placement System, more complex construction functions are realized, such as withdrawing instructions, archiving, area operations, etc.Unity (Rewrite Plugin)
Mono
Leader & Developer & Artist
Jan 2022 - Jan 2022
In this AR game, white squares in bright light are separated from opposite black shadows. Players need to build blocks in a way that matches the given puzzle projection. Players can switch between black and white environments to observe the existence of blocks or shadows. The game runs through a fairy tale full of philosophical sayings.Unity (Vuforia, DOTween)
Best AR Game, Global Game Jam 2022, Hong Kong
Li QingZhao
Leader & Developer
Oct 2019 - Feb 2020
A VR visual novel game that tells the life of Li Qingzhao, a famous ancient female lyricist. The game includes small games such as answering questions and escaping.Unity (DOTween, Fungus)
Commendation Award, PolyU Virtual Reality Contest