← All workOther work · Interaction & design research2025

Co-Evolving Kitchen

Fig. 01Mixed-reality composite of the round kitchen, a visitor and the space responding; it is not a photograph of a built kitchen

Context: WorkshopDigitalFUTURES 2025
Year: 2025
Instructors: Yan ChaoLi Te
Teaching assistants: Yanzi JinJiaxin Yao
Participants: Qizhi DongXi ChenJunyi LuRuiying JiaZishuo LiYiwen FangYuyao JingXiaohan HuXingyi Chen

From responding to anticipating

The core research question of this study is: in the context of increasing integration between digital intelligence and the built environment, how can an intelligent system that continuously senses, learns, and predicts user behavior be integrated with physical space to enable real-time human–environment feedback and behavioral responses? More specifically, how can AI-driven smart spaces, through contextual recognition and prediction of user behavior patterns, influence and support individuals’ everyday activity decisions, thereby fostering mutual adaptation and co-evolution between spatial environments and user behavior?

To explore this question, we designed and implemented a smart countertop prototype called the "Co-Evolving Kitchen." By capturing and learning users' cooking actions, it actively rotates the countertop to anticipate the user's next action, aiming to provide a seamless cooking experience for older adults and individuals with lower limb disabilities.

Original text · Source: Jia, Chen, Dong, Lu, Jing, Li, Hu, Chen, Fang, Yan and Xu, “Co-Evolving Kitchen: A Predictive AI-driven Methodology for Translating Embodied Action into Adaptive Environment”, conference paper manuscript (2025)

Aim: record body movement in real time and build a kitchen fitting that changes in real time to suit what the body is doing.

Translated from the Chinese original · Source: Project statement for the DigitalFUTURES 2025 workshop (Word document)

Four modules

Perception: This module begins the "translation" process by capturing the user's real-time upper-body kinematic data via a ZED 3D depth camera. This data serves as the raw, embodied input for the cybernetic loop.

Prediction: Serving as the system's cognitive core, this module employs K-Means (for feature extraction) and an LSTM network (for time series prediction). Its purpose is to decode the continuous stream of perceived data into a discrete, predictive "next intention."

Execution: This module enacts the system's dynamic spatial response, converting the AI's prediction into a physical, adaptive intervention. It receives signals via GhPython to drive the platform's rotation, reconfiguring the physical space to achieve the accessibility goal of "operating in place."

Feedback: This module is designed to address potential "human-machine trust" and "loss of agency" issues inherent in predictive systems. It employs Augmented Reality (AR) as a key intervention: before physical rotation, the system uses AR to provide "anticipatory feedback." This "visual negotiation" transforms "black box" AI predictions into a transparent human-computer dialogue, ultimately closing the cybernetic loop.

Fig. 02Workflow, functional zoning and dimensions, from the workshop board

Translating movement into data

First, we defined 12 core cooking actions based on four principles: process coverage, posture distinguishability, detectability, and criticality. We then employed hierarchical coding (e.g., 1.0 for "washing vegetables," 1.1 for "finishing washing vegetables") to capture key action states.

Second, we collected skeletal motion data from subjects performing the aforementioned 12 actions in a simulated kitchen environment. We then "translated" the 3D keypoint coordinates captured by the ZED camera into 20 more robust "mapping features".

Third, to objectively transform "tacit knowledge" into "explicit logic," we performed an offline K-Means clustering analysis on the dataset. This analysis utilized a "core feature + auxiliary feature" matching logic, thereby allowing designers to retain a degree of "interpretability" and "interventionability" in the system.

Fig. 053D keypoints the camera can capture

Fig. 07K-means clustering of some core features

Predicting the next step

Building a co-evolving cybernetic system requires modeling its temporal dimension. Cooking, as a sequential activity, is not a series of isolated events but a flow of intentions contingent on history and context.

To train the LSTM, we first addressed the challenge of data scarcity common in prototype-driven research. We constructed a focused dataset of 12 action categories derived from 10 typical Chinese recipes. To maximize the utility of this data, we employed a sliding window method, which yielded 356 input-target sequence pairs. This approach implements a "four-step-history" strategy—predicting the fifth action based on the previous four—a method chosen specifically to enable the model to learn robust behavioral patterns from a necessarily constrained dataset.

Fig. 08Data processing, LSTM prediction and cyclical training

A countertop that turns

The prototype model in the Rhino environment features a rotatable circular countertop comprising four functional zones: a storage area, a washing area, a chopping area, and a cooking area. This is supplemented by a height-adjustable hanging storage cabinet. When the LSTM model predicts the next action (e.g., "chopping vegetables" or "cooking"), the countertop automatically rotates to present the corresponding functional zone directly to the user. This is designed to reduce body displacement and optimize workflow. Similarly, the hanging storage cabinet automatically adjusts its height based on the user's "retrieve-put back" action recognition, improving ease of use and accessibility.

At the real-time control level, when the LSTM model predicts an impending change in user intention (e.g., that action "cutting vegetables (2.0)" will follow "finished washing vegetables (1.1)"), the system logic immediately outputs a Boolean trigger signal. This signal activates the "change-driven module" in Grasshopper, which executes the countertop rotation or storage cabinet adjustment. Grasshopper then synchronizes the updated Mesh data back to the Meta Quest 3 headset. This achieves a consistent augmented reality (AR) presentation, allowing users to instantly perceive the spatial feedback that corresponds to their predicted intention.

In the final presentation, visitors wearing a Meta Quest headset could see this moving kitchen. The design does not intend people to cook in a Meta Quest. Mixed reality was used to show a demo, in the hope of building a real, physical moving kitchen.

Translated from the Chinese original · Source: Project statement for the DigitalFUTURES 2025 workshop (Word document)

Fig. 09The prototype: a rotating countertop with four zones and a hanging cabinet

Fig. 10In AR, the countertop turns to the predicted zone (1/3)

Fig. 11In AR, the countertop turns to the predicted zone (2/3)

Fig. 12In AR, the countertop turns to the predicted zone (3/3)

What the test showed, and its limits

The prototype's validation in a simulated cooking task provides a clear, empirical illustration of the "behavior-space" co-evolution we propose. The experimental results trace a complete cybernetic loop: (1) The subject's completion of "washing vegetables (1.1)" was perceived as the initial input. (2) The system translated this into a predictive "next intention" (AR: "Prediction: Chopping Vegetables"). (3) This prediction triggered a real-time, adaptive spatial response, with the countertop rotating the chopping area (Area C) to the subject. (4) The subject, in turn, seamlessly transitioned to "chopping vegetables (2.0)."

Despite validating the feasibility of the proposed behavior-responsive framework, several limitations underscore the need for more rigorous empirical grounding. First, the study lacks quantitative evaluation, such as model accuracy, task performance comparison, and behavioral efficiency metrics. This limitation makes it difficult to substantiate the system’s advantages over non-predictive environments. Second, the training dataset was limited both in size and in behavioral diversity, as it was derived from a small number of repeated demonstrations. Such a narrow behavioral spectrum may reduce the model’s ability to generalize and increases the risk of overfitting. Third, the absence of data from older adults or other user groups restricts the framework’s applicability to population-level variability. Finally, the controlled laboratory setting and the sensing constraints of the ZED camera limit ecological validity.

Next · Landscape & urban designFloating Terrain, Framed Axis