Overview
Voxel world builder AI is an experimental vibe-coded prototype, which explores new creative workflows between AI and interactive design. Users can generate voxel assets and immersive environments using natural language prompts, and seamlessly apply interactions, animations, triggers and systems without code. This project demonstrates how creators with non-technical backgrounds can build highly interactive experience.
*The goal of this experimental project is to build a simple racing game.
Role
Responsibility
UX Design
Prompt Engineering
Team
1 Designer
1 Dev
1 Product Manager
Timeline
Problem statement
Despite the explosive growth of AI,
For interaction and 3D
It remains heavily focused on static asset.
Creating Interaction and system is still complicated.
Building 3D experiences and physical environments still requires a steep learning curve.
For asset interation
Non-devs lose control their intention after generation.
When AI generates 3D, the output is flat:
Due to a lack of structural recognition, it is impossible to refine specific parts.
Product vision
01
From natural
language
For non-devs who have never implemented 3D experiences, the entire process begins with natural language .
02
Semantic structures,
Selective refinement
Auto-detects the form and structure of assets, allowing selectively edit only specific parts. Design it to reflect the intent as closely as possible.
System architecture
Design
Set the theme.
Build an environment.
Prompt to generate an environment. The system automatically identifies hierarchy, and structure and displays them as layers. (ground, track, start line, curbs_0, curbs_1...) You can select each layer to edit specific layer locally. You can also edit the entire environment in addition to local edits.
Text to generate and update environments
Auto-detects the structure and parts of generated environments
Selectively refines only specific parts
Design
Create assets.
Refine it contextually.
Prompt to generate an asset. The system automatically identifies hierarchy, and structure and displays a hierarchical layer. Clicking on a part of the asset automatically selects that part, allowing you to edit it locally without destroying the rest of the asset. Then users can add it to the scene or save it to the library.
Prompt to generate assets
Auto-detect the structure semantically
and divide it into layers
Selectively refine only the parts
that were auto-detected in 3D view
Interaction and system
Add a behavior.
Create a mechanism.
Select the asset and add behaviors to the asset and play type. Users can make it playable, AI bot or leave it as an object.
Then Users can also control properties via the preset options.
Select a car, make it playable
Select another car, make it AI opponent
Write a experience type and game rules,
including win conditions and interfaces in detail
The system recognizes racing game and requires a track
Record a track and play
Reflection
Takeaways.
Having my own experience through this project, the capability to make localized edits to specific parts while preserving the overall structure offers significant value to creators when iterating on AI-generated content.
There are certain UX decisions that can be visible when the prototype is actually in operation.
By implementing vibe-coded prototype, I identified issues about interaction flow, timing, and behavior—problems that were invisible in static screen designs—much more immediately. Real-time prototypes have completely transformed the depth of design decision-making.
The role of a product designer is shifting toward system orchestration—connecting AI models, user intent, interaction logic, and runtime behavior into a single cohesive experience. I realized that in AI-native products, the ability to design the relationships and flows between systems will become even more important than the interface.