Once the scene had a character, curved roads, animation, shadows, HTML overlays, map travel, environment changes, and mobile behaviour, “it sometimes feels janky” became the least useful bug report I could receive. The same visible hitch could come from player movement, a React commit, a texture upload, a shadow update, a camera cinematic, a browser lifecycle event, or the debugging panel itself.

The more capable the village became, the less a screenshot could explain. So I built a control room beside it: tools that make the running world inspectable, give failures a repeatable shape, and publish enough state for a human or an AI agent to investigate without guessing.

A Three.js Canvas Is Opaque by Default

Most of the state that matters in a real-time 3D application does not appear in the DOM. It lives in transforms, animation mixers, collision verdicts, camera state, GPU resources, texture updates, and per-frame hooks. The Canvas shows the final pixels, but it does not explain how those pixels were produced.

That makes a visual check necessary but insufficient. A screenshot can show whether a model overlaps another model, whether the camera composition works, or whether a character appears to float. It cannot prove that a collider redirected the player correctly, that an input command was consumed, that an HTML prompt did not rerender the Canvas, or that an environment switch avoided another GPU upload.

The same limitation matters even more in AI-assisted development. An AI agent can inspect code, DOM nodes, and a screenshot, but it should not have to infer the live state of a Three.js world from pixels or improvise a route with arbitrary key presses. Every additional screenshot, coordinate guess, page scan, and fixed wait is another unreliable loop. Debugging tools are therefore not only developer conveniences; they are the interface an AI needs to reason about the running application honestly.

Scene Lab Turned Hidden State Into Controllable Experiments

The first tool was Scene Lab. It exposes camera composition, model placement, animation clips, playback speed, model bounds, grounding markers, shadows, collision boxes, and environment controls while the scene is running.

It was also a direct scene editor. I could select an entity, drag it across the X/Z ground plane, adjust its transform, rotate it around Y, resize its visual footprint, and override its collider size. Those overrides feed the same collision system as the authored scene configuration, so the editor did not create a separate pretend world. It let me test what actually happens when a building, prop, or interaction moves.

Scene Lab could preview animations on different entity types, not only the player. I could force an animation clip, change playback speed, and inspect whether the visible motion, bounds, and ground contact agreed. The authoring camera could also move beyond the normal visitor-facing follow restrictions, making it possible to inspect occlusion, framing, and collision relationships from angles a visitor would never see.

This changed the kind of question I could ask. Instead of editing code, refreshing, and hoping I had recreated the same conditions, I could isolate one variable: force a walk animation while the player stayed still; show the model bounds and ground marker together; move an object while watching its collision footprint; change a shadow policy while keeping the rest of the scene stable.

For Foxy, this mattered because visible animation, transform position, collision footprint, and ground contact are separate facts. If its feet appeared to float, an image with the animated model, bounds, and grounding marker was much more useful than an ordinary beauty shot. Scene Lab did not fix the bug by itself. It made the disagreement visible enough to form a real hypothesis.

Named Scenarios Replaced Improvised Input

Manual input is a poor benchmark. “Walk around for a while” does not define a route, a starting point, a duration, or a success condition. Two runs may use different paths, miss a key release, begin before assets are ready, or complete less work while appearing faster.

The Test Kit turns common questions into named scenarios: walking routes, collision checks, approaching an interaction, rapid alternating input, idle rendering, passing behind a street lamp, and environment switches. Each scenario has a stable ID, a start policy, a declared workload, an expected outcome, and integrity checks.

The start policy comes from the same scene configuration the application uses. A collision test approaches the actual bookstore or cat rather than copying world coordinates into a separate script. An interaction test derives its location from the real interaction metadata. When the authored world changes, the test follows the world instead of becoming a stale coordinate puzzle.

The workload also enters through the same external input channel consumed by player locomotion. This is important: the test is not a second physics implementation that predicts what the app should do. It drives the real mechanics while removing unreliable keyboard focus and timing from the experiment.

For a rapid-direction scenario, 80 transitions are issued over a fixed frame window. I only interpret frame-time or React results after the scenario confirms that the commands were consumed and the intended physical and presentation transitions occurred. A performance test that silently performs less work is not a passing test.

The Observatory Correlated Runtime State With Render Work

An FPS counter can tell me that a frame was expensive. It cannot tell me whether the player was moving, an overlay opened, a cinematic changed the camera, the environment switched, a texture returned to the GPU, or a shadow path started submitting work.

The Observatory puts those clues next to each other. It records frame-time information, renderer counts, application state, player and travel state, a short event history, and the results of named scenarios. It makes a time-based performance signal useful by placing it beside the state transition and workload that produced it.

That made specific engineering questions testable:

• Does rapid physical input require a React presentation commit for every tiny pause?
• Does opening a nearby HTML action reconcile the persistent 3D composition?
• Does switching a cached environment variant upload a texture again?
• Does a declared collision or route complete without hiding a workload failure?

The answers are scoped to named runs and their saved evidence, not vague claims that the whole site is “optimised.” The control room lets the project preserve what happened, in which environment, under what workload, and what the result can actually support.

AI Agents Need a Public Debugging Surface

AI can be useful in a complex codebase, but it should not become the source of truth about a running 3D scene. The job of the app is to expose the right evidence so the agent can choose the right test and report the result with its limitations.

The Observatory publishes a stable, versioned snapshot that advertises the available scenarios, controls, lifecycle fields, panel status, and typed run result. An agent begins from one bootstrap locator, discovers what the app can currently do, selects an advertised scenario, runs it, waits for a terminal status, and reads the complete result.

That is a much stronger interface than asking an agent to search for selectors, hold an arrow key for an estimated number of milliseconds, and inspect a screenshot. It removes repeated page-wide scans, hard-coded coordinate discovery, arbitrary sleeps, and private access to React or React Three Fiber state. New scenarios are discoverable from the application rather than requiring every browser driver or AI workflow to be taught another custom command.

Readable app state matters just as much. The application publishes whether the scene is ready, which surface is active, whether player control is enabled, which interaction is active, what travel phase is in progress, and whether the player is grounded or airborne. An agent can then distinguish “the player did not move because controls were disabled” from “the player was blocked by a collider,” rather than guessing from one frame.

This is an important design principle for the AI-assisted generation of software: build tools that reduce ambiguity before asking an AI to diagnose it. Stable controls, explicit state, repeatable workloads, and focused diagnostics make the experience easier to debug for a person, a browser runner, and an AI agent at the same time. The agent becomes an investigator of evidence, not a narrator of plausible guesses.

One Failure Report Is Better Than Continuous Polling

An investigator needs detail when a test fails, but streaming every internal value every frame would make both the application and the AI loop worse. The Observatory therefore keeps a bounded recent-event history and offers one diagnostic report containing the player position, current state, and relevant events.

The normal workflow stays compact: run a scenario, wait for its terminal status, read the typed result, and collect diagnostics only when there is something to explain. That reduces both telemetry overhead and reasoning overhead. A short causal record is more useful than an unbounded stream of unrelated values.

This also keeps the evidence hierarchy clear. Images are for visual claims. Source code is for hypotheses. A named scenario is for a stated behavior. Retained measurements are for performance claims. Browser and device runners are for lifecycle and environment behavior. A green badge is never a universal statement that the application is healthy; it means that one declared workload met one declared criterion.

A Browser Observes What the Canvas Cannot

The in-page tools understand the village’s own concepts: its scene entities, player state, collision policy, travel phases, and scenario catalog. Browser-level runners observe different things: focus, visibility, navigation, renderer restarts, process memory, and the behavior of a particular browser engine.

The two layers share a public contract instead of reaching into private state. The application publishes readiness, controls, scenario status, and results through meaningful controls and state attributes. Chrome and Mobile Safari runners drive the same declared workflows while contributing the environment-level evidence the Canvas cannot see.

That separation matters on mobile. A narrow desktop viewport cannot prove that Mobile Safari will preserve memory, survive repeated navigation, or handle visibility changes correctly. The project uses the control room to keep the app’s internal behavior inspectable, then uses browser runners to test the runtime around it.

The Village and the Control Room Are One Project

I began by learning how to make a world people could walk through. I learned how to place models in a Canvas, move through world space, follow a player with a camera, manage assets, and map a 3D village into a readable navigation system.

Building the debug tools deepened that learning. Scene Lab taught me to expose and compare visual state. The Test Kit taught me to define workloads rather than compare impressions. The Observatory taught me to connect renderer cost to application behavior. Making those tools legible to AI taught me that good automation starts with a self-explaining system.

The live village is at 3d.sakshikale.com. The control room is the reason I can keep changing it with more confidence.

The result is a small 3D village built with React, TypeScript, Vite, React Three Fiber, and Three.js. It began as a black block moving along a road and grew into a world with a character, buildings, a map, interiors, day and night, cinematic travel, and tooling for inspecting what the renderer is doing. You can explore the current project at 3d.sakshikale.com.

Each addition forced me to learn a different part of 3D work: how a Canvas differs from a DOM layout, how positions and collision work in world space, how a camera changes the meaning of movement, and how quickly rendering and browser constraints become part of the product.

A Canvas Requires Different Thinking From the DOM

In a conventional portfolio, layout is mostly a document problem. Elements flow, CSS places them, and a link takes a visitor to a section. A Three.js portfolio starts with a different question: where does something exist in a world?

The Canvas does not give me a page layout. It gives me a scene graph, a camera, a renderer, lights, meshes, and coordinates. In the project, the world is described in scene configuration: bounds, ground, roads, buildings, props, player settings, and the default camera offset. React Three Fiber lets me compose those pieces as React components, but the work inside the Canvas is still 3D work. A building needs a position, scale, rotation, collider, material, and relationship to the camera—not just a CSS class.

That was my first major lesson: React made the scene easier to organise, but it did not remove the need to understand Three.js concepts. I had to think about world space, the X/Z ground plane, the Y axis as height, asset loading, the render loop, and the difference between changing interface state and changing a real-time scene.

A Linear Road Made the First Problems Visible

The first playable version was intentionally small: a narrow road, a black rectangular player, three destinations, keyboard input, and a following camera. The road was not visually impressive, but it was a useful test environment because it made spatial mistakes obvious.

Most movement happened along a visible direction. If the player drifted sideways, it looked wrong immediately. If a building trigger fired too early, there were only a few possible causes to inspect. If the camera lagged behind the player, there was no scenery to hide the mismatch. That simple scene taught me more about the relationship between input, movement, camera framing, proximity, and UI entry than a more decorative first version would have.

The interaction model was also deliberately physical. Reaching a building could surface an action; choosing Enter could open its content; closing the content returned the player to the same world state. Movement, arrival, entry, and return are separate product states. Treating them as one vague “go to a section” interaction would have made the system much harder to reason about later.

Player Movement Is a World-Space System

I did not use a physics engine. Instead, I built explicit movement and collision rules for the kind of world this portfolio needs.

The player is treated as a circle on the X/Z plane. Buildings and props are treated as rotated rectangular footprints. Every frame, the locomotion loop consumes the latest normalised input, scales it by movement speed and frame delta, tries the requested move against colliders, clamps the player to scene bounds, settles height against the ground surface, and rotates the model toward the movement direction.

Diagonal movement exposed a practical collision problem. A full diagonal step can hit a building even when one component of that step is still valid. When that happens, the system tries the X-only movement and then the Z-only movement. The result is wall sliding rather than an abrupt stop. This was one of the moments where Three.js stopped feeling like “put models in a Canvas” and started feeling like systems design: a tiny rule about vectors changes how the whole world feels to navigate.

Collision also had to match the visual world without blindly copying it. The scene uses oriented bounds because buildings can rotate. A model’s visible overhang does not have to become an unfair physical wall, and soft decor can remain passable while a small animal can be explicitly blocking. The player is pushed out of overlap with a small epsilon so the same collider does not jitter on the next frame.

Assets Turned Scene Composition Into a Pipeline

Once the road acquired cobblestone, buildings, vegetation, shadows, and a character, the work changed again. Every imported GLB had a scale, origin, material cost, bounding box, visual silhouette, loading cost, and collision implication. A 3D asset is not merely an image that happens to be placed in space.

I built Scene Lab so that camera placement, diorama blur, model transforms, shadow settings, bounds, grounding, and asset placement could be inspected in the running scene rather than through repeated source edits and refreshes. It became a way to learn how composition and rendering parameters interact in Three.js: moving the camera changes what feels important; changing a shadow may improve depth while adding render cost; moving a model may require its collider and ground contact to change too.

The asset pipeline reflects that distinction. Source and experimental models stay separate from runtime assets. Runtime GLBs are checked against size budgets and validated before they are shipped; the optimisation path uses texture caps, geometry simplification, tangent generation, and Meshopt compression. I learned that visual ambition needs an asset policy, especially once the scene has to load and render on a phone.

Foxy Exposed the Difference Between a Transform and a Character

Replacing the black block with Foxy made every shortcut visible. A block only needs a position. Foxy has visible feet, a facing direction, an idle clip, a walk clip, a body scale, and an animation whose apparent movement can disagree with actual velocity.

That meant several values had to agree: the player’s transform, the ground-height function, the model bounds, the collision footprint, the direction of movement, and the selected animation clip. If Foxy’s feet floated while the ground calculation was technically correct, the scene still looked wrong. If the model stopped while its animation continued, the world felt broken even if the position was unchanged.

The movement system separates physical movement from presentation state. Collision and position reporting use the exact per-frame movement result. Animation and footsteps use a semantic activity state with a short 120 ms idle grace period. That avoids React, animation-action, and audio churn during rapid directional input while preserving the real movement mechanics. It was a concrete lesson in keeping frame-loop facts and UI-facing state from becoming the same thing.

A Village Changed Navigation From One Axis to a Topology Problem

A linear road is easy to understand. The visitor can move forward or backward along a largely visible route, and the camera can communicate most of the world from one position. Once I expanded the scene across both the X and Z axes, that stopped being true.

Curved roads, side paths, buildings, trees, props, and interior destinations made the world more interesting, but they also made it harder to infer where anything was. The same X/Z coordinates now had to support walking, camera composition, collision, named destinations, direct travel, and map rendering. A visitor could arrive at a building from several directions, lose sight of a landmark behind a structure, or not know whether a road led to portfolio content or decoration.

The map was not a miniature screenshot of the village. It became a second representation of the same world: the Canvas carries atmosphere, depth, and discovery; the map carries topology. It projects world positions into stable map coordinates and keeps only the routes, destination markers, and labels needed for a decision. That created new problems of its own—label crowding, responsive map layout, direct travel, camera state, and keeping map destinations in sync with the scene’s configuration.

The first map used direct relocation. Later travel made those non-local jumps legible as camera and sky-drop phases rather than pretending that nothing had happened between two distant places. Both the map and the live village can be explored at 3d.sakshikale.com.

React State Cannot Own Every Frame of a Three.js Scene

One of the most useful lessons from this project was learning where React should stop. The app uses React for composition, overlays, content panels, and state transitions; it does not make React responsible for every locomotion update or camera frame.

The expensive Canvas boundary is memoized so that opening an HTML prompt or panel does not reconcile the 3D scene. Input channels—keyboard, touch joystick, and external test input—are merged into revisioned snapshots that the locomotion loop consumes atomically. Cinematic camera travel uses a frame-loop mailbox rather than app-shell React state, so starting a sequence does not re-render the Canvas mid-walk.

The render path has similar boundaries. Static model subtrees can freeze their local matrices instead of recomputing transforms every frame. Shadow maps update on demand when the player moves or lighting changes. A cinematic can temporarily lower device pixel ratio while camera motion hides the trade-off, then restore it after the camera settles. None of these are isolated “performance tricks”; they are decisions about what changes, how often, and where that change should live.

A Beautiful Scene Still Needs Observability

As the village became more detailed, visual inspection stopped being enough. A hitch could come from a React commit, a texture upload, an environment change, a shadow pass, a browser visibility event, or the debugging UI itself. “It feels janky” is not a diagnosis.

That is why I built the Observatory and the scenario-based Test Kit. The project can run declared Walk, Smoke, Loop, collision, prompt-isolation, rapid-input, idle, and environment-switch scenarios. Each scenario has a starting policy, workload, expected result, and a bounded measurement window. It records frame-time percentiles, renderer information, app state, input revisions, physical movement, presentation transitions, and scoped React work.

The point is not to collect metrics for their own sake. It is to connect a rendering event to the state of the application. A rapid-input workload, for example, issues 80 direction transitions over 375 frames and verifies that commands were consumed, physical transitions occurred, and presentation revisions were represented. A faster result is not useful if it silently dropped part of the workload.

The next part of the project was making those debugging tools useful to AI-assisted development as well as to a human investigator: Building an AI-Ready Debugging System for a Three.js Village.

Mobile Made the Canvas a Browser Runtime Problem

Making the village responsive was not a matter of shrinking the Canvas. Phones change input, camera composition, GPU budget, texture and geometry pressure, tab lifecycle, and browser process behaviour.

The project keeps the Canvas mounted across content transitions so that entering and leaving a portfolio section does not rebuild the world. It uses mobile-specific assets and a selective rendering policy to protect memory and frame time without removing the landmarks that make the village recognisable. It also tests repeated navigation and stress workflows in Chrome and native Mobile Safari, because a narrow desktop viewport cannot prove how WebKit will behave over a real session.

This was another lesson that only arrived by building something concrete: a 3D page is not finished because it renders once. It has to retain the right world state, manage GPU work, survive browser lifecycle changes, and remain understandable through touch input and a smaller composition.

What Building the Village Taught Me About Three.js

The village is my way of learning Three.js through constraints that matter. I learned Canvas by having to place a portfolio in world space. I learned movement by making a player, camera, bounds, and colliders agree. I learned scene composition by importing assets that changed scale, shadows, and rendering cost. I learned that navigation in 3D needs its own information architecture, and that performance needs evidence rather than a feeling.

The project is still a portfolio, but the process gave it a more useful purpose: it became a place to learn how a real-time 3D web experience is assembled and maintained. Explore the current version at 3d.sakshikale.com.