3D Brush Techniques and In-Game World Sculpting

We want players to sculpt the world. Not place prefabs on a grid. Not toggle blocks on and off. Actually reshape the terrain: carve rivers, raise mountains, smooth cliffsides, dig caves. The kind of thing that ZBrush and Blender's sculpt mode do for artists, but running at 60fps inside a multiplayer browser game.

That's a hard engineering problem that spans data representation, mesh extraction, GPU compute, brush mathematics, and network synchronization. This guide documents everything we found.

The Two Worlds of Terrain Representation

Every sculpting system starts with a choice about how terrain data is stored. That choice determines what kinds of edits are possible, how fast they run, and how much memory they cost.

Heightmaps

A heightmap stores one height value per grid point. You can think of it as a grayscale image where brightness equals elevation. Our current world/client terrain works exactly this way: noise.ts generates height via FBM value noise, and each Chunk stores a Float32Array heightmap that gets projected onto a PlaneGeometry.

Heightmaps are fast. Sampling is a single array lookup with bilinear interpolation. LOD is trivial because you just reduce the grid resolution. Splat-based texture blending maps directly onto the UV grid. Physics collisions reduce to a height query.

The limitation is topology. A heightmap can only represent one height per (x, z) coordinate. No caves. No overhangs. No arches. No tunnels. If a player sculpts a cliff that folds back on itself, a heightmap can't store it. For terrain that's mostly rolling hills and mountains, this is fine. For freeform sculpting where players can dig into the ground, it's a dead end.

Volumetric (3D Scalar Fields)

The alternative is storing a value at every point in 3D space. If the value is negative inside solid material and positive outside (or vice versa), you have a Signed Distance Field (SDF). If the value is just a density (above some threshold is solid, below is empty), you have a density field.

Volumetric representations handle any topology. Caves, overhangs, floating islands, tunnels through mountains. The tradeoff is memory and complexity. A 256^3 grid with 32-bit floats costs 64 MB. A 512^3 grid costs 512 MB. And that's for a single chunk. You need sparse data structures (octrees, brick maps) to make this practical.

The mesh extraction step is also nontrivial. You can't just set vertex Y positions and be done. You need an algorithm that reads the scalar field and produces a triangle mesh that approximates the surface where the field crosses zero.

Mesh Extraction: Marching Cubes, Surface Nets, and Dual Contouring

Marching Cubes

Marching Cubes is the oldest and most widely implemented isosurface extraction algorithm. Published by Lorensen and Cline in 1987, it works by examining every cube in the voxel grid where each corner has a scalar value. If some corners are inside the surface (negative) and some are outside (positive), a triangle mesh patch gets placed inside that cube.

Each cube has 8 corners, each either inside or outside, producing 256 possible configurations (2^8). These reduce to 15 unique cases via symmetry. A lookup table maps each case to a set of triangles. Edge intersection points are found by linear interpolation along the edges where the sign changes.

Recent GPU implementations have made Marching Cubes fast enough for real-time sculpting. A 2025 implementation on UE5 assigns each GPU thread to one cube, processing thousands simultaneously. The key insight is that each cube's triangulation is independent of its neighbors, making the algorithm embarrassingly parallel.

MCHex (arxiv 2511.02064, 2025) extends Marching Cubes to adaptive hexahedral mesh generation with guaranteed positive Jacobian values, improving boundary approximation for simulation meshes.

rupMC achieves dozens of times faster performance than serial implementations and 4x faster than parallel DMC variants using CPU/GPU heterogeneous architectures.

The main limitation: Marching Cubes struggles with sharp features. A 90-degree edge gets rounded into a smooth curve. For terrain sculpting this is usually acceptable (natural terrain is mostly smooth), but for architectural features it's a problem.

Surface Nets

Surface Nets is a newer family of algorithms that produces smoother meshes from discrete scalar fields. Instead of placing vertices on cube edges (like Marching Cubes), Surface Nets places one vertex per cube that contains the surface, then connects adjacent vertices to form quads.

The result is naturally smoother. A 2024 paper (arxiv 2401.14906) demonstrated a high-performance parallel Surface Nets implementation that runs one to two orders of magnitude faster than sequential algorithms. The fast-surface-nets Rust crate generates roughly 20 million triangles per second on a single 2.5 GHz core using small lookup tables and SIMD acceleration.

bevy-sculpter (v0.18.0, January 2026) uses Surface Nets as its primary meshing strategy. The crate provides SDF-based volumetric sculpting with four brush types: hard CSG (instant add/remove), smooth continuous (for held input), blur (surface smoothing), and flatten (set to target height). It also includes SDF redistancing via the Fast Sweeping Method to restore proper signed distance field properties after edits.

Surface Nets are a good middle ground between Marching Cubes (simple, fast, but produces aliased meshes on binary data) and Dual Contouring (feature-preserving but complex).

Dual Contouring

Dual Contouring preserves sharp features that Marching Cubes and Surface Nets can't. It does this by using not just the sign of the field at each corner, but also the gradient (normal) at edge intersections. A QEF (Quadratic Error Function) minimization places the vertex inside each cell at the position that best satisfies all the edge intersection constraints.

The result: sharp edges and corners are preserved in the extracted mesh. A cube with perpendicular faces stays a cube.

The tradeoff is complexity. The QEF solve has inter-cell dependencies that make GPU parallelization hard. It can produce non-manifold meshes (edges shared by more than two polygons). And the implementation is more involved than Marching Cubes, though Johannes Jendersie notes that a working Dual Contouring implementation is about 200 lines of code versus 500+ for a robust Marching Cubes.

Cubical Marching Squares (CMS) has been proposed as a middle ground: intercell-independent (GPU-friendly) while still offering some feature preservation.

Which to Use

For player-facing terrain sculpting in a browser:

Surface Nets is the strongest candidate. It produces smooth meshes (natural-looking terrain) without the aliasing artifacts of Marching Cubes on binary data. It's fast enough for real-time re-meshing. And it's simpler to implement than Dual Contouring.

Marching Cubes remains a solid choice when GPU parallelism is the priority (every cube is independent) or when you need the widest library support. The WebGPU SDF Editor by Reinder Nijhoff (January 2026) implements both Marching Cubes and Surface Nets in its extraction pipeline, running entirely on the GPU.

Dual Contouring is best reserved for cases where architectural precision matters more than performance. Not ideal for real-time terrain sculpting in a browser context.

The Transvoxel Algorithm: Solving LOD Stitching

When voxel terrain is meshed at different resolutions (LOD0 near the player, LOD2 far away), cracks form at the boundaries. For heightmaps this is a simple problem: interpolate edge vertices to match the lower-resolution neighbor. Our current chunk.ts does exactly this in stitchEdge().

For volumetric terrain, the problem is much harder. A cave mouth at LOD0 might produce 30 triangles on the boundary. The same region at LOD1 might produce 8 triangles with a completely different topology. There's no simple way to linearly interpolate between them.

Eric Lengyel's Transvoxel Algorithm (2009) solves this with "transition cells." At the boundary between two LOD levels, the algorithm considers 9 high-resolution samples (instead of 8 cube corners), producing 512 possible configurations that fall into 73 equivalence classes. Each class maps to a predefined triangle pattern that perfectly fills the gap between the two resolutions.

The algorithm operates on local voxel data, so retriangulating a modified region is fast. This is critical for real-time sculpting: when a player edits terrain near an LOD boundary, only the transition cells need to be rebuilt.

A Rust implementation exists as the transvoxel crate. The original lookup tables are available at transvoxel.org.

Brush Mathematics

A sculpting brush is a function that modifies scalar field values within a radius around a target point. The mathematics are surprisingly similar across all implementations, from Blender to Unreal Engine to runtime game systems.

Falloff Functions

The brush falloff determines how the edit strength decreases from the center to the edge. Blender 5.1 defines these standard profiles:

Smooth: f(d) = 3d^2 - 2d^3 (Hermite interpolation, same smoothstep as our noise.ts)

Sphere: Strong at center with steep falloff near border. Approximated as f(d) = sqrt(1 - d^2).

Sharp: f(d) = (1 - d)^n with n > 2. Creates a fine point.

Linear: f(d) = 1 - d where d is normalized distance from center (0 at center, 1 at edge).

Constant: f(d) = 1 for d < 1, hard cutoff at the brush boundary.

Inverse Square: Hybrid between smooth and sphere for a natural "clay-like" feel.

In all cases, d = distance_to_center / brush_radius, clamped to [0, 1]. The falloff value multiplies the brush strength to produce the actual field modification at each point.

Falloff Space

Blender distinguishes between sphere falloff (distance computed in 3D world space) and projected falloff (distance computed in 2D screen space). Projected falloff means that two points that appear close on screen affect each other equally, even if they're at very different depths in world space. For terrain sculpting, 3D world space falloff is usually more intuitive.

Core Brush Operations

Raise/Lower (Displacement): Add or subtract from the scalar field within the brush radius, weighted by falloff. For heightmaps: height[i] += strength * falloff(d). For SDFs: sdf[i] -= strength * falloff(d) (subtracting makes material more solid, raising the surface).

Smooth (Laplacian): Replace each value with the average of its neighbors, weighted by falloff. This erases detail and reduces noise. The Laplacian filter samples a small kernel (3x3 for heightmaps, 3x3x3 for volumes) and blends toward the mean. HC (Humphrey's Classes) smoothing is a variant that preserves volume better than raw Laplacian.

Flatten: Set the field value to a target height (or distance in SDF space), blended by falloff. The target is usually sampled at the brush center when the stroke begins, then held constant. This creates flat plateaus.

Pinch/Inflate: Move vertices toward or away from the surface normal. In SDF space, this is equivalent to displacing along the gradient direction.

Grab: Translate a region of the field, as if you were pulling clay. The displacement vector is the mouse delta projected into world space, applied to field values within the radius.

Noise: Add procedural noise to the field within the brush radius. Useful for roughening smooth surfaces.

Stamp: Apply a 2D grayscale image as displacement, projecting it onto the surface under the cursor. Unreal Engine's Landscape tool supports this for terrain brushes.

Adaptive Tessellation

sculpt-3D (React + Three.js browser sculpting) implements adaptive tessellation: as the brush moves across the mesh, triangles near the brush center are subdivided to provide more vertices for deformation. This prevents the "low-poly stretch" problem where a coarse mesh gets distorted by sculpting. The subdivision uses symmetric splitting for uniform triangle quality.

For volumetric systems, adaptive tessellation isn't needed in the same way because the mesh is regenerated from the field. Instead, you can locally increase the voxel resolution near edits (adaptive octrees) for the same effect.

SDF Sculpting: The Dreams Approach

Media Molecule's Dreams (PS4, 2020) is the most ambitious in-game sculpting system ever shipped. Alex Evans presented the technical approach at SIGGRAPH 2015.

Representation

Dreams stores geometry as a compound SDF function in 83^3 fp16 volume texture blocks. Each sculpt is a list of 1 to 100,000 "edits," where each edit is a CSG operation (add, subtract, color) with a primitive shape (sphere, cube, cylinder, cone, ellipsoid, torus, etc.) and a blend mode.

Blend modes use soft-max and soft-min functions. A "soft" blend produces rounded transitions between primitives (like clay pushed together). A "hard" blend produces crisp boolean cuts. The blend radius is user-controllable.

Rendering

Dreams doesn't extract a triangle mesh. Instead, it renders directly from the SDF using a custom point-cloud renderer ("flecks"). Each fleck is a tiny disc oriented along the surface normal. The SDF is sampled to find surfaces, and flecks are distributed across them. This avoids the mesh extraction bottleneck entirely but requires a custom renderer.

For a Three.js/WebGL world, this approach isn't directly applicable. We'd need to extract meshes. But the CSG edit list concept is highly relevant for undo/redo and network synchronization.

Mike Turitzin's Dynamic SDF Engine (2026)

A game engine currently in development by Mike Turitzin uses dynamic SDFs as the core representation. The engine supports:

Detailed modifications during gameplay: Adding and removing matter smoothly or with sharp edges. Non-destructive changes like moving holes or creating temporary tunnels that disappear behind the player.

Brick maps and brick atlases for sparse caching. Instead of storing the entire SDF field in a dense 3D grid, the field is divided into "bricks" (small 3D tiles). Only bricks that contain the surface boundary are allocated. This dramatically reduces memory for scenes with mostly empty or solid space.

Geometry clipmaps (Losasso & Hoppe, SIGGRAPH 2004) for LOD. Nested regular grids at increasing resolution surround the camera position. The innermost grid has the finest resolution; outer grids are progressively coarser. This provides dramatic memory reduction while supporting vast spaces. Clipmaps update incrementally as the camera moves, making them efficient for streaming open worlds.

Physics and collision work directly against the SDF. Sphere-tracing (ray marching with the SDF distance as the step size) provides efficient raycasting. Collision detection uses the SDF gradient as the surface normal and the distance value as the penetration depth.

Teardown: Voxel Destruction at Scale

Teardown (Voxagon) represents the other end of the spectrum: every object in the world is a voxel volume that can be destroyed piece by piece.

Architecture

Objects are stored as voxel grids on regular spacing. The engine doesn't use Marching Cubes or SDFs for rendering. Instead, it ray-traces voxels directly using a modified DDA (Digital Differential Analyzer) algorithm in fragment shaders, built on OpenGL 3.3. Mipmaps form a dense octree structure for accelerating empty space traversal during ray intersection.

For each object, the engine rasterizes its oriented bounding box (OBB) and traces a ray through it to find voxel intersections. Only backfaces of the OBB are rendered, allowing the camera to clip into the bounding volume.

Destruction Synchronization (Multiplayer)

Teardown's March 2026 multiplayer update uses a semi-deterministic approach. Structural destruction (cutting holes, changing ownership, reconnecting joints) is handled via fixed-point integer math on a reliable network stream. All clients execute the same deterministic commands and arrive at the same world state. Non-structural changes (debris, particles) use unreliable state synchronization.

This is an important insight for our multiplayer world: terrain edits must be deterministic. If Player A sculpts a mountain, all clients must produce the same mesh from the same field data. The edit commands (brush position, radius, strength, operation type) should be the authoritative data, not the resulting mesh.

ALICE-SDF: Compression and CSG Trees

ALICE-SDF (Adaptive Lightweight Implicit Compression Engine, v1.3.0 March 2026) provides a Rust implementation of SDF-based spatial data with 10-1000x compression versus polygon meshes. It supports:

126 building blocks: 72 primitives, 24 operations, 7 transforms, and 23 modifiers. Smooth blending operations (union, subtraction, intersection), chamfer and stairs blends for hard-edge bevels and stepped CSG transitions.

CSG tree diff/patch for undo/redo and network synchronization. This is the key feature for multiplayer sculpting: instead of sending the entire field state, you send the structural diff between two CSG trees. The client applies the patch to reconstruct the new state. This is far more bandwidth-efficient than delta-compressing raw voxel data.

CSG tree optimization including identity transform removal, nested transform merging, and modifier demotion. This keeps the tree compact as edits accumulate.

Mesh generation via both Marching Cubes and Dual Contouring. Physics collision detection works directly against the SDF.

WebAssembly support enables browser integration. The engine is written in Rust with WASM bindings, making it a realistic option for a Three.js application.

WebGPU Compute for Terrain

WebGPU is now available across all major browsers as of late 2025. Chrome 113+, Edge 113+, Firefox 141+, and Safari 26+ all ship with it enabled. This opens up compute shader pipelines that were previously GPU-only.

Performance

WebGPU compute shaders generate terrain roughly 100x faster than CPU methods through massive parallelization. The GPU executes thousands of calculations simultaneously, and terrain generation is almost entirely parallel (each vertex/voxel is independent).

Work is organized into three tiers: Dispatch Level (workload distribution across the GPU), Workgroup Level (shared memory within a processing unit), and Thread Level (individual calculations). WGSL (WebGPU Shading Language) is the shader language.

Real-Time Terrain Sculpting Pipeline

A WebGPU sculpting pipeline would look like this:

1. Brush application (compute shader): Update the scalar field values within the brush radius. Each thread handles one voxel. Reads the brush parameters (position, radius, strength, falloff type, operation) from a uniform buffer and applies the modification.

2. Mesh extraction (compute shader): Run Surface Nets or Marching Cubes on the modified region. The WebGPU SDF Editor by Nijhoff implements this as a multi-stage pipeline: space partitioning across 16,384 cells, octree-based cell splitting, then surface extraction.

3. Vertex buffer update (GPU-side): Write the extracted vertices directly into a render buffer without roundtripping through CPU memory.

4. Normal computation (compute shader): Calculate vertex normals from the mesh or from the SDF gradient.

5. Render (standard pipeline): Draw the mesh with standard PBR materials.

Steps 1-4 can all run on the GPU without any data returning to JavaScript. The CPU only needs to send the brush parameters each frame.

The WebGPU SDF Editor

Reinder Nijhoff's WebGPU SDF Editor (January 2026) demonstrates this approach running in Chrome. It supports six primitives (cone, cylinder, capsule, torus, box, sphere), three blend operations (union, subtraction, intersection) with configurable smooth blending, and hierarchical scene graphs. Each primitive occupies 112 bytes in a single GPU buffer.

The rendering pipeline uses 1,024 shadow maps for ambient occlusion and temporal anti-aliasing. This runs at interactive framerates on high-end GPUs.

Heightmap Sculpting: The Simpler Path

If cave and overhang support isn't required, heightmap sculpting avoids the entire volumetric pipeline. This is how most shipped games handle terrain editing.

Runtime Implementation Pattern

Unity Runtime Terrain (JohannHotzel, January 2026) demonstrates the standard pattern:

1. Raycast from the camera through the mouse position to find the terrain hit point.
2. Map the hit point to heightmap coordinates.
3. Apply the brush to nearby heightmap values, weighted by falloff.
4. Update the mesh by setting vertex Y positions from the modified heightmap.
5. Rebuild the physics collider to match the new mesh.

For our Three.js terrain, steps 1-4 map directly onto the existing architecture. The Chunk class already stores heightmaps and builds meshes from them. Adding sculpting would mean:

• A raycasting system against chunk meshes (Three.js Raycaster)
• Brush application functions that modify chunk.heightmap values
• Mesh vertex updates (set Y positions, recalculate normals)
• Splat map recalculation for the affected region (so texture blending reflects the new slope/height)
• Network broadcast of the edit (brush position, radius, strength, operation) to other clients via the existing WebSocket protocol

The Clipmap Approach

Landow.dev describes a "wandering clipmap" for heightmap terrain: a single mesh with variable subdivision density that follows the player. Instead of chunking the heightmap into separate meshes with different LOD levels (what we do now), the clipmap is a continuous mesh that's dense near the camera and coarse at the edges.

This eliminates LOD stitching entirely. The mesh simply has more triangles where you need them and fewer where you don't. The tradeoff is that sculpting requires updating a single large mesh rather than individual chunks, which can be expensive for large edits.

Non-Destructive SDF Heightmaps

Landow.dev also describes a technique where the heightmap itself is generated from an SDF composition. Shape instances (spheres, boxes, noise functions) are composed in a compute shader using CSG operations, and the output is sampled as a heightmap. This provides non-destructive editing (you can move or delete any shape instance at any time) while keeping the simplicity of heightmap rendering.

This is a compelling hybrid: the data representation is volumetric (SDF CSG tree), but the rendering path is a standard heightmap mesh. You get undo/redo and network-friendly edit operations from the SDF side, and simple rendering and physics from the heightmap side. The limitation remains: no caves or overhangs.

Sparse Voxel Octrees for Large Worlds

Dense 3D grids don't scale. A world that's 1 km on each side at 0.5m resolution would need 8 billion voxels. Sparse Voxel Octrees (SVOs) solve this by recursively subdividing space and only allocating storage for octants that contain the surface boundary.

An SVO naturally provides hierarchical LOD: the tree depth at any point determines the effective resolution. Near the player, the tree is fully expanded (maximum detail). Far away, it's truncated at a coarser level.

For rendering, SVOs can be ray-traced directly (no mesh extraction needed). A GPU ray marcher intersects rays with axis-aligned boxes at each tree level, skipping empty subtrees entirely. This eliminates overdraw problems of chunk-based rendering and avoids greedy meshing artifacts.

The Vulkan-based SVO builder by AdamYuan demonstrates significant performance: 19ms build time for Crytek Sponza at 2^10 resolution on a GTX 1660 Ti.

For sculpting, SVO modification is efficient: only the leaf nodes within the brush radius need to be updated, and the tree structure handles varying resolution naturally. Adding detail where the player sculpts (splitting nodes to higher resolution) and removing detail where they smooth (merging nodes to lower resolution) falls out of the data structure.

The challenge for browser deployment is that WebGL doesn't support compute shaders, and while WebGPU does, the SVO construction and traversal algorithms are complex to implement in WGSL.

Network Synchronization for Multiplayer Sculpting

Our world already has multiplayer via Cloudflare Durable Objects (world-chunk-do.ts). Adding sculpting means synchronizing terrain modifications across all connected clients.

Delta Compression

Sending raw voxel data is expensive. A 2024 study from Oulu University achieved 2-8x payload improvement by combining delta-encoding with DEFLATE compression, packing voxel updates to less than one byte per voxel. The SDEC codec demonstrates bit-packed delta encoding producing 259-byte average packets versus 1114 bytes with generic serialization.

Operation-Based Sync (Recommended)

Instead of synchronizing field state, synchronize operations. Each sculpting action becomes a message:

interface TerrainEditMsg {
  t: MsgType.TerrainEdit
  brush: {
    position: [number, number, number]
    radius: number
    strength: number
    falloff: 'smooth' | 'linear' | 'sharp' | 'constant'
    operation: 'raise' | 'lower' | 'smooth' | 'flatten' | 'noise'
    targetHeight?: number
  }
}

The server broadcasts this to all clients, and each client applies the same deterministic brush operation to its local terrain data. This is the same approach Teardown uses for structural destruction: deterministic commands on a reliable stream.

ALICE-SDF's CSG tree diff/patch takes this further: instead of individual brush strokes, the diff represents the structural change to the entire CSG tree. This enables efficient undo/redo across the network (send the inverse patch) and late-joining clients can reconstruct the full world state by replaying the operation log.

Priority and Throttling

Terrain edits near connected players should be high-priority (immediate broadcast). Edits far from any player can be batched and sent at lower frequency. Enshrouded's voxel networking uses this pattern: 60Hz updates for player-proximate terrain, 10Hz for background regions.

ZSTD compression on the wire reduces packet sizes by up to 60% for terrain update messages.

Collaborative Sculpting: Concurrent Edits

When multiple players sculpt the same region simultaneously, you need conflict resolution. cSculpt (CNR Visual Computing Lab, 2016) solved this with a multiresolution merge algorithm. Each edit is represented at multiple scales, and concurrent overlapping edits are merged by blending their multiresolution representations.

For our purposes, a simpler approach works: last-write-wins with server ordering. The Durable Object timestamps each edit and broadcasts them in order. All clients apply edits in the same sequence. Since brush strokes are small, localized, and additive/subtractive, the visual result of slightly reordered concurrent edits is usually indistinguishable from the "correct" order.

INST-Sculpt: Neural SDF Editing (Research Frontier)

INST-Sculpt (arxiv 2502.02891, February 2025) enables stroke-based editing of neural SDFs. Users draw strokes on the surface, and the system deforms the underlying neural field along tubular neighborhoods around the stroke path. Custom brush profiles (configurable cross-sections) control the shape of the deformation.

This is interesting for AI-generated terrain: if the base world is represented as a neural SDF (a small neural network that maps 3D coordinates to signed distance), sculpting modifies the network weights rather than explicit voxel data. The representation is extremely compact (a few MB for an entire world) but evaluation is more expensive than a lookup table.

This is research-stage technology. The inference cost of neural SDFs on consumer hardware is too high for real-time game use today. But it's worth watching, especially as WebGPU shader capabilities improve and model inference gets faster.

World Creator 2026.3: Commercial Terrain State of the Art

World Creator (BiteTheBytes, March 2026) represents the commercial state of the art for terrain authoring tools. Version 2026.3 added GPU-based terrain generation with automatic terrain adaptation (terrain conforms to placed objects), camera-focused object distribution for LOD optimization, and real-world elevation data import (GeoTIFF, HGT, DTED).

Their approach uses GPU compute for all terrain operations: erosion simulation, river carving, texture painting. The brush tools are GPU-accelerated with real-time viewport feedback. This matches the WebGPU compute pipeline described above, running on desktop GPUs.

What We've Already Built: 24 Spikes and a Production World

The world/spikes/ directory contains 24 self-contained prototypes. These aren't toy demos. They're a progressive R&D pipeline where each spike solved a specific problem, benchmarked it against a target, and informed the next one. The sculpting system builds on all of them, not just the later volumetric ones.

The Production Heightmap Terrain (world/client/)

The live world uses a chunked heightmap system in Three.js WebGL:

• noise.ts generates terrain height via FBM value noise (5 octaves for hills, 4 for ridges, 3 for micro-detail) with a deterministic terrainHeight(wx, wz) function
• chunk.ts builds PlaneGeometry meshes from Float32Array heightmaps with 3 LOD levels (32/8/4 segments per 64-unit chunk), and places instanced trees/billboards with seeded random placement and per-object colliders
• chunk-manager.ts streams chunks in rings around the player (radius 1 at LOD0, radius 3 at LOD1, radius 6 at LOD2) with edge stitching via linear interpolation in stitchEdge(), and provides getHeight(), getNormal(), and resolveCollisions() for the physics layer
• terrain-material.ts does 4-layer splat-based texture blending (grass/rock/sand/dirt) via MeshStandardMaterial.onBeforeCompile with slope and height-driven weights, plus per-layer normal map blending
• character-controller.ts samples terrain height every frame for gravity, grounding, and slope rejection (max slope cos 50 degrees). Sculpting must feed modified heights into this system instantly or the player falls through edited terrain
• placement.ts already has a Raycaster hitting chunk meshes for the object placement tool. The brush tool should follow this exact pattern rather than building raycasting from scratch
• protocol.ts defines MessagePack-encoded messages for multiplayer sync via world-chunk-do.ts Durable Object, which currently handles PlayerState, PlaceObject, RemoveObject, and Snapshot messages. Terrain edits will need a new message type
• world-chunk-do.ts (Cloudflare Worker) persists placed objects to Durable Object storage and broadcasts to connected players at 50ms intervals. It has no concept of terrain modifications yet

Spikes 01-11: The Foundation Layer

These spikes validated the core systems that sculpting will depend on. Skipping them means missing constraints the sculpting system must respect.

Spike 01 (Terrain + Instancing): The first terrain prototype in Three.js. Established the PlaneGeometry + heightmap pattern and instanced object placement that chunk.ts still uses.

Spike 02 (Rapier Physics Worker): Rapier 3D running in a Web Worker with a ColliderDesc.heightfield() collider. Built a kinematic character controller with autostep, slope limits, and snap-to-ground. This spike proved that physics can run off the main thread against a heightfield. If we sculpt the terrain, the physics heightfield must be rebuilt or replaced with a trimesh collider for MC chunks.

Spike 05 (LLM Behaviors): Not directly terrain-related but established the JSON behavior schema for game objects. Relevant because sculpted terrain features could trigger behaviors (e.g., a carved river spawns water effects).

Spike 06 (Chunk Streaming): First chunk load/swap system with dynamic loading as the player moves. Established the pattern that chunk-manager.ts uses: colored regions that load and unload. Sculpting must preserve edit state when chunks are unloaded and reloaded.

Spike 07 (GPU Vegetation from Density Maps): Instanced grass and trees placed via density maps that sample terrain height and slope. Sculpting invalidates vegetation placement: if the terrain height changes, trees may end up floating or buried. The density map must be regenerated for edited chunks.

Spike 08 (Terrain Material Shader Cost): Benchmarked triplanar projection, normal maps, and 4-layer blending. Measured exact ms cost of each feature. Found that triplanar + normals + 4 layers stays within budget at 45+ FPS. This budget matters for sculpted terrain: if we add a 5th layer for "edited soil" or change the blending for carved surfaces, we know exactly how much headroom exists.

Spike 09 (CSM Shadows Budget): Cascaded shadow maps with 3 cascades at 1024^2 resolution. Measured shadow cost at ~1.5ms. Sculpted terrain changes shadow maps, but the cost is constant regardless of terrain shape.

Spike 10 (Geometry Clipmaps + Geomorphing): Nested clipmap rings with geomorphing between LOD levels to eliminate pop-in. The constant triangle count means predictable GPU cost. Geomorphing matters for sculpting: when the player sculpts near an LOD boundary, the morph between LOD levels must reflect the edit. If the edit only exists in the high-res ring, the geomorph target is wrong.

Spike 11 (Heightmap Chunk Streaming): More advanced chunk streaming with a visual grid showing loaded/loading/unloaded states per LOD level. Established the streaming budget: max chunks loaded per frame, prioritization of chunks that need LOD upgrades. Sculpting adds a new priority signal: chunks the player is actively editing should never be unloaded.

Spikes 12-14: WebGPU + Three.js Integration

Spike 12 (WebGPU Marching Cubes): The first volumetric spike. Four 64^3 SDF chunks with animated sphere caves, running entirely on the GPU. Uses raw WebGPU: compute pipelines for SDF evaluation, MC extraction with the Twinklebear case table (256 configurations, 16 entries each), atomic vertex counter, and indirect draw. Performance target was <4ms per chunk, <12ms for all 4. This validated that GPU MC is fast enough for real-time re-meshing in the browser. Every subsequent volumetric spike reuses the MC case table and WGSL shaders defined here.

Spike 13 (Reset Baseline from Spike 12): Ported Spike 12's raw WebGPU draw path to run inside Three.js's WebGPURenderer, accessing the backend's device directly. The render pipeline still uses raw WebGPU (drawIndirect with struct Vertex vec4+vec4). This proved that custom compute and Three.js scene rendering can coexist on the same GPU device.

Spike 14 (Three.js WebGPU Incremental Hardening): Replaced the raw render pipeline with Three.js StorageBufferAttribute for positions and normals. MC compute writes directly into these GPU-resident buffers. The drawIndirect buffer controls how many vertices Three.js draws. This is the pattern all subsequent spikes use: compute stays as raw WebGPU, rendering goes through Three.js scene graph. The Three.js version across these spikes progressed from 0.170.0 to 0.172.0 as the WebGPU backend stabilized.

Spike 15-17: Transvoxel LOD Stitching

Spike 15 (Transvoxel Seam Scaffold): Added the three-zone architecture: MC chunk (volumetric center), transition strip (seam between MC boundary and heightmap), and terrain ring (surrounding heightmap). All three share a material pass. The transition strip is a placeholder mesh at this stage, not real Transvoxel cells.

Spike 16 (Transvoxel +X Face with Shared Heightmap): Two critical breakthroughs in one spike. First, replaced the flat terrain plane in the SDF with a shared Perlin heightmap: a 257x257 Float32Array uploaded to the GPU as a storage buffer, sampled via bilinear interpolation in the SDF compute shader. The MC surface and the heightmap mesh now agree on the same ground truth. Second, implemented real Transvoxel transition cells for the +X face by fetching Eric Lengyel's reference data tables from GitHub (transitionCellClass, transitionVertexData, transitionCellData) and the npm transvoxel-data package. The CPU evaluates 9-sample transition cells (512 configurations, 73 equivalence classes), places vertices by interpolating SDF values at grid points, and handles winding inversion for mirrored cases.

Spike 17 (Dual MC 1x/2x LOD): Two MC chunks side by side at different resolutions. High-res: 62 cells with cell_scale=1.0. Low-res: 31 cells with cell_scale=2.0. The MC shader gained cell_scale and grid_points uniforms. Introduced transition_shrink: the low-res chunk's face-0 boundary vertices are pulled inward by 15% of cell_scale, creating a thin gap that Transvoxel transition cells fill without z-fighting. This is the LOD model the production system needs: close chunks at full resolution, far chunks at half resolution, Transvoxel at every boundary.

Spikes 18-21: Transvoxel Corner Cases and GPU Acceleration

These four spikes each solved a specific failure case in the Transvoxel implementation. Grouping them obscures the distinct problems.

Spike 18 (Heightmap 2:1 Seam): Applied Transvoxel to a pure heightmap boundary where one side is twice the resolution of the other. No MC involved. The seam between a 62-cell and 31-cell heightmap chunk is generated from Transvoxel transition tables with 15% low-face shrink. This validated that Transvoxel works for the heightmap-only case, not just MC.

Spike 19 (64/32/32/16 Corner Grid): The hardest stitching case: four chunks of different resolutions meeting at a corner point (64, 32, 32, and 16 cells). The seam system must generate transition cells along four edges (A-B, A-C, B-D, C-D) with correct winding for each direction. This spike proved that the Transvoxel tables handle the multi-resolution corner without custom-case logic.

Spike 20 (GPU Transvoxel Corner): Moved the Transvoxel transition cell generation to the GPU for the 64/32/32/16 corner layout. The CPU was a bottleneck when regenerating transition cells every frame for animated terrain. GPU compute generates the seam vertices in the same pass as the MC extraction.

Spike 21 (GPU MC + Transvoxel Corner): Combined full GPU MC extraction with GPU Transvoxel seam generation in a single compute dispatch sequence. Both MC chunks and all four seams are generated on the GPU, with vertex counts managed via atomic counters and drawn with drawIndirect. This is the complete GPU pipeline for multi-resolution volumetric terrain with seamless LOD transitions.

Spikes 22-24: The Hybrid Architecture

Spike 22 (Hybrid MC/Heightmap Policy): The key architecture spike. Chunks are heightmaps by default. When the animated deformation sphere intersects a chunk's AABB, that chunk switches to MC mode. The rest stay as static heightmap meshes. Layout: 64, 32/32, and 16-cell chunks at different resolutions. Transvoxel seams handle every boundary, including MC-to-heightmap transitions. The spike tracks MC chunk count vs HM chunk count and vertex overflow per frame.

Spike 23 (Policy-Driven Chunk Modes): Loaded as a patch on top of Spike 22. Added camera-distance hysteresis (chunks don't flicker between modes when the camera is near a threshold) and an edit mask (chunks that have been deformed stay in MC mode even if the deformation source moves away). This is the "sticky edit" behavior sculpting needs: once a player carves a cave, that chunk stays volumetric forever.

Spike 24 (Policy + Clipmap Rings): The most advanced spike. Combines the near-field policy system from Spike 23 with far-field geometry clipmap rings from Spike 10. Upgraded to Three.js 0.183.1. Near-field uses HM/MC hybrid with Transvoxel seams at 64/32/16 resolution. Far-field uses static-center clipmap rings that follow the camera. This is the complete terrain rendering architecture: chunked volumetric sculpting where needed, cheap clipmap terrain everywhere else.

Why Marching Cubes, Not Surface Nets

The external research section of this guide suggests Surface Nets as the strongest candidate for browser terrain sculpting. But every spike in the pipeline uses Marching Cubes. That's not an accident.

MC's defining advantage is embarrassing parallelism: each cube is completely independent. The WGSL compute shaders in Spikes 12-24 dispatch one thread per cube with zero inter-cell communication. Atomic counters handle vertex allocation. This maps perfectly to GPU workgroups.

Surface Nets places one vertex per surface-containing cell, then connects neighbors. That neighbor connectivity is an inter-cell dependency. The fast-surface-nets crate handles it on the CPU with careful iteration order. On the GPU, it requires either a two-pass approach (find vertices, then connect) or shared memory within workgroups. Both are possible in WebGPU but add complexity.

The practical recommendation: stay with Marching Cubes for the sculpting pipeline. It's proven in our codebase, the WGSL shaders exist and are benchmarked, and the Transvoxel seam system is built around MC's edge-based vertex placement. Surface Nets is worth revisiting if MC's aliasing on binary data becomes a visible problem, but for SDF terrain where values are smooth gradients, MC produces clean results.

Practical Architecture for Sculpting

The spike sequence solved the rendering pipeline. What remains is the brush system, cascading side effects through the game systems, and multiplayer synchronization. Here's the plan, building on every spike.

Phase 1: Heightmap Sculpting (Minimal Change, Maximum Reach)

Add brush tools that modify chunk heightmaps in the production world/client/ code. This works within the existing WebGL renderer and doesn't require WebGPU.

Brush input: Follow the PlacementTool pattern in placement.ts. It already has a Raycaster hitting chunkManager.getChunkMeshes() and tracking a ghost mesh at the hit point. A TerrainBrushTool would do the same raycast but modify the chunk heightmap instead of placing an object. The World.onMouseDown handler already dispatches based on tool state.

Chunk modification (Chunk.applyBrush): Map the brush world position to heightmap grid coordinates. For each grid point within the brush radius, compute the falloff-weighted displacement and add/subtract from the heightmap value. Then update the mesh: set vertex Y positions from the modified heightmap, recompute normals via central differencing (the same terrainHeight(wx +/- eps, wz) pattern already used in chunk.ts line 155-158), and regenerate the splat map via createSplatMap() in terrain-material.ts for the affected region so slope-driven texture blending updates.

Character controller: CharacterController.update() calls getHeight() every frame for grounding. ChunkManager.getHeight() delegates to Chunk.sampleHeight(), which reads from the chunk's heightmap Float32Array. Since we're modifying that array directly, the character controller picks up the change on the next frame with zero additional wiring.

Object invalidation: Tree instances in chunk.ts are placed by sampling terrainHeight() at spawn time. After sculpting, trees in the affected area may be at the wrong height. Phase 1 can defer this (trees float slightly on small edits). Phase 2 needs a chunk.invalidateObjects() that re-samples heights and rebuilds the instance matrices. Same for the colliders used in resolveCollisions().

Rapier physics (if integrated): Spike 02 proved heightfield colliders work. If Rapier is active, the heightfield collider must be rebuilt or patched for the modified chunk. Rapier's ColliderDesc.heightfield() takes a flat Float32Array, so it's a straight replacement.

Network sync: Add MsgType.TerrainEdit = 10 to protocol.ts:

interface TerrainEditMsg {
  t: MsgType.TerrainEdit
  cx: number
  cz: number
  brush: {
    wx: number
    wz: number
    radius: number
    strength: number
    falloff: number
    operation: number
  }
}

The WorldChunkDO broadcasts this to all clients and appends it to a per-chunk edit log stored in Durable Object storage. Late-joining clients receive the edit log in the Snapshot message and replay it to reconstruct the terrain state. All clients apply the same deterministic brush function, so they converge on the same heightmap.

Chunk unload/reload: Spike 06 and Spike 11 established the streaming pattern. When a chunk is unloaded and later reloaded, the edit log for that chunk must be replayed against the base procedural heightmap. The edit log is stored server-side (Durable Object) and included in the Snapshot message.

Phase 2: Volumetric Sculpting with the Spike 22-24 Architecture

Port the Spike 24 pipeline into the production world. When a player sculpts below the surface (carving a cave, digging a tunnel), the affected chunk transitions from heightmap mode to MC mode.

WebGPU renderer migration: Spikes 13-14 proved that Three.js's WebGPURenderer can host custom compute alongside the scene graph. The production world moves from WebGLRenderer to WebGPURenderer with StorageBufferAttribute for MC chunks. Fallback to the Phase 1 heightmap-only path when WebGPU is unavailable.

Per-chunk SDF allocation: Follow Spike 22's hybrid pattern. Each chunk starts as a heightmap. On the first volumetric brush stroke, allocate a 64^3 Float32Array, initialize it by sampling the heightmap (the SDF value at each point is world.y - heightmap_value), and switch to MC rendering. The policy system from Spike 23 ensures the chunk stays in MC mode permanently (the "sticky edit" behavior from the edit mask).

Shared heightmap in the SDF shader: Spike 16's height_at() function. Upload the chunk's heightmap to a GPU storage buffer. The SDF compute shader evaluates max(height_sdf, edit_sdf) where height_sdf = world.y - height_at(world.xz) and edit_sdf contains the brush modifications. MC and heightmap chunks agree on ground truth at their boundaries.

Transvoxel seams: The full stack from Spikes 15-21. MC-to-heightmap boundaries use transition cells with the shrink gap. MC-to-MC boundaries at different resolutions use Spike 17's dual-LOD pattern. The corner case from Spike 19 handles 4-way intersections. Spike 21's GPU compute generates all seam geometry in the same dispatch.

Clipmap far-field: Spike 24's clipmap rings for terrain outside the sculpting range. Sculpting never touches these rings; they sample the base procedural heightmap.

Geomorphing: Spike 10's geomorphing eliminates pop-in at LOD transitions. For edited chunks, the geomorph target must include the edit. If a chunk is MC at LOD0 and its LOD1 neighbor is a heightmap, the geomorph blends between the two representations. This requires sampling the edit log even at lower LODs.

Material budget: Spike 08 benchmarked 4-layer triplanar + normals at 45+ FPS. MC chunks need the same material. The splat map can be generated from the SDF gradient (steep = rock, flat = grass) instead of the heightmap slope. This stays within the 4-layer budget.

Vegetation invalidation: Spike 07's density-map vegetation depends on terrain height and slope. When a chunk transitions to MC mode, tree instances must be regenerated by sampling the SDF surface. Trees on overhangs or inside caves must be culled. The instanced mesh matrices from chunk.ts are rebuilt from the new surface.

Phase 3: CSG Edit Trees for Undo/Redo and Network Sync

Replace raw SDF mutation with a CSG operation tree. Each brush stroke appends a primitive (sphere, capsule, box) with an operation (add, subtract, smooth blend). The SDF is recomputed from the tree.

Benefits:

• Non-destructive: any edit can be removed from the tree to undo it
• Network-efficient: broadcast the CSG operation, not the raw field values
• Deterministic: all clients build the same SDF from the same operation sequence
• ALICE-SDF's CSG tree diff/patch provides bandwidth-efficient synchronization and undo/redo across the network

Durable Object storage: The edit tree per chunk replaces the flat edit log from Phase 1. The WorldChunkDO stores the CSG tree structure, not raw heightmap deltas. Snapshot messages include the tree, and late-joining clients evaluate it to produce the local SDF.

Phase 4: Collaborative Sculpting

Add concurrent edit support with server-ordered operation replay. The Durable Object timestamps each edit and broadcasts in order. Late-joining clients receive the operation log and reconstruct the world state. The existing Snapshot message type extends to include terrain edit history per chunk.

Since brush strokes are small, localized, and additive/subtractive, the visual result of slightly reordered concurrent edits is usually indistinguishable from the "correct" order. Last-write-wins with server ordering is sufficient. The Durable Object's tick() function (currently running at 50ms intervals for player state) adds terrain edit broadcasts to the same loop.

Key References

Algorithms:

• Lorensen & Cline, "Marching Cubes" (1987)
• Eric Lengyel, "Transvoxel Algorithm" (2009), transvoxel.org
• Losasso & Hoppe, "Geometry Clipmaps" (SIGGRAPH 2004)
• "A High-Performance SurfaceNets Discrete Isocontouring Algorithm" (arxiv 2401.14906, 2024)
• MCHex (arxiv 2511.02064, 2025)

Implementations:

• bevy-sculpter v0.18.0 (Rust, Surface Nets + SDF brushes)
• fast-surface-nets (Rust, 20M tri/sec)
• ALICE-SDF v1.3.0 (Rust + WASM, CSG tree diff/patch)
• WebGPU SDF Editor (Nijhoff, January 2026)
• SculptingPro (Unity runtime sculpting API)
• TerraBrush (Godot terrain sculpting GDExtension)

Games:

• Dreams (Media Molecule, SDF + point cloud rendering, SIGGRAPH 2015)
• Teardown (Voxagon, voxel DDA ray tracing, deterministic multiplayer destruction)
• Mike Turitzin's SDF engine (brick maps + geometry clipmaps, January 2026)

Network:

• "Optimizing payload size for voxel state synchronization" (Oulu, 2024)
• Teardown multiplayer (semi-deterministic destruction sync, March 2026)
• cSculpt (collaborative mesh sculpting with multiresolution merge)