What the data says about wishlists, demos, and timing. Real numbers from 200+ games.

If you're planning around the calendar, Steam runs Next Fest three times in 2026: February 23 to March 2 (already done), June 15 to 22, and October 19 to 26. The June and October editions run from 10:00 AM PDT on the start day to 10:00 AM PDT on the end day, landing just before the Summer and Winter sales (February ran on Pacific time too, before daylight saving). A game can only join one Next Fest, ever, so pick the edition that fits your release window. Your demo has to be publicly playable by the time the fest opens, and you can't release until after it closes.

Here's the thing about Next Fest: everyone treats it like a lottery. Throw your demo in, cross your fingers, hope the algorithm smiles on you. That's not how it works.

I've been reading postmortems obsessively. Hundreds of them from 2025 alone. The pattern is so consistent it's almost boring: the games that "win" Next Fest aren't discovering success during the festival. They're amplifying momentum they already built.

Let me show you what I mean.

The Formula Nobody Wants to Hear

Wishlists Before × 2 = Wishlists Gained

This ratio shows up in postmortem after postmortem.

Eilean Mor went into October 2025 with 2,117 wishlists. Gained 2,082 during the festival. Almost exactly 2×.

[IMAGE: Eilean Mor trailer screenshot or embed - https://www.youtube.com/watch?v=jFv7SjsQEws]

Dice of Kalma went in with 335 wishlists. Gained 659. Just under 2×. Their developer wrote: "Our wishlists nearly tripled, which is awesome, but we have still lot of work to do."

That's the uncomfortable truth. Next Fest doubles what you already have. If you have nothing, you get nothing doubled.

The 2,000 Wishlist Cliff

Data from 208 games in February 2025 shows a sharp inflection point at 2,000 wishlists.

Here's what a developer from that survey said: "You really start to see how the wishlists a game earns takes off for games that have 2000+ wishlists before Steam Next Fest."

It's not an algorithm thing exactly. It's a validation thing. 2,000 wishlists means you've figured out something about marketing. You know how to reach people. That skill compounds during the festival.

The First 48 Hours Are Everything

Valve gives everyone a shot on days 1 and 2. After that, the algorithm decides who deserves more visibility based on performance.

Bronze tier games? Their impressions fall off a cliff on day 3. Diamond tier games? Their impressions triple.

Day 1-2: Push everything. Stream. Post. Coordinate with creators. Day 3+: The algorithm has already decided. You're along for the ride.

What Actually Won in 2025

Do No Harm (February 2025)

[IMAGE: Do No Harm trailer screenshot or embed - https://www.youtube.com/watch?v=GBlOHAL1LkY]

The numbers: 45,655 wishlists gained during Next Fest. Started with 52,102.

What they did:

• Trailer dropped on GameTrailers January 29 (weeks before Next Fest)
• PR outreach timed with trailer
• Demo went live February 4
• Paid ads to amplify organic momentum

This is the playbook. They didn't rely on Next Fest to discover them. They used Next Fest to multiply attention they'd already earned.

Cairn (October 2025)

[IMAGE: Cairn trailer screenshot or embed - https://www.youtube.com/watch?v=x6KWhvvk3bM]

Hit #5 on Popular Upcoming on day 1. Top Demos and Trending Upcoming by day 2.

What they did:

• Demo launched December 2024 (10 months before their Next Fest)
• 32,000 followers before the festival
• Accepted into 18 festivals before Next Fest
• The Game Bakers (Furi, Haven) - established studio with track record

They didn't need Next Fest. But they used it anyway because it's free amplification when you've already done the work.

YAPYAP (October 2025)

[IMAGE: YAPYAP gameplay screenshot or embed - https://www.youtube.com/watch?v=7d8Fd9XXAtQ]

The "friend slop" phenomenon. Co-op horror where you play as wizards vandalizing a tower. Went viral on social media, then dominated Next Fest.

Their trailer dropped August 25, 2025 - one month before Next Fest. It earned 434K views. That was the inflection point. Everything after was just riding the wave.

What February 2026 Confirmed

The February 2026 edition didn't change the playbook, it hardened it. The fest crossed 3,500 games, up 51% year over year, and the median title walked away with about 200 wishlists and 11 new followers. More games, thinner slices. The top 5% gained roughly 7,000 wishlists, down from about 10,000 a year earlier, so even the winners are splitting a more crowded pie. Co-op kept dominating, and the "shadow-drop your demo right before the fest" idea got debunked again: of the ten top earners analyzed, only one launched its demo just days before. The rest had demos out for a month or longer. Build the audience first, every edition keeps saying the same thing.

The Genres That Keep Winning

From the February 2025 diamond tier:

• Horror (×2) - Steam loves horror. Always has.
• Strategy/Management (×4) - "Crafty-buildy-simulationy" is the vibe
• Co-op "friend slop" - Lethal Company opened the floodgates
• Point-and-click (×2) - but both were sequels with existing audiences

The point-and-click thing is interesting. Beholder: Conductor was the 4th game in its series. Duck Detective was a sequel. They had built-in audiences. First-time point-and-click devs don't see those results.

What The Small Games Say

Not everyone is chasing diamond tier. Here's what developers at the smaller end reported from October 2025:

"62% conversion rate from visits to wishlists. Store page itself seems to work. But impressions were high and people didn't click. Capsule art needs an update."

"57% of store visitors played our demo. That sounds high? But only 15% who played also wishlisted. Something didn't meet expectations."

"Got permanently banned from r/cozygames for asking if they found our game cozy. Always try to follow the rules but this was pretty surprising."

The grind is real. These developers are testing capsule art, tweaking tags, getting banned from subreddits. It's not glamorous.

One pattern that's gotten stronger: co-op "friend slop" games rack up huge demo play counts but convert fewer of those players into wishlists, because the audience showing up is casual and there to mess around with friends, not to track a release. High plays don't automatically mean high wishlists.

The Honest Playbook

Months before Next Fest:

1. Launch your demo early. Like, way early. The winners had demos out for 6-12 months.
2. Enter smaller genre festivals first. Use them as polish runs.
3. Build to 2,000+ wishlists. That's your real threshold.

During Next Fest:

1. Days 1-2 are make or break. Push everything.
2. Stream if you can. It shows as "live" on your page.
3. Coordinate with streamers. Pay them if you have to.
4. Put your demo on a separate Steam page so it can collect reviews.

After Next Fest:

1. Don't launch immediately. A few weeks later actually performs better.
2. Use the momentum for your actual launch marketing.

When To Skip It

Real talk: Next Fest isn't for everyone.

Skip it if:

• You have under 1,000 wishlists (you'll be invisible by day 3)
• Your demo isn't polished (you get one shot at first impressions)
• It's your first festival ever (do smaller ones first)
• You're launching immediately after (wait a few weeks)

There are dozens of smaller festivals every month. More focused audiences. Less competition. Check howtomarketagame.com/festivals for the list.

Next Fest is the default because Valve prompts everyone to join. But the best marketing isn't the stuff everyone gets prompted to do.

About the Author

Oleg Sidorkin builds tools for indie game developers. He's been studying game marketing data and developer postmortems to understand what actually works versus what gets hyped.

Notes for Game Developer editors:

IMAGE PLACEMENTS:

1. After "Almost exactly 2×" paragraph - Eilean Mor video embed or screenshot
2. After "Do No Harm (February 2025)" header - Do No Harm video embed or screenshot
3. After "Cairn (October 2025)" header - Cairn video embed or screenshot
4. After "YAPYAP (October 2025)" header - YAPYAP video embed or screenshot

VIDEO URLS:

• Eilean Mor: https://www.youtube.com/watch?v=jFv7SjsQEws
• Do No Harm: https://www.youtube.com/watch?v=GBlOHAL1LkY
• Cairn: https://www.youtube.com/watch?v=x6KWhvvk3bM
• YAPYAP: https://www.youtube.com/watch?v=7d8Fd9XXAtQ

STEAM LINKS (for reference/linking):

• Eilean Mor: https://store.steampowered.com/app/3871570/Eilean_Mor_The_Lost_Keepers/
• Do No Harm: https://store.steampowered.com/app/3138780/Do_No_Harm/
• Cairn: https://store.steampowered.com/app/1588550/Cairn/
• YAPYAP: https://store.steampowered.com/app/3834090/YAPYAP/

Unity can export to the web, but its WebGL builds are heavy, its pricing has whiplashed developers since 2023, and for a browser-first game it's rarely the lightest path. If you're looking for a Unity alternative specifically for web games, this guide covers the engines that actually ship small, fast browser builds in 2026, and which one to pick based on why you're leaving Unity.

If you want the whole field side by side, start with our best web game engines for 2026 comparison. This page is for the specific case of replacing Unity for the web.

Why developers look for Unity alternatives

A few reasons come up again and again:

• Pricing trust. The Unity Runtime Fee was cancelled in September 2024 and hasn't returned, but it taught a lot of teams to plan around the question "what if the terms change again?" An engine whose license can't be repriced removes that risk.
• Web build size. Unity's empty WebGL builds start around 8 to 11 MB. Web-native engines ship in the 1 to 5 MB range, which players feel directly in load time.
• Mobile browser support. Unity WebGL on phones has historically been fragile. Lighter engines built for the web behave better on mobile Safari and Chrome.
• Open source. Some teams want a codebase they can read, fork, and depend on without a vendor.

The good news: for web, the alternatives are strong, and several are free forever.

Quick comparison

There's no single "Unity replacement." The right pick depends on whether you need 3D, whether you write code, and why Unity wasn't working for you.

Godot: the closest all-round replacement

Godot is the alternative most ex-Unity developers land on. It's a full 2D and 3D engine, free under the MIT license with no seats, royalties, or revenue caps, and its web export is solid. Build size is around 5 MB compressed with Brotli, smaller than Unity's by default. Since Godot 4.3 you can export single-threaded web builds that drop the SharedArrayBuffer requirement, which also fixed the old iOS and Safari playback problems.

The one catch for web: Godot's C# (.NET) projects can't export to the browser yet, only GDScript does. If you're moving a C# codebase, that matters. We cover the trade-offs in detail in Godot vs Unity for web games.

Pick Godot if you want the most Unity-like all-round engine, free forever, with a clean 2D and 3D web path.

PlayCanvas: web-first 3D with an editor

PlayCanvas was built for the browser from day one. Its runtime is roughly 1 to 2 MB, it supports both WebGL2 and WebGPU, and it has a collaborative cloud editor that feels familiar if you liked Unity's scene workflow. Games are scripted in JavaScript. It's a strong choice when you need real-time 3D in the browser and want a visual editor rather than a code-only library.

Pick PlayCanvas if you want Unity-style 3D scene editing but a tiny web build.

Three.js and Cinevva: maximum control, smallest footprint

If you want the lightest possible web build and full control over rendering, Three.js is the standard. The core library is around 150 KB and it powers a huge share of 3D on the web. It's a rendering library rather than a full engine, so you bring your own game systems or a framework on top.

That's exactly why we built Cinevva Engine on Three.js: open-source, web-first, with the game systems layered on so you don't start from an empty canvas. One codebase ships to the browser, mobile, desktop, Steam, and Discord. No seats, no install fees, no terms that can change after you've built your game, which is the thing the Runtime Fee episode made people care about.

Pick Three.js or Cinevva if you want web-native 3D, open source, and the smallest footprint, and you're comfortable in JavaScript or TypeScript.

Defold: the smallest, fastest 2D web builds

Defold ships the smallest web builds of any full engine, around 1.14 MB, and loads fast. It does 2D and lighter 3D, scripts in Lua, and is free with no revenue thresholds. If your Unity game is 2D and load time is the priority, Defold is hard to beat on the web.

Pick Defold for casual and mobile web games where every kilobyte counts.

Cocos Creator: mini-games and the Asian market

Cocos Creator is a full 2D and 3D engine with WebGL, WebGPU, and Wasm support, builds in the 2 to 4 MB range, and scripts in TypeScript. It's especially strong for mini-game platforms (WeChat, TikTok, Taobao) where Unity's size is a non-starter. It's free.

Pick Cocos Creator for mini-games or if you're targeting the Asian web ecosystem.

Phaser: the 2D web standard

If your game is 2D and you write JavaScript, Phaser is the most popular HTML5 game framework, with the largest community and ecosystem. Builds are around 500 KB. It's free under the MIT license. For 2D web games it's often a better fit than Unity ever was. See best 2D game engines for 2026 for how it compares to Defold, Construct, and GDevelop.

Pick Phaser for 2D web games when you want code, community, and a small build.

Construct and GDevelop: leaving Unity without code

If part of why Unity frustrated you was the amount of code and setup, the visual engines are a real alternative. Construct and GDevelop both build 2D games with no programming and export clean web builds. GDevelop has a free tier and is open-source-friendly; Construct is subscription-based with a polished editor.

Pick one of these if you want to build web games visually with little or no code.

How to choose, by reason for leaving Unity

• Left over pricing or trust? Go open source: Godot, Defold, Phaser, Three.js, or Cinevva. None can be repriced out from under you.
• Need 3D in the browser? PlayCanvas (with an editor) or Three.js and Cinevva (for control and the smallest size).
• Building 2D? Defold for the smallest builds, Phaser for the biggest community.
• Don't want to code? Construct or GDevelop.
• Targeting mini-game platforms? Cocos Creator.

Whatever you pick, the web rewards small builds and fast loads. Once you've chosen, see ship a web game that loads fast.

Common Questions

What is the best Unity alternative for web games?

For most teams it's Godot: a free, open-source, full 2D and 3D engine with smaller web builds than Unity and a clean single-threaded export path for iOS and Safari. If you specifically need browser-first 3D, PlayCanvas or Three.js are lighter still. The best pick depends on whether you need 3D and whether you write code.

Is there a free alternative to Unity?

Yes, several. Godot, Defold, Phaser, PixiJS, Excalibur.js, and Three.js are all free and open-source with no revenue caps or seat fees. Cocos Creator and GDevelop also have free options. For web games specifically, these free engines often ship smaller, faster builds than Unity.

Why switch from Unity for web games?

The common reasons are build size (Unity's empty WebGL builds start around 8 to 11 MB versus 1 to 5 MB for web-native engines), mobile browser reliability, pricing trust after the 2023 Runtime Fee episode, and a preference for open source. If web is your main target, a web-first engine usually gives a better result.

Can I replace Unity with an open-source engine?

Yes. Godot is the closest open-source equivalent for general 2D and 3D, and it's MIT licensed with no fees. For web-native 3D, Three.js (and engines built on it like Cinevva) are open source too. The main migration cost is rebuilding in a new language, since each engine has its own scripting.

Which Unity alternative has the smallest web build?

Among full engines, Defold is the smallest at about 1.14 MB. Three.js as a library is even smaller, with a core around 150 KB, though you add your own game code. Godot lands around 5 MB compressed, still well under Unity's 8 MB-plus starting point.

Related

• Best Web Game Engines for 2026 (Compared) — the full field, ranked
• Godot vs Unity for Web Games — the closest alternative, head to head
• Unity Runtime Fee Cancelled: the official story — why pricing trust became a deciding factor
• Best 2D Game Engines for 2026 — if your game is 2D
• PlayCanvas vs Three.js — engine versus library for web 3D
• Cinevva Engine — our open-source, web-first engine built on Three.js

2D is alive and well in 2026, and you've got more good engines than ever. The right one depends on whether you code or not, whether you're shipping to the web, and how small you need the build. Here's how the main options compare, and who each one is for.

If you're weighing 3D or full cross-platform engines too, see the broader web game engines comparison.

Quick Reference

The Engines

Godot

The open-source favorite, and a genuinely excellent 2D engine. Godot 4.5 (September 2025) has a dedicated 2D renderer, a lightweight editor, and beginner-friendly GDScript. It's free under the MIT license with no royalties. Web export is solid on WebGL2, with the caveat that only GDScript exports to the browser, not C#. If you want one free tool that does great 2D and decent 3D, this is the default pick.

Best for: all-round 2D, open-source projects, pixel art.

GameMaker

The engine behind hits like Undertale and a long line of commercial 2D games. It uses GML, its own fast scripting language, and ships HTML5 export even on the free tier. Pricing changed to a friendly model: free for non-commercial use and a one-time $99.99 Pro license for commercial work, instead of a subscription.

Best for: commercial 2D where you want to ship fast, pixel art.

Construct

A browser-based, no-code engine built around an event system. You build games visually with no programming, and because it's HTML5-native, web export is excellent. Pricing is subscription-based, roughly $5 to $39 a month depending on tier, with a limited free option.

Best for: beginners, no-code creators, and web-first 2D.

Defold

A free engine from former King developers that produces astonishingly small web builds, often under 2 MB gzipped. That makes it ideal for casual web games and mini-game platforms where load time is everything. It uses Lua and is source-available under the Defold License.

Best for: the smallest, fastest-loading web games.

Phaser

The most popular HTML5 game framework, and a code-first one. Phaser 4 (released April 2026) brought a new WebGL renderer rebuilt from the ground up. It's free and open-source, you write JavaScript or TypeScript, and because it's a web framework there's no "export" step, your game already runs in the browser.

Best for: developers who want a code-first, web-native 2D framework.

GDevelop

Open-source and no-code, GDevelop is one of the most beginner-friendly engines around. It's free with an optional premium tier from $5 a month, and it exports unlimited HTML5 builds. A good middle ground if you want no-code but also want open-source.

Best for: free no-code 2D.

Unity

Unity's 2D toolset is mature, and the ecosystem and asset store are unmatched. It's overkill for a small 2D game, but if you want cross-platform reach or plan to grow into 3D, it earns its place. Personal is free under $200K revenue, Pro is about $2,200 a year per seat. Unlike Godot, Unity's C# does export to the web.

Best for: teams that want the ecosystem or cross-platform reach.

LÖVE (Love2D)

A minimalist Lua framework, not a full editor. You write code, it runs. LÖVE 11.5 is the current stable release, with 12.0 in development. Web export exists through the third-party love.js, but it isn't first-class. Reach for LÖVE if you love writing code and want almost no abstraction between you and the game loop.

Best for: code-focused hobbyists and lightweight prototypes.

How to Choose

• You don't want to code: Construct or GDevelop. Construct if you're shipping to web, GDevelop if you want open-source.
• You want the smallest, fastest web build: Defold.
• You write code and target the web: Phaser.
• You want one free engine for everything: Godot.
• You're shipping a commercial 2D game: GameMaker or Unity.
• You need the biggest ecosystem or plan to add 3D: Unity.

Whichever you pick, you'll need art and audio. See where to find free game assets for sources you can drop straight in.

Common Questions

What is the best 2D game engine in 2026?

For most people, Godot is the best all-round 2D engine: free, open-source, with a first-class 2D renderer. If you don't want to code, Construct (web-first) or GDevelop (open-source) are better. For commercial 2D, GameMaker and Unity are proven. The "best" one depends on whether you code and where you're shipping.

Which 2D game engine is best for beginners?

Construct and GDevelop are the most beginner-friendly because they're no-code, you build with visual event systems instead of programming. GDevelop is free and open-source, Construct is subscription-based but has the smoothest web export. If you want to learn to code along the way, Godot with GDScript is the gentlest of the code-based engines.

What is the best free 2D game engine?

Godot is the strongest free, open-source 2D engine with no fees or royalties. GDevelop is the best free no-code option, Defold is free with the smallest web builds, and Phaser is a free open-source framework if you code. GameMaker is free for non-commercial use, then a one-time $99.99 license for commercial projects.

Which 2D engine makes the smallest web games?

Defold produces the smallest web builds, frequently under 2 MB gzipped, which is why it's popular for casual and mini-game platforms where load time decides whether players stick around. Phaser and Godot are also reasonable, but Defold leads on raw footprint.

Related

• Best Web Game Engines for 2026 (Compared) — 2D and 3D, the full field
• Godot vs Unity for Web Games — the two big engines head to head
• Construct vs GDevelop — the two no-code 2D engines compared
• Where to Find Free Game Assets — art and audio for your 2D game
• Canvas 2D game loop — build a 2D game with no engine at all
• Cinevva Engine — our open-source, web-first engine

If you want to make a game without writing code, these two lead the pack. Both build games with visual event logic instead of programming, and both export to the web. The big split is philosophy and price: Construct is a polished subscription product, GDevelop is free and open source. Here's how they compare in 2026.

For the wider field including code-based engines, see the web game engines comparison and best 2D game engines for 2026.

Quick Verdict

No-Code, Two Ways

Both engines let you build a game by stacking visual conditions and actions, no programming required. Construct uses "event sheets," a clean, spreadsheet-like flow that's widely praised for how fast you can prototype in it. GDevelop uses a similar event system and is just as approachable for beginners.

The difference that matters most: Construct is a paid, closed-source product; GDevelop is free and open source under the MIT license. If open source or zero cost is a priority, that alone may decide it.

Pricing

Construct 3 runs on a subscription. There's a limited free edition, and the Personal plan is around $15.99 a month or $129.99 a year, with Education and Business tiers above that. You're paying for a polished, actively developed product.

GDevelop's engine is free and open source. Optional premium tiers (around $5.49 Silver, $10.99 Gold, and $32.99 Pro per month) add cloud builds and online services, but you can build and export games fully for free. Prices rose modestly in early 2026.

For a hobbyist or anyone cost-sensitive, GDevelop's free tier is hard to argue with. For a studio that wants a refined commercial tool, Construct's subscription is reasonable.

3D, Scripting, and Multiplayer

3D: GDevelop pulled ahead here. Its 5.6 release (December 2025) shipped a full real-time 3D editor with Jolt-based physics, shadows, animated models, 3D particles, and ready-made character and vehicle behaviors. Construct added 3D too, but it's more limited, a 3D camera plus 3D object plugins that load glTF models, layered on a fundamentally 2D engine.

Scripting: when you outgrow events, both let you drop into code. Construct supports JavaScript and TypeScript alongside event sheets. GDevelop supports JavaScript through its events and extensions. Both keep the no-code path primary.

Multiplayer: Construct ships a built-in Multiplayer object over WebRTC data channels for peer-to-peer games. GDevelop offers multiplayer through its online services and extensions.

When to Pick Which

Pick GDevelop if you want free and open source, cross-platform export, and the more capable real-time 3D editor. It's the better fit for hobbyists, students, and anyone who wants to own their tooling.

Pick Construct if you want the most polished event-sheet workflow, mature and reliable HTML5 export, and built-in WebRTC multiplayer, and you don't mind a subscription for a refined commercial product.

Either way, you'll need art and audio. See where to find free game assets for sources you can use right away.

Common Questions

Is Construct or GDevelop better for beginners?

Both are designed for beginners and use visual event logic with no coding required. GDevelop wins on cost (it's free and open source) and now has a stronger 3D editor. Construct wins on workflow polish and built-in multiplayer. If budget matters, start with GDevelop. If you want the most refined 2D event workflow, try Construct's free edition first.

Is GDevelop really free?

Yes. The GDevelop engine is free and open source under the MIT license, and you can build and export HTML5 games at no cost. Optional premium tiers (from about $5.49/month) add cloud builds and online services, but they're not required to make and ship a game.

Can Construct and GDevelop make 3D games?

Both have 3D, but GDevelop is further along. GDevelop 5.6 (December 2025) added a full real-time 3D editor with physics, shadows, and animated models. Construct offers a 3D camera and 3D object plugins that load glTF models, but it remains primarily a 2D engine. For 3D-leaning projects, GDevelop is the stronger no-code option.

Do Construct and GDevelop export to the web?

Yes, both export HTML5 games that run in the browser. Construct is HTML5-native with mature, reliable web export. GDevelop exports unlimited HTML5 builds, including locally for free. Both are solid choices for browser games.

Related

• Best 2D Game Engines for 2026 — the full 2D field, code and no-code
• Best Web Game Engines for 2026 (Compared) — 2D and 3D
• Where to Find Free Game Assets — art and audio for your game
• How to Launch Your Game on itch.io — getting your game in front of players
• Cinevva — make games with AI-powered tools, web-first

Both Godot and Unity can ship a game to the browser. They get there very differently, and for web specifically the gap between them is sharper than the general "Godot vs Unity" debate suggests. This guide compares them on the things that actually decide a web build: download size, rendering, language support, mobile browsers, and cost.

If you want the wider picture across every browser engine, start with our web game engines comparison. This page zooms in on the two big full-engine options.

Quick Verdict

The single biggest deciding factor: if your game is written in C#, Unity can ship it to the web and Godot still cannot. Everything else flows from there.

Build Size and Load Time

Download size is the first thing a web player feels. Godot's web export ships the engine as a WebAssembly binary that's around 40 MB uncompressed but compresses to roughly 5 MB with Brotli. The size comes from the prebuilt template carrying the full engine (3D renderer, audio, networking) because it can't know which features your game uses.

Unity's empty web builds, measured on Unity 6 with Brotli, land around 7.7 MB for a 2D built-in pipeline project and 10.7 MB for a 3D URP template. An aggressively stripped minimal build can get down near 2 MB. So a carefully trimmed Unity 2D build can approach Godot's footprint, but Godot's default is smaller with less effort.

For both engines, your assets usually dwarf the engine binary anyway. Compress textures and audio, stream what you can, and the engine choice matters less than your asset budget.

The C# Limitation (Read This Before You Choose)

This is the one that catches teams out. As of 2026, Godot projects written in C# (.NET) cannot officially export to the web. GDScript exports fine. A working C# web prototype was demonstrated at GodotCon Boston in May 2025 by statically linking Mono, but it shipped a large payload and the team would not commit to a release timeline.

Unity, by contrast, compiles C# to WebAssembly and has shipped C# to the browser for years. So if you have an existing C# codebase, or your team only knows C#, Unity is the practical choice for web. If you're starting fresh and happy in GDScript, Godot's web path is clean.

Rendering: WebGL2 vs WebGPU

Godot's web export targets WebGL2 through its Compatibility renderer. Its broader WebGPU work isn't production-ready for web export yet. Godot 4.5 (September 2025) did add WebAssembly SIMD, which helps web performance without code changes.

Unity 6 added an experimental WebGPU backend that exposes compute shaders and modern GPU features beyond WebGL2, with WebGL2 still the safe fallback. WebGPU isn't available on every browser yet (see our web games tech stack guide for current support), so treat it as an upgrade path, not a baseline. Unity 6 also raised the WebAssembly heap to 4 GB and added Wasm SIMD and native exceptions.

If you need compute shaders or heavier 3D in the browser today, Unity's WebGPU path is ahead. For 2D and lighter 3D on WebGL2, both are fine.

Threading, iOS, and Hosting Headaches

Godot 4 originally required SharedArrayBuffer, which browsers only expose when the page is cross-origin isolated (the COOP and COEP headers). That's a hosting headache, and on itch.io it only works in Chrome. Since Godot 4.3 you can export single-threaded builds that drop the requirement entirely, and that change also fixed the long-standing iOS and Safari playback problems. For a web-first Godot game, single-threaded export is usually the safer default.

Unity's web builds use WebAssembly too, with mobile web support starting at Unity 6. If you're targeting broad mobile browsers, test on real devices early in both engines, because that's where web builds break first.

Pricing

Godot is free under the MIT license. No seats, no royalties, no revenue cap, no fees, ever.

Unity reverted to seat-based subscriptions after canceling the Runtime Fee in September 2024 (it hasn't returned). Unity Personal stays free up to $200K in revenue or funding, and the "Made with Unity" splash became optional on Unity 6. Unity Pro is about $2,200 per seat per year, with a roughly 5% increase to paid tiers in early 2026. For a solo dev or small team under the cap, both engines are effectively free to ship with.

When to Pick Which

Pick Godot for web when you want a free, small-footprint 2D or lighter-3D browser game, smooth iOS and Safari support through single-threaded export, and you're comfortable writing GDScript.

Pick Unity for web when you need heavier 3D or compute shaders via WebGPU, you want the larger asset ecosystem, or you're a C# team, and you can accept larger builds and the experimental-WebGPU caveat.

Either way, your build still has to load fast and run on a phone. See ship a web game that loads fast once you've picked.

Common Questions

Can Godot export C# games to the web?

Not officially, as of 2026. Godot's web export works for GDScript, but C# (.NET) projects cannot export to the browser yet. A prototype was shown at GodotCon Boston in May 2025, but there's no committed release date. If you need C# on the web, Unity is currently the only one of the two that ships it.

Is Godot or Unity smaller for web builds?

Godot's empty web build is around 5 MB compressed with Brotli. Unity 6 empty builds run about 7.7 MB for 2D and 10.7 MB for 3D, though an aggressively stripped Unity build can reach roughly 2 MB. Godot is smaller by default, but a tuned Unity 2D build can get close. Your own assets usually matter more than the engine binary.

Does Unity support WebGPU in the browser?

Yes, Unity 6 added an experimental WebGPU backend with compute shaders and modern GPU features, with WebGL2 as the fallback. It's labeled experimental and isn't supported on every browser, so ship WebGL2 as your baseline and treat WebGPU as an enhancement.

Which is better for browser games, Godot or Unity?

For free, lightweight 2D or lighter 3D games with good iOS support, Godot is the cleaner web path, as long as you use GDScript. For heavier 3D, compute shaders, the bigger ecosystem, or an existing C# codebase, Unity wins on web. The C# question usually decides it.

Related

• Best Web Game Engines for 2026 (Compared) — the full field, beyond these two
• Unity Alternatives for Web Games — every web-capable option, picked by why you're leaving Unity
• Best 2D Game Engines for 2026 — if your game is 2D, start here
• Web Games Tech Stack in 2026 — WebGL, WebGPU, and Wasm explained
• Ship a web game that loads fast — build-size and load-time tactics
• Cinevva Engine — our open-source, web-first engine built on Three.js

People line these two up as rivals, but they're not the same kind of tool. PlayCanvas is a full game engine with a visual editor. Three.js is a rendering library you build on with code. Picking between them is really a question of how much you want handed to you versus how much you want to control. Here's the breakdown for web 3D in 2026.

For the wider field, see our web game engines comparison.

Quick Verdict

Engine vs Library

This is the whole story in one line. PlayCanvas gives you a game engine: an entity-component system, physics, animation, audio, input, and an asset pipeline, all wired together, plus a browser-based editor where a team can build a scene together in real time (think collaborative editing, like a design tool for 3D).

Three.js gives you a renderer. Scene graph, cameras, lights, materials, and a fast WebGL/WebGPU renderer. No editor, no physics, no game loop opinions. You add physics with something like Rapier or cannon-es, structure your own game logic, and build whatever tooling you need. That's freedom and also more work.

So the real question isn't "which is better," it's "do I want an engine or a renderer?"

Rendering and WebGPU

Both are modern on the GPU front. PlayCanvas runs a dual WebGL2 and WebGPU backend. Its v2.0 release (August 2024) dropped WebGL1 entirely, and recent engine builds added a compute-based WebGPU path for 3D Gaussian splats. Three.js shipped a production-ready WebGPURenderer around r171 (September 2025) with near zero-config setup and an automatic WebGL2 fallback, plus TSL (Three.js Shading Language) so you can author shaders once and target both backends.

For most projects in 2026 either will render beautifully. WebGPU is the upgrade path on both, with WebGL2 as the safe baseline. See our web games tech stack guide for current browser support.

Workflow, Licensing, and Cost

PlayCanvas's editor is its headline feature: a hosted, collaborative scene editor with an asset pipeline. The engine is open source under MIT, and the editor frontend was open-sourced in 2025 too. The hosted editor has tiers, a free plan, a Personal plan around $15 a month, and an Organization plan around $50 per seat a month, mostly differing on storage and private project limits.

Three.js is MIT and free, full stop. There's no editor to pay for because there's no editor. Your "cost" is the engineering time to build the structure PlayCanvas hands you.

When to Pick Which

Pick PlayCanvas when you want a complete engine and a visual editor out of the box, your team needs to collaborate on scenes, and you'd rather configure than build from scratch. It's especially strong for product configurators, interactive 3D, and teams shipping quickly.

Pick Three.js when you want full control over rendering and architecture, you're comfortable assembling your own stack, or you're building something bespoke where an engine's opinions would get in the way. It's also the larger ecosystem and the most in-demand web-3D skill. We built Cinevva Engine on Three.js for exactly that reason: web-first, open, and no ceiling on what we can customize.

Common Questions

Is PlayCanvas or Three.js better for web games?

Neither is strictly better, they're different tools. PlayCanvas is a full engine with a visual editor, so it's faster to ship a structured game or interactive app, especially for a team. Three.js is a rendering library that gives you total control but expects you to build the game systems yourself. Choose PlayCanvas for turnkey, Three.js for custom.

Is Three.js a game engine?

No. Three.js is a 3D rendering library. It handles the scene graph, cameras, lights, materials, and rendering through WebGL2 or WebGPU, but it has no built-in physics, audio, input, or game loop. You add those with other libraries (like Rapier for physics) or build them yourself. PlayCanvas, by contrast, is a full engine.

Do PlayCanvas and Three.js support WebGPU?

Yes, both do in 2026. PlayCanvas runs a dual WebGL2/WebGPU backend. Three.js has a production-ready WebGPURenderer (stable since around r171) with an automatic WebGL2 fallback and TSL for cross-target shaders. Ship WebGL2 as your baseline and treat WebGPU as the performance upgrade.

Is PlayCanvas free?

The PlayCanvas engine is free and open source under the MIT license, and the editor frontend was open-sourced in 2025. The hosted cloud editor has a free tier plus paid plans (around $15/month Personal and $50/seat/month Organization) that mainly add storage and private projects. Three.js is entirely free with no editor to pay for.

Related

• Best Web Game Engines for 2026 (Compared) — the full field
• Three.js vs Babylon.js — if you've narrowed it to web-3D libraries
• Godot vs Unity for Web Games — full engines instead of libraries
• Web Games Tech Stack in 2026 — WebGL, WebGPU, and Wasm
• Cinevva Engine — our open-source engine built on Three.js

These are the two heavyweights of web 3D, and the choice between them comes down to philosophy. Three.js is a lean renderer you extend. Babylon.js is a full engine with the tools already in the box. Both are excellent, both are free, both do WebGPU in 2026. Here's how to pick.

For 2D engines or full game engines instead, see the web game engines comparison.

Quick Verdict

Library vs Framework

Three.js keeps a small core and leaves the rest to you and the community. You get a great renderer, scene graph, and materials, then reach for add-ons when you need physics, post-processing, or controls. It's the most popular web-3D library and has the largest ecosystem, which means more examples, more tutorials, and more hiring pool.

Babylon.js is a fuller engine, backed by Microsoft. It ships physics, a GUI system, animation, a built-in Inspector for debugging scenes, an online Playground for sharing code, and a Node Material Editor for building shaders visually. If you want tooling and structure provided rather than assembled, Babylon hands you more on day one.

WebGPU in 2026

Both are serious about WebGPU now. Babylon.js 8.0 (March 2025) made all core engine shaders ship in both GLSL and WGSL, which removed a multi-megabyte conversion layer and made the engine roughly half the size when targeting WebGPU. Babylon.js 9.0 (March 2026) pushed further with clustered lighting, volumetric lighting via compute shaders, a frame graph, and area lights.

Three.js has its production-ready WebGPURenderer with automatic WebGL2 fallback, and TSL lets you write shaders once for both backends. Recent releases keep refining it.

Net: WebGPU is solid in both. Babylon leans into compute-shader-driven features; Three.js keeps its renderer lean and flexible.

TypeScript, Bundle Size, and Ecosystem

Babylon.js is TypeScript-first, written almost entirely in TS, which many teams prefer for large codebases. Three.js is JavaScript with separate type definitions that work well but aren't the source of truth.

On size, Three.js has the smaller, more tree-shakeable core, so a minimal scene ships less code. Babylon is larger but modular through scoped packages, and its WGSL-native path narrowed the WebGPU gap.

Ecosystem goes to Three.js on raw size and community output. Tooling-in-the-box goes to Babylon.

When to Pick Which

Pick Three.js when you want a lightweight, flexible renderer, the biggest ecosystem, and freedom to architect things your way. It's the default for custom web-3D work and the most transferable skill.

Pick Babylon.js when you want a full engine with strong built-in tooling (Inspector, Playground, Node Material Editor), TypeScript-first development, and corporate-backed stability, and you're happy trading a bit of bundle size for batteries included.

Both are great picks. We chose Three.js for Cinevva Engine because we wanted a lean, web-first foundation we could shape ourselves.

Common Questions

Is Three.js or Babylon.js better?

Both are excellent and free. Three.js is a minimal renderer with the largest ecosystem, best when you want flexibility and to build your own structure. Babylon.js is a fuller engine with built-in physics, GUI, an Inspector, and a Playground, best when you want tooling provided and TypeScript-first development. It's a philosophy choice, not a quality gap.

Do Three.js and Babylon.js support WebGPU?

Yes, both support WebGPU in 2026. Three.js has a production-ready WebGPURenderer with WebGL2 fallback and TSL for cross-target shaders. Babylon.js 8.0 made its core shaders WGSL-native (cutting bundle size on WebGPU), and 9.0 added compute-shader features like volumetric lighting. Ship WebGL2 as the baseline and use WebGPU where it's available.

Is Babylon.js a game engine and Three.js not?

Roughly, yes. Babylon.js is a full engine: it includes physics, a GUI system, animation, an Inspector, and editor-style tooling. Three.js is a rendering library focused on drawing 3D, so you add physics, controls, and game systems through other packages or your own code. That's the core difference between them.

Which is smaller, Three.js or Babylon.js?

Three.js has the smaller, more tree-shakeable core, so a minimal scene ships less JavaScript. Babylon.js is larger overall but modular through scoped packages, and its WGSL-native shaders in 8.0 roughly halved its size when targeting WebGPU. For the tightest bundle, Three.js still wins.

Related

• PlayCanvas vs Three.js — add a visual editor to the comparison
• Best Web Game Engines for 2026 (Compared) — the full field
• Web Games Tech Stack in 2026 — WebGL, WebGPU, and Wasm
• Three.js + USDC in the Browser — loading USD 3D assets in Three.js
• Cinevva Engine — our open-source engine built on Three.js

Nobody opens a game and thinks "I want to apply a smooth falloff function with Hermite interpolation to a signed distance field." They think "I want to dig a cave" or "I want to build a mountain" or "I want to make a river."

The technical side of terrain sculpting (marching cubes, SDF brushes, transvoxel seams) is covered in our companion research guide. This guide is about the other half of the problem: how to put sculpting power into casual players' hands and make it feel like play, not like a 3D modeling tool.

The Metaphor Problem

Every shipped game that lets players reshape terrain has solved the same design problem: raw sculpting parameters (radius, strength, falloff, operation type) are meaningless to most players. The solution is always the same: wrap the brush in a metaphor the player already understands.

Minecraft: The Block as Universal Metaphor

Minecraft's genius is that terrain editing is just inventory management. You break a block. You get a block. You place a block. The "brush" is a single voxel, the "falloff" is binary (affected or not), and the "operation" is add or remove.

TNT scales this up. You craft it from gunpowder and sand (materials you already collected while playing), place it like any other block, light it with flint and steel, and it carves a sphere out of the terrain. The blast radius is 4-7 blocks. Blast resistance varies by material: dirt and sand disintegrate, stone partially survives, obsidian is immune. Water nullifies block destruction but not entity damage.

The player never thinks about blast radius as a parameter. They think "TNT is strong, obsidian is tough, water is a shield." The terrain editing system is fully expressed through materials and items the player already knows.

Valheim: The Farmer's Toolkit

Valheim takes a different approach. Instead of blocks, it gives players tools from pre-industrial farming:

The Hoe (5 wood + 2 stone, available in the first 10 minutes) has four modes: Level Ground (flattens to your standing height), Raise Ground (adds dirt, costs 2 stone per use), Path (removes grass), and Pave (lays cobblestone). The leveling mechanic is brilliant: you stand where you want the ground to be, and the hoe makes everything around you match that height. No numeric input. Your feet are the reference.

The Cultivator (5 bronze + 5 core wood, requires smelting) does planting: till soil, plant seeds, grow grass, and regrow vegetation. It doesn't shape terrain but it dresses it. After you flatten a hillside with the hoe, you cultivate it and plant a forest. The terrain tells a story: someone was here, they worked this land.

The Pickaxe handles the destructive side. It digs into terrain, the inverse of the hoe's raise. The three tools together (pickaxe digs, hoe raises/flattens, cultivator plants) form a complete vocabulary that maps to real-world farming actions.

The cost model matters. Raising ground costs stone. You feel the weight of each edit. You don't terraform a mountain for free. The resource cost makes sculpting a decision, not a checkbox.

Animal Crossing: Earned Terraforming

Animal Crossing: New Horizons gates terraforming behind a 3-star island rating. You need to build a campsite, invite villagers, construct housing plots, and develop your island for days (real-world days, not game-hours) before Tom Nook hands you the Island Designer app. By the time you unlock terraforming, you understand the island. You know where the rivers are, which cliffs block your favorite views, and what you want to change.

The tools themselves use a permit system. Paths are free. Waterscaping (creating rivers, ponds, waterfalls) costs 6,000 Nook Miles. Cliff Construction costs another 6,000. Additional path types cost 2,000 each. You're spending currency you earned through gameplay to expand your creative toolkit.

The editing is tile-based. One square at a time. No radius slider. No strength parameter. You push one tile of cliff up or dig one tile of river out. The constraint makes it approachable: the worst mistake you can make takes one button press to undo. The design encourages planning (the community built Happy Island Designer, a browser tool for pre-planning layouts) and incremental refinement over time.

Cliffs can go up to 3 levels. Rivers must connect to the ocean. Certain landmarks (airport, resident services) are immovable. These constraints aren't limitations. They're design. They prevent the "blank canvas paralysis" that kills casual engagement in freeform editors.

No Man's Sky: The Multi-Tool

No Man's Sky puts terrain manipulation on a gun. The Terrain Manipulator is a multi-tool upgrade with four modes: Mine (removes terrain in a sphere), Create (adds terrain in sphere or cube shapes), Restore (reverts to the original generated state), and Flatten (levels large areas).

The metaphor is a sci-fi device that fits in your hand. You point and shoot to edit the world. T and R keys change the sphere size between three presets (small, medium, large). There's no numeric radius. Three sizes. That's the whole UI.

The tool is introduced in the first mission. By the time you start exploring, you already know how to carve a cave or flatten a landing pad. The create mode lets you choose terrain types (rock, dirt, etc.), which maps to the splat-map texture blending under the hood.

Energy drain keeps sculpting in check. In Normal mode it's generous. In Survival mode, every edit costs meaningful resources. The same tool at two difficulty levels serves both casual sculptors and resource-conscious survivalists.

Astroneer: The Vacuum Gun

Astroneer takes the simplest possible metaphor: a vacuum cleaner for dirt. Point at terrain, hold the trigger, and it sucks up soil. Point at empty space and it blows soil back out. The terrain deforms in real time around your cursor.

Augments modify the behavior: the Alignment Mod makes surfaces flat relative to the planet's curve. The Boost Mod increases speed. The Wide Mod increases the area of effect. The Inhibitor Mod lets you collect resources without deforming terrain at all.

The genius is that terrain is simultaneously a resource. The soil you vacuum up fills canisters. The soil in canisters can be processed into materials at a Soil Centrifuge. Sculpting the world and gathering resources are the same action. You don't dig because you want a cave. You dig because you need soil to make compound. The cave is a side effect that becomes useful later.

Deep Rock Galactic: Mining as Teamwork

Deep Rock Galactic maps terrain destruction to class roles. The Driller has power drills mounted on their forearms that chew through rock at 2m wide by 2.2m tall by 1m deep per second. The drills overheat after sustained use (5.5 seconds to red, 8 seconds cooldown) and run on limited fuel. Every dwarf has a pickaxe for precision work.

The Driller's job is to make the path. Carve a tunnel from point A to point B through solid rock. The team follows. The terrain editing serves the mission objective (reach the extraction pod) rather than being a creative tool. But the side effect is that every cave system the team plays through is uniquely shaped by their decisions. No two runs look the same.

The co-op structure makes sculpting social. The Driller doesn't dig alone. The Scout lights up the cave with flares. The Engineer places platforms on walls the Driller exposed. The Gunner defends the tunnel entrance. The terrain editing is one verb in a shared sentence.

Teardown: Destruction as Puzzle

Teardown wraps terrain destruction in heist planning. You have unlimited time to prepare: smash walls, drive vehicles through buildings, place explosives, build ramps. Then the alarm triggers and you have 60 seconds to hit every objective and escape.

Every tool has a physical identity. The sledgehammer breaks things up close. The blowtorch cuts steel. Explosives blow out walls. The fire extinguisher clears fire. Vehicles crash through structures. The player doesn't "edit terrain" in the abstract. They swing a hammer at a wall and the wall breaks.

The physics make it feel real. Structures collapse when support is removed. Fire spreads. Smoke fills rooms. Debris falls. The feedback loop is immediate and physical: hit something, watch it break, see the consequences cascade. Casual players understand this because they've swung a hammer in real life (or wanted to).

Eco: Consequences as Game Design

Eco is the rare game where terrain editing has permanent, shared consequences. Cut too many trees and air quality drops. Mine too aggressively and you pollute the water table. Overhunt and species go extinct. The ecosystem is fully simulated.

Players can propose and vote on laws: "No logging within 200m of the river" or "Mining tax of 10 gold per cubic meter." The governance system makes terrain editing a political act. You can't just dig wherever you want. The community might stop you.

This is the extreme end of gamification: the sculpting system itself becomes the core game loop. The terrain is the shared resource that every player action affects. Sculpting isn't a creative tool. It's the central tension of the game.

Design Patterns for Cinevva

These games suggest a toolkit of patterns we can use. Not all at once. In layers, starting with the most accessible and building toward the most complex.

Pattern 1: Object-Based Sculpting (The TNT Pattern)

Don't give players a brush. Give them objects that happen to reshape terrain when used.

TNT / Dynamite: Player crafts or finds an explosive. Places it in the world. Lights the fuse (or throws it). A sphere of terrain gets subtracted. The radius depends on the explosive type. Small firecracker: 2m radius. Standard dynamite: 5m. Mega bomb (rare drop): 15m. The player is blowing things up, not "editing terrain."

Pickaxe / Shovel / Drill: Handheld tools with different characteristics. The pickaxe chips away at rock (small radius, fast, works on hard materials). The shovel scoops dirt (medium radius, fast on soft ground, useless on rock). The drill bores tunnels (narrow and deep, leaves smooth cylindrical holes). Each tool maps to a different SDF brush operation under the hood, but the player experiences them as distinct physical objects with distinct feedback.

Bucket / Watering Can: Pour water on terrain and it flows downhill, eroding a channel. The erosion brush follows the terrain gradient automatically. The player doesn't aim a brush. They pour water and watch the river form. Under the hood it's a smooth-lower operation biased by the terrain slope.

Seeds / Saplings: Plant them and terrain grows. A seed packet placed on flat ground sprouts into a small hill over time (over several seconds, with an animation). A tree sapling placed on bare rock generates a mound of soil around its base. "Growing" terrain is just the raise brush with a nature skin.

Terraform Beacon: Place a beacon and it slowly flattens everything within its radius to a target height over time. Good for building sites. The player places the object and walks away. The flatten brush runs automatically at low intensity until the ground matches the beacon height.

Pattern 2: Tool Progression (The Stardew Pattern)

Start players with weak tools that do small edits. Upgrade through gameplay.

Tier 1 (Free, immediate): Bare hands. Can pat down small bumps (micro-smooth, 0.5m radius). Can scoop up a handful of dirt (micro-lower, 0.3m radius). Negligible impact. Teaches the system exists.

Tier 2 (Crafted from basic materials): Stone shovel, wooden pickaxe. Usable for small edits. Flatten a campsite. Dig a shallow trench. 1-2m radius. This is where most casual players live.

Tier 3 (Requires exploration or trading): Metal tools. Larger radius (3-5m). New operations unlocked: raise terrain with the hoe, carve smooth curves with the chisel.

Tier 4 (Endgame / rare drops): Power tools, explosives, terraform devices. 10-20m radius. Cave boring. Mountain building. These are spectacle tools that reward long-term players.

The upgrade path doubles as a tutorial. Each tier introduces one new concept. By the time you reach Tier 4, you've internalized the vocabulary of terrain editing through incremental exposure, not a tutorial screen.

Pattern 3: Contextual Tools (The Valheim Pattern)

Match the tool to what the player is trying to do, not what the terrain needs.

Building mode: When the player is in building mode (placing walls, floors, roofs), the terrain auto-flattens under foundations. They don't need a separate flatten tool. The building system handles it. This is how Valheim's building works: place a floor, and the ground adjusts.

Farming mode: When planting crops, the cultivator tills the soil (changes the surface material) and levels small bumps. The player is farming, not terraforming. The terrain editing is a side effect of the agricultural action.

Mining mode: When harvesting resources from terrain (ore veins, clay deposits, mineral nodes), the extraction leaves a hole. The terrain editing is a side effect of resource gathering. Astroneer does this perfectly: vacuum up soil, get resources, leave a cave.

Combat mode: Explosions from weapons create craters. Fire burns away vegetation. Acid dissolves rock. Terrain damage is a side effect of combat. Deep Rock Galactic's drills work this way: they're a weapon and a terrain tool simultaneously.

Pattern 4: Social Sculpting (The Eco Pattern)

In a shared world, every edit is a social act. Design for that.

Visible authorship: When a player sculpts terrain, leave a subtle indicator of who did it. A faint glow. A player-colored tint on the edited surface. A small flag. Other players can see that someone was here and changed the landscape.

Collaborative projects: Terrain edits within a claimed area accumulate toward a shared goal. "Dig a river from the mountain to the lake." Progress bar visible to everyone in the area. When complete, the river fills with water. The sculpting was a group activity with a shared reward.

Approval zones: In protected areas, terrain edits require approval from the landowner or a vote from nearby players. This prevents griefing without disabling sculpting. The edit is previewed as a ghost (transparent, outlined) until approved.

Undo by consensus: Any player can propose reverting a terrain edit. If enough nearby players agree, the edit fades away over a few seconds. The restore mode from No Man's Sky, but social.

Pattern 5: Natural Forces (The Erosion Pattern)

Instead of direct brush strokes, let players trigger natural processes.

Rain summoner: Place a cloud above a hill and it rains. The rain erodes the hillside over time, cutting channels and softening edges. The player controls where the rain falls and how long it lasts. The erosion shader does the sculpting.

Earthquake device: Triggers a local earthquake that cracks the ground, creates fissures, and lowers the terrain in the affected area. The player chooses where. The physics simulation determines how.

Lava flow: Release lava from a point and it flows downhill, filling valleys and hardening into rock. New terrain created by a natural metaphor. Under the hood it's the raise brush following the terrain gradient with a rock material.

Wind sculptor: A fan or blowing device that pushes soft terrain (sand, dirt) in the wind direction. Over time, it creates dunes and smooth slopes. The smooth brush with a directional bias.

Freeze / Thaw: Freeze water in a river to create an ice bridge. Thaw a frozen hillside to trigger a mudslide. Temperature as a terrain editing parameter, expressed through weather items.

Pattern 6: Discovery and Surprise (The Infinite Craft Pattern)

Reward experimentation.

Combination effects: TNT near water creates a hot spring (crater + water fill + steam particles). Seeds planted in TNT craters grow faster (the crater has "fertile soil"). A frozen river that gets dynamited reveals a cave underneath.

Hidden materials: Digging deep enough reveals different terrain layers: topsoil, clay, stone, crystal, lava. Each layer has different properties and looks. Players who dig deeper see things nobody else has seen.

Fossils and artifacts: Random terrain edits occasionally unearth buried objects: fossils, ancient artifacts, treasure chests, rare seeds. The act of sculpting becomes an exploration mechanic. You don't just dig for the cave. You dig because you might find something.

Terrain memory: Edited terrain remembers what happened to it. An area that was blown up by TNT has scorch marks and charred edges. An area that was grown with seeds has richer soil color. An area that was eroded by rain has smooth, water-worn textures. The terrain tells its own history.

The Brush-to-Metaphor Mapping

Every metaphor the player interacts with maps to a technical brush operation from our sculpting pipeline. Here's the translation table:

Pickaxe / Shovel maps to the lower brush (subtract from heightmap or SDF) with small radius, high strength, sharp falloff. The tool animation triggers the same Chunk.applyBrush() call.

TNT / Explosives map to the lower brush with large radius, maximum strength, sphere falloff. A single explosive event applies one brush stroke centered on the detonation point. Blast resistance per material type maps to a strength multiplier on the brush.

Hoe / Flattener maps to the flatten brush with medium radius, smooth falloff. The target height is sampled from the player's foot position when the tool is activated, exactly like Valheim.

Seeds / Growth maps to the raise brush with small radius, smooth falloff, applied over several frames (animated growth). Each frame applies a small increment. The animation sells the metaphor while the brush handles the geometry.

Watering Can / Rain maps to the smooth brush with directional bias. The bias follows the terrain slope gradient from ChunkManager.getNormal(). Points below the brush center get lowered more than points above it, creating natural erosion channels.

Terraform Beacon maps to the flatten brush applied at low strength every frame until convergence. The beacon's world-space position sets the target height. The radius is the beacon's area of effect.

Drill maps to the lower brush projected along a ray direction instead of a sphere. The ray cast from the player's aim direction determines the brush center at each depth step, creating a cylindrical tunnel.

Progression and Economy

What Players Earn

Tool unlocks: Start with hands. Craft basic tools from common materials. Discover recipes for advanced tools through exploration, trading, or quest completion.

Capacity upgrades: Bigger explosives. Wider hoes. Faster drills. The brush radius and strength increase with tier. Upgrading your shovel from wood to iron doubles the radius.

Material types: New terrain materials unlock as players explore biomes. You can only "create" terrain types you've encountered. Visit a desert to unlock sand creation. Visit a volcano to unlock basite. The create mode from No Man's Sky, gated by exploration.

Recipes: Combination effects are discovered, not given. Plant a seed in a TNT crater and discover "Fertile Blast." Pour water on lava terrain and discover "Obsidian Formation." Each discovery is cataloged and shareable.

What It Costs

Resources: Every terrain edit consumes materials. Raising terrain costs dirt/stone/sand (the material being placed). Lowering terrain yields materials (which go to your inventory). Flattening redistributes. The resource flow keeps the economy connected to the terrain system.

Energy / Durability: Tools have durability or energy limits. The pickaxe breaks after N uses. The terraform beacon runs on fuel. This prevents infinite editing and creates natural session boundaries. Players take breaks to resupply.

Time: Some operations are instant (pickaxe swing). Others are slow (beacon flattening, seed growing, rain erosion). Slow operations reward patience and discourage spam. They also look better: watching a seed sprout into a hill is more satisfying than clicking a "raise" button.

What Others See

Edit footprint: Every terrain modification has a visual signature. TNT craters have dark, cracked edges. Hoe-flattened areas have a smooth, worked-earth texture. Grown terrain has rich, organic coloring. Rain-eroded channels have watermarks. Players can read the history of the landscape.

Creator tags: Hover over edited terrain to see who changed it and when. "Sculpted by @PlayerName, 2 hours ago." This creates pride of authorship and social accountability.

Before/After: A "time lapse" view that shows the terrain at any point in its edit history. Scrub a timeline slider to see the landscape transform. This is possible because the CSG edit tree from Phase 3 of the technical guide stores every operation in order.

What We Build First

Phase 1 of the technical architecture works with heightmap sculpting on the existing WebGL renderer. The casual tools we can ship on top of that:

1. Shovel (lower brush, small radius). The most basic tool. Click terrain, dig a hole. Yields dirt resources.
2. Hoe (flatten brush, medium radius). Stand where you want the ground, use the hoe, terrain levels to your feet.
3. TNT (lower brush, large radius, one-shot). Craft from materials, place, ignite, watch the boom. Yields rubble.
4. Seeds (raise brush, animated over frames). Plant on flat ground, watch terrain rise into a small hill, vegetation appears on top.
5. Watering Can (smooth brush with slope bias). Pour on slopes to erode them. Creates natural-looking channels.

Five tools. Five distinct metaphors. One underlying brush system. Each tool is a different configuration of (radius, strength, falloff, operation, animation) wrapped in an item the player crafts, equips, and uses through the same interaction model as the existing PlacementTool.

We want players to sculpt the world. Not place prefabs on a grid. Not toggle blocks on and off. Actually reshape the terrain: carve rivers, raise mountains, smooth cliffsides, dig caves. The kind of thing that ZBrush and Blender's sculpt mode do for artists, but running at 60fps inside a multiplayer browser game.

That's a hard engineering problem that spans data representation, mesh extraction, GPU compute, brush mathematics, and network synchronization. This guide documents everything we found.

The Two Worlds of Terrain Representation

Every sculpting system starts with a choice about how terrain data is stored. That choice determines what kinds of edits are possible, how fast they run, and how much memory they cost.

Heightmaps

A heightmap stores one height value per grid point. You can think of it as a grayscale image where brightness equals elevation. Our current world/client terrain works exactly this way: noise.ts generates height via FBM value noise, and each Chunk stores a Float32Array heightmap that gets projected onto a PlaneGeometry.

Heightmaps are fast. Sampling is a single array lookup with bilinear interpolation. LOD is trivial because you just reduce the grid resolution. Splat-based texture blending maps directly onto the UV grid. Physics collisions reduce to a height query.

The limitation is topology. A heightmap can only represent one height per (x, z) coordinate. No caves. No overhangs. No arches. No tunnels. If a player sculpts a cliff that folds back on itself, a heightmap can't store it. For terrain that's mostly rolling hills and mountains, this is fine. For freeform sculpting where players can dig into the ground, it's a dead end.

Volumetric (3D Scalar Fields)

The alternative is storing a value at every point in 3D space. If the value is negative inside solid material and positive outside (or vice versa), you have a Signed Distance Field (SDF). If the value is just a density (above some threshold is solid, below is empty), you have a density field.

Volumetric representations handle any topology. Caves, overhangs, floating islands, tunnels through mountains. The tradeoff is memory and complexity. A 256^3 grid with 32-bit floats costs 64 MB. A 512^3 grid costs 512 MB. And that's for a single chunk. You need sparse data structures (octrees, brick maps) to make this practical.

The mesh extraction step is also nontrivial. You can't just set vertex Y positions and be done. You need an algorithm that reads the scalar field and produces a triangle mesh that approximates the surface where the field crosses zero.

Mesh Extraction: Marching Cubes, Surface Nets, and Dual Contouring

Marching Cubes

Marching Cubes is the oldest and most widely implemented isosurface extraction algorithm. Published by Lorensen and Cline in 1987, it works by examining every cube in the voxel grid where each corner has a scalar value. If some corners are inside the surface (negative) and some are outside (positive), a triangle mesh patch gets placed inside that cube.

Each cube has 8 corners, each either inside or outside, producing 256 possible configurations (2^8). These reduce to 15 unique cases via symmetry. A lookup table maps each case to a set of triangles. Edge intersection points are found by linear interpolation along the edges where the sign changes.

Recent GPU implementations have made Marching Cubes fast enough for real-time sculpting. A 2025 implementation on UE5 assigns each GPU thread to one cube, processing thousands simultaneously. The key insight is that each cube's triangulation is independent of its neighbors, making the algorithm embarrassingly parallel.

MCHex (arxiv 2511.02064, 2025) extends Marching Cubes to adaptive hexahedral mesh generation with guaranteed positive Jacobian values, improving boundary approximation for simulation meshes.

rupMC achieves dozens of times faster performance than serial implementations and 4x faster than parallel DMC variants using CPU/GPU heterogeneous architectures.

The main limitation: Marching Cubes struggles with sharp features. A 90-degree edge gets rounded into a smooth curve. For terrain sculpting this is usually acceptable (natural terrain is mostly smooth), but for architectural features it's a problem.

Surface Nets

Surface Nets is a newer family of algorithms that produces smoother meshes from discrete scalar fields. Instead of placing vertices on cube edges (like Marching Cubes), Surface Nets places one vertex per cube that contains the surface, then connects adjacent vertices to form quads.

The result is naturally smoother. A 2024 paper (arxiv 2401.14906) demonstrated a high-performance parallel Surface Nets implementation that runs one to two orders of magnitude faster than sequential algorithms. The fast-surface-nets Rust crate generates roughly 20 million triangles per second on a single 2.5 GHz core using small lookup tables and SIMD acceleration.

bevy-sculpter (v0.18.0, January 2026) uses Surface Nets as its primary meshing strategy. The crate provides SDF-based volumetric sculpting with four brush types: hard CSG (instant add/remove), smooth continuous (for held input), blur (surface smoothing), and flatten (set to target height). It also includes SDF redistancing via the Fast Sweeping Method to restore proper signed distance field properties after edits.

Surface Nets are a good middle ground between Marching Cubes (simple, fast, but produces aliased meshes on binary data) and Dual Contouring (feature-preserving but complex).

Dual Contouring

Dual Contouring preserves sharp features that Marching Cubes and Surface Nets can't. It does this by using not just the sign of the field at each corner, but also the gradient (normal) at edge intersections. A QEF (Quadratic Error Function) minimization places the vertex inside each cell at the position that best satisfies all the edge intersection constraints.

The result: sharp edges and corners are preserved in the extracted mesh. A cube with perpendicular faces stays a cube.

The tradeoff is complexity. The QEF solve has inter-cell dependencies that make GPU parallelization hard. It can produce non-manifold meshes (edges shared by more than two polygons). And the implementation is more involved than Marching Cubes, though Johannes Jendersie notes that a working Dual Contouring implementation is about 200 lines of code versus 500+ for a robust Marching Cubes.

Cubical Marching Squares (CMS) has been proposed as a middle ground: intercell-independent (GPU-friendly) while still offering some feature preservation.

Which to Use

For player-facing terrain sculpting in a browser:

Surface Nets is the strongest candidate. It produces smooth meshes (natural-looking terrain) without the aliasing artifacts of Marching Cubes on binary data. It's fast enough for real-time re-meshing. And it's simpler to implement than Dual Contouring.

Marching Cubes remains a solid choice when GPU parallelism is the priority (every cube is independent) or when you need the widest library support. The WebGPU SDF Editor by Reinder Nijhoff (January 2026) implements both Marching Cubes and Surface Nets in its extraction pipeline, running entirely on the GPU.

Dual Contouring is best reserved for cases where architectural precision matters more than performance. Not ideal for real-time terrain sculpting in a browser context.

The Transvoxel Algorithm: Solving LOD Stitching

When voxel terrain is meshed at different resolutions (LOD0 near the player, LOD2 far away), cracks form at the boundaries. For heightmaps this is a simple problem: interpolate edge vertices to match the lower-resolution neighbor. Our current chunk.ts does exactly this in stitchEdge().

For volumetric terrain, the problem is much harder. A cave mouth at LOD0 might produce 30 triangles on the boundary. The same region at LOD1 might produce 8 triangles with a completely different topology. There's no simple way to linearly interpolate between them.

Eric Lengyel's Transvoxel Algorithm (2009) solves this with "transition cells." At the boundary between two LOD levels, the algorithm considers 9 high-resolution samples (instead of 8 cube corners), producing 512 possible configurations that fall into 73 equivalence classes. Each class maps to a predefined triangle pattern that perfectly fills the gap between the two resolutions.

The algorithm operates on local voxel data, so retriangulating a modified region is fast. This is critical for real-time sculpting: when a player edits terrain near an LOD boundary, only the transition cells need to be rebuilt.

A Rust implementation exists as the transvoxel crate. The original lookup tables are available at transvoxel.org.

Brush Mathematics

A sculpting brush is a function that modifies scalar field values within a radius around a target point. The mathematics are surprisingly similar across all implementations, from Blender to Unreal Engine to runtime game systems.

Falloff Functions

The brush falloff determines how the edit strength decreases from the center to the edge. Blender 5.1 defines these standard profiles:

Smooth: f(d) = 3d^2 - 2d^3 (Hermite interpolation, same smoothstep as our noise.ts)

Sphere: Strong at center with steep falloff near border. Approximated as f(d) = sqrt(1 - d^2).

Sharp: f(d) = (1 - d)^n with n > 2. Creates a fine point.

Linear: f(d) = 1 - d where d is normalized distance from center (0 at center, 1 at edge).

Constant: f(d) = 1 for d < 1, hard cutoff at the brush boundary.

Inverse Square: Hybrid between smooth and sphere for a natural "clay-like" feel.

In all cases, d = distance_to_center / brush_radius, clamped to [0, 1]. The falloff value multiplies the brush strength to produce the actual field modification at each point.

Falloff Space

Blender distinguishes between sphere falloff (distance computed in 3D world space) and projected falloff (distance computed in 2D screen space). Projected falloff means that two points that appear close on screen affect each other equally, even if they're at very different depths in world space. For terrain sculpting, 3D world space falloff is usually more intuitive.

Core Brush Operations

Raise/Lower (Displacement): Add or subtract from the scalar field within the brush radius, weighted by falloff. For heightmaps: height[i] += strength * falloff(d). For SDFs: sdf[i] -= strength * falloff(d) (subtracting makes material more solid, raising the surface).

Smooth (Laplacian): Replace each value with the average of its neighbors, weighted by falloff. This erases detail and reduces noise. The Laplacian filter samples a small kernel (3x3 for heightmaps, 3x3x3 for volumes) and blends toward the mean. HC (Humphrey's Classes) smoothing is a variant that preserves volume better than raw Laplacian.

Flatten: Set the field value to a target height (or distance in SDF space), blended by falloff. The target is usually sampled at the brush center when the stroke begins, then held constant. This creates flat plateaus.

Pinch/Inflate: Move vertices toward or away from the surface normal. In SDF space, this is equivalent to displacing along the gradient direction.

Grab: Translate a region of the field, as if you were pulling clay. The displacement vector is the mouse delta projected into world space, applied to field values within the radius.

Noise: Add procedural noise to the field within the brush radius. Useful for roughening smooth surfaces.

Stamp: Apply a 2D grayscale image as displacement, projecting it onto the surface under the cursor. Unreal Engine's Landscape tool supports this for terrain brushes.

Adaptive Tessellation

sculpt-3D (React + Three.js browser sculpting) implements adaptive tessellation: as the brush moves across the mesh, triangles near the brush center are subdivided to provide more vertices for deformation. This prevents the "low-poly stretch" problem where a coarse mesh gets distorted by sculpting. The subdivision uses symmetric splitting for uniform triangle quality.

For volumetric systems, adaptive tessellation isn't needed in the same way because the mesh is regenerated from the field. Instead, you can locally increase the voxel resolution near edits (adaptive octrees) for the same effect.

SDF Sculpting: The Dreams Approach

Media Molecule's Dreams (PS4, 2020) is the most ambitious in-game sculpting system ever shipped. Alex Evans presented the technical approach at SIGGRAPH 2015.

Representation

Dreams stores geometry as a compound SDF function in 83^3 fp16 volume texture blocks. Each sculpt is a list of 1 to 100,000 "edits," where each edit is a CSG operation (add, subtract, color) with a primitive shape (sphere, cube, cylinder, cone, ellipsoid, torus, etc.) and a blend mode.

Blend modes use soft-max and soft-min functions. A "soft" blend produces rounded transitions between primitives (like clay pushed together). A "hard" blend produces crisp boolean cuts. The blend radius is user-controllable.

Rendering

Dreams doesn't extract a triangle mesh. Instead, it renders directly from the SDF using a custom point-cloud renderer ("flecks"). Each fleck is a tiny disc oriented along the surface normal. The SDF is sampled to find surfaces, and flecks are distributed across them. This avoids the mesh extraction bottleneck entirely but requires a custom renderer.

For a Three.js/WebGL world, this approach isn't directly applicable. We'd need to extract meshes. But the CSG edit list concept is highly relevant for undo/redo and network synchronization.

Mike Turitzin's Dynamic SDF Engine (2026)

A game engine currently in development by Mike Turitzin uses dynamic SDFs as the core representation. The engine supports:

Detailed modifications during gameplay: Adding and removing matter smoothly or with sharp edges. Non-destructive changes like moving holes or creating temporary tunnels that disappear behind the player.

Brick maps and brick atlases for sparse caching. Instead of storing the entire SDF field in a dense 3D grid, the field is divided into "bricks" (small 3D tiles). Only bricks that contain the surface boundary are allocated. This dramatically reduces memory for scenes with mostly empty or solid space.

Geometry clipmaps (Losasso & Hoppe, SIGGRAPH 2004) for LOD. Nested regular grids at increasing resolution surround the camera position. The innermost grid has the finest resolution; outer grids are progressively coarser. This provides dramatic memory reduction while supporting vast spaces. Clipmaps update incrementally as the camera moves, making them efficient for streaming open worlds.

Physics and collision work directly against the SDF. Sphere-tracing (ray marching with the SDF distance as the step size) provides efficient raycasting. Collision detection uses the SDF gradient as the surface normal and the distance value as the penetration depth.

Teardown: Voxel Destruction at Scale

Teardown (Voxagon) represents the other end of the spectrum: every object in the world is a voxel volume that can be destroyed piece by piece.

Architecture

Objects are stored as voxel grids on regular spacing. The engine doesn't use Marching Cubes or SDFs for rendering. Instead, it ray-traces voxels directly using a modified DDA (Digital Differential Analyzer) algorithm in fragment shaders, built on OpenGL 3.3. Mipmaps form a dense octree structure for accelerating empty space traversal during ray intersection.

For each object, the engine rasterizes its oriented bounding box (OBB) and traces a ray through it to find voxel intersections. Only backfaces of the OBB are rendered, allowing the camera to clip into the bounding volume.

Destruction Synchronization (Multiplayer)

Teardown's March 2026 multiplayer update uses a semi-deterministic approach. Structural destruction (cutting holes, changing ownership, reconnecting joints) is handled via fixed-point integer math on a reliable network stream. All clients execute the same deterministic commands and arrive at the same world state. Non-structural changes (debris, particles) use unreliable state synchronization.

This is an important insight for our multiplayer world: terrain edits must be deterministic. If Player A sculpts a mountain, all clients must produce the same mesh from the same field data. The edit commands (brush position, radius, strength, operation type) should be the authoritative data, not the resulting mesh.

ALICE-SDF: Compression and CSG Trees

ALICE-SDF (Adaptive Lightweight Implicit Compression Engine, v1.3.0 March 2026) provides a Rust implementation of SDF-based spatial data with 10-1000x compression versus polygon meshes. It supports:

126 building blocks: 72 primitives, 24 operations, 7 transforms, and 23 modifiers. Smooth blending operations (union, subtraction, intersection), chamfer and stairs blends for hard-edge bevels and stepped CSG transitions.

CSG tree diff/patch for undo/redo and network synchronization. This is the key feature for multiplayer sculpting: instead of sending the entire field state, you send the structural diff between two CSG trees. The client applies the patch to reconstruct the new state. This is far more bandwidth-efficient than delta-compressing raw voxel data.

CSG tree optimization including identity transform removal, nested transform merging, and modifier demotion. This keeps the tree compact as edits accumulate.

Mesh generation via both Marching Cubes and Dual Contouring. Physics collision detection works directly against the SDF.

WebAssembly support enables browser integration. The engine is written in Rust with WASM bindings, making it a realistic option for a Three.js application.

WebGPU Compute for Terrain

WebGPU is now available across all major browsers as of late 2025. Chrome 113+, Edge 113+, Firefox 141+, and Safari 26+ all ship with it enabled. This opens up compute shader pipelines that were previously GPU-only.

Performance

WebGPU compute shaders generate terrain roughly 100x faster than CPU methods through massive parallelization. The GPU executes thousands of calculations simultaneously, and terrain generation is almost entirely parallel (each vertex/voxel is independent).

Work is organized into three tiers: Dispatch Level (workload distribution across the GPU), Workgroup Level (shared memory within a processing unit), and Thread Level (individual calculations). WGSL (WebGPU Shading Language) is the shader language.

Real-Time Terrain Sculpting Pipeline

A WebGPU sculpting pipeline would look like this:

1. Brush application (compute shader): Update the scalar field values within the brush radius. Each thread handles one voxel. Reads the brush parameters (position, radius, strength, falloff type, operation) from a uniform buffer and applies the modification.

2. Mesh extraction (compute shader): Run Surface Nets or Marching Cubes on the modified region. The WebGPU SDF Editor by Nijhoff implements this as a multi-stage pipeline: space partitioning across 16,384 cells, octree-based cell splitting, then surface extraction.

3. Vertex buffer update (GPU-side): Write the extracted vertices directly into a render buffer without roundtripping through CPU memory.

4. Normal computation (compute shader): Calculate vertex normals from the mesh or from the SDF gradient.

5. Render (standard pipeline): Draw the mesh with standard PBR materials.

Steps 1-4 can all run on the GPU without any data returning to JavaScript. The CPU only needs to send the brush parameters each frame.

The WebGPU SDF Editor

Reinder Nijhoff's WebGPU SDF Editor (January 2026) demonstrates this approach running in Chrome. It supports six primitives (cone, cylinder, capsule, torus, box, sphere), three blend operations (union, subtraction, intersection) with configurable smooth blending, and hierarchical scene graphs. Each primitive occupies 112 bytes in a single GPU buffer.

The rendering pipeline uses 1,024 shadow maps for ambient occlusion and temporal anti-aliasing. This runs at interactive framerates on high-end GPUs.

Heightmap Sculpting: The Simpler Path

If cave and overhang support isn't required, heightmap sculpting avoids the entire volumetric pipeline. This is how most shipped games handle terrain editing.

Runtime Implementation Pattern

Unity Runtime Terrain (JohannHotzel, January 2026) demonstrates the standard pattern:

1. Raycast from the camera through the mouse position to find the terrain hit point.
2. Map the hit point to heightmap coordinates.
3. Apply the brush to nearby heightmap values, weighted by falloff.
4. Update the mesh by setting vertex Y positions from the modified heightmap.
5. Rebuild the physics collider to match the new mesh.

For our Three.js terrain, steps 1-4 map directly onto the existing architecture. The Chunk class already stores heightmaps and builds meshes from them. Adding sculpting would mean:

• A raycasting system against chunk meshes (Three.js Raycaster)
• Brush application functions that modify chunk.heightmap values
• Mesh vertex updates (set Y positions, recalculate normals)
• Splat map recalculation for the affected region (so texture blending reflects the new slope/height)
• Network broadcast of the edit (brush position, radius, strength, operation) to other clients via the existing WebSocket protocol

The Clipmap Approach

Landow.dev describes a "wandering clipmap" for heightmap terrain: a single mesh with variable subdivision density that follows the player. Instead of chunking the heightmap into separate meshes with different LOD levels (what we do now), the clipmap is a continuous mesh that's dense near the camera and coarse at the edges.

This eliminates LOD stitching entirely. The mesh simply has more triangles where you need them and fewer where you don't. The tradeoff is that sculpting requires updating a single large mesh rather than individual chunks, which can be expensive for large edits.

Non-Destructive SDF Heightmaps

Landow.dev also describes a technique where the heightmap itself is generated from an SDF composition. Shape instances (spheres, boxes, noise functions) are composed in a compute shader using CSG operations, and the output is sampled as a heightmap. This provides non-destructive editing (you can move or delete any shape instance at any time) while keeping the simplicity of heightmap rendering.

This is a compelling hybrid: the data representation is volumetric (SDF CSG tree), but the rendering path is a standard heightmap mesh. You get undo/redo and network-friendly edit operations from the SDF side, and simple rendering and physics from the heightmap side. The limitation remains: no caves or overhangs.

Sparse Voxel Octrees for Large Worlds

Dense 3D grids don't scale. A world that's 1 km on each side at 0.5m resolution would need 8 billion voxels. Sparse Voxel Octrees (SVOs) solve this by recursively subdividing space and only allocating storage for octants that contain the surface boundary.

An SVO naturally provides hierarchical LOD: the tree depth at any point determines the effective resolution. Near the player, the tree is fully expanded (maximum detail). Far away, it's truncated at a coarser level.

For rendering, SVOs can be ray-traced directly (no mesh extraction needed). A GPU ray marcher intersects rays with axis-aligned boxes at each tree level, skipping empty subtrees entirely. This eliminates overdraw problems of chunk-based rendering and avoids greedy meshing artifacts.

The Vulkan-based SVO builder by AdamYuan demonstrates significant performance: 19ms build time for Crytek Sponza at 2^10 resolution on a GTX 1660 Ti.

For sculpting, SVO modification is efficient: only the leaf nodes within the brush radius need to be updated, and the tree structure handles varying resolution naturally. Adding detail where the player sculpts (splitting nodes to higher resolution) and removing detail where they smooth (merging nodes to lower resolution) falls out of the data structure.

The challenge for browser deployment is that WebGL doesn't support compute shaders, and while WebGPU does, the SVO construction and traversal algorithms are complex to implement in WGSL.

Network Synchronization for Multiplayer Sculpting

Our world already has multiplayer via Cloudflare Durable Objects (world-chunk-do.ts). Adding sculpting means synchronizing terrain modifications across all connected clients.

Delta Compression

Sending raw voxel data is expensive. A 2024 study from Oulu University achieved 2-8x payload improvement by combining delta-encoding with DEFLATE compression, packing voxel updates to less than one byte per voxel. The SDEC codec demonstrates bit-packed delta encoding producing 259-byte average packets versus 1114 bytes with generic serialization.

Operation-Based Sync (Recommended)

Instead of synchronizing field state, synchronize operations. Each sculpting action becomes a message:

interface TerrainEditMsg {
  t: MsgType.TerrainEdit
  brush: {
    position: [number, number, number]
    radius: number
    strength: number
    falloff: 'smooth' | 'linear' | 'sharp' | 'constant'
    operation: 'raise' | 'lower' | 'smooth' | 'flatten' | 'noise'
    targetHeight?: number
  }
}

The server broadcasts this to all clients, and each client applies the same deterministic brush operation to its local terrain data. This is the same approach Teardown uses for structural destruction: deterministic commands on a reliable stream.

ALICE-SDF's CSG tree diff/patch takes this further: instead of individual brush strokes, the diff represents the structural change to the entire CSG tree. This enables efficient undo/redo across the network (send the inverse patch) and late-joining clients can reconstruct the full world state by replaying the operation log.

Priority and Throttling

Terrain edits near connected players should be high-priority (immediate broadcast). Edits far from any player can be batched and sent at lower frequency. Enshrouded's voxel networking uses this pattern: 60Hz updates for player-proximate terrain, 10Hz for background regions.

ZSTD compression on the wire reduces packet sizes by up to 60% for terrain update messages.

Collaborative Sculpting: Concurrent Edits

When multiple players sculpt the same region simultaneously, you need conflict resolution. cSculpt (CNR Visual Computing Lab, 2016) solved this with a multiresolution merge algorithm. Each edit is represented at multiple scales, and concurrent overlapping edits are merged by blending their multiresolution representations.

For our purposes, a simpler approach works: last-write-wins with server ordering. The Durable Object timestamps each edit and broadcasts them in order. All clients apply edits in the same sequence. Since brush strokes are small, localized, and additive/subtractive, the visual result of slightly reordered concurrent edits is usually indistinguishable from the "correct" order.

INST-Sculpt: Neural SDF Editing (Research Frontier)

INST-Sculpt (arxiv 2502.02891, February 2025) enables stroke-based editing of neural SDFs. Users draw strokes on the surface, and the system deforms the underlying neural field along tubular neighborhoods around the stroke path. Custom brush profiles (configurable cross-sections) control the shape of the deformation.

This is interesting for AI-generated terrain: if the base world is represented as a neural SDF (a small neural network that maps 3D coordinates to signed distance), sculpting modifies the network weights rather than explicit voxel data. The representation is extremely compact (a few MB for an entire world) but evaluation is more expensive than a lookup table.

This is research-stage technology. The inference cost of neural SDFs on consumer hardware is too high for real-time game use today. But it's worth watching, especially as WebGPU shader capabilities improve and model inference gets faster.

World Creator 2026.3: Commercial Terrain State of the Art

World Creator (BiteTheBytes, March 2026) represents the commercial state of the art for terrain authoring tools. Version 2026.3 added GPU-based terrain generation with automatic terrain adaptation (terrain conforms to placed objects), camera-focused object distribution for LOD optimization, and real-world elevation data import (GeoTIFF, HGT, DTED).

Since then, World Creator 2026.4 (April 28, 2026) added mathematical expressions in numerical fields, terrain normal blending to seat hero objects into the surface, full decal support, and VRAM scaling that adjusts the maximum object count to available GPU memory. BiteTheBytes also shipped a free Community Edition that's feature-complete but export-disabled, effectively an unlimited trial.

Their approach uses GPU compute for all terrain operations: erosion simulation, river carving, texture painting. The brush tools are GPU-accelerated with real-time viewport feedback. This matches the WebGPU compute pipeline described above, running on desktop GPUs.

What We've Already Built: 24 Spikes and a Production World

The world/spikes/ directory contains 24 self-contained prototypes. These aren't toy demos. They're a progressive R&D pipeline where each spike solved a specific problem, benchmarked it against a target, and informed the next one. The sculpting system builds on all of them, not just the later volumetric ones.

The Production Heightmap Terrain (world/client/)

The live world uses a chunked heightmap system in Three.js WebGL:

• noise.ts generates terrain height via FBM value noise (5 octaves for hills, 4 for ridges, 3 for micro-detail) with a deterministic terrainHeight(wx, wz) function
• chunk.ts builds PlaneGeometry meshes from Float32Array heightmaps with 3 LOD levels (32/8/4 segments per 64-unit chunk), and places instanced trees/billboards with seeded random placement and per-object colliders
• chunk-manager.ts streams chunks in rings around the player (radius 1 at LOD0, radius 3 at LOD1, radius 6 at LOD2) with edge stitching via linear interpolation in stitchEdge(), and provides getHeight(), getNormal(), and resolveCollisions() for the physics layer
• terrain-material.ts does 4-layer splat-based texture blending (grass/rock/sand/dirt) via MeshStandardMaterial.onBeforeCompile with slope and height-driven weights, plus per-layer normal map blending
• character-controller.ts samples terrain height every frame for gravity, grounding, and slope rejection (max slope cos 50 degrees). Sculpting must feed modified heights into this system instantly or the player falls through edited terrain
• placement.ts already has a Raycaster hitting chunk meshes for the object placement tool. The brush tool should follow this exact pattern rather than building raycasting from scratch
• protocol.ts defines MessagePack-encoded messages for multiplayer sync via world-chunk-do.ts Durable Object, which currently handles PlayerState, PlaceObject, RemoveObject, and Snapshot messages. Terrain edits will need a new message type
• world-chunk-do.ts (Cloudflare Worker) persists placed objects to Durable Object storage and broadcasts to connected players at 50ms intervals. It has no concept of terrain modifications yet

Spikes 01-11: The Foundation Layer

These spikes validated the core systems that sculpting will depend on. Skipping them means missing constraints the sculpting system must respect.

Spike 01 (Terrain + Instancing): The first terrain prototype in Three.js. Established the PlaneGeometry + heightmap pattern and instanced object placement that chunk.ts still uses.

Spike 02 (Rapier Physics Worker): Rapier 3D running in a Web Worker with a ColliderDesc.heightfield() collider. Built a kinematic character controller with autostep, slope limits, and snap-to-ground. This spike proved that physics can run off the main thread against a heightfield. If we sculpt the terrain, the physics heightfield must be rebuilt or replaced with a trimesh collider for MC chunks.

Spike 05 (LLM Behaviors): Not directly terrain-related but established the JSON behavior schema for game objects. Relevant because sculpted terrain features could trigger behaviors (e.g., a carved river spawns water effects).

Spike 06 (Chunk Streaming): First chunk load/swap system with dynamic loading as the player moves. Established the pattern that chunk-manager.ts uses: colored regions that load and unload. Sculpting must preserve edit state when chunks are unloaded and reloaded.

Spike 07 (GPU Vegetation from Density Maps): Instanced grass and trees placed via density maps that sample terrain height and slope. Sculpting invalidates vegetation placement: if the terrain height changes, trees may end up floating or buried. The density map must be regenerated for edited chunks.

Spike 08 (Terrain Material Shader Cost): Benchmarked triplanar projection, normal maps, and 4-layer blending. Measured exact ms cost of each feature. Found that triplanar + normals + 4 layers stays within budget at 45+ FPS. This budget matters for sculpted terrain: if we add a 5th layer for "edited soil" or change the blending for carved surfaces, we know exactly how much headroom exists.

Spike 09 (CSM Shadows Budget): Cascaded shadow maps with 3 cascades at 1024^2 resolution. Measured shadow cost at ~1.5ms. Sculpted terrain changes shadow maps, but the cost is constant regardless of terrain shape.

Spike 10 (Geometry Clipmaps + Geomorphing): Nested clipmap rings with geomorphing between LOD levels to eliminate pop-in. The constant triangle count means predictable GPU cost. Geomorphing matters for sculpting: when the player sculpts near an LOD boundary, the morph between LOD levels must reflect the edit. If the edit only exists in the high-res ring, the geomorph target is wrong.

Spike 11 (Heightmap Chunk Streaming): More advanced chunk streaming with a visual grid showing loaded/loading/unloaded states per LOD level. Established the streaming budget: max chunks loaded per frame, prioritization of chunks that need LOD upgrades. Sculpting adds a new priority signal: chunks the player is actively editing should never be unloaded.

Spikes 12-14: WebGPU + Three.js Integration

Spike 12 (WebGPU Marching Cubes): The first volumetric spike. Four 64^3 SDF chunks with animated sphere caves, running entirely on the GPU. Uses raw WebGPU: compute pipelines for SDF evaluation, MC extraction with the Twinklebear case table (256 configurations, 16 entries each), atomic vertex counter, and indirect draw. Performance target was <4ms per chunk, <12ms for all 4. This validated that GPU MC is fast enough for real-time re-meshing in the browser. Every subsequent volumetric spike reuses the MC case table and WGSL shaders defined here.

Spike 13 (Reset Baseline from Spike 12): Ported Spike 12's raw WebGPU draw path to run inside Three.js's WebGPURenderer, accessing the backend's device directly. The render pipeline still uses raw WebGPU (drawIndirect with struct Vertex vec4+vec4). This proved that custom compute and Three.js scene rendering can coexist on the same GPU device.

Spike 14 (Three.js WebGPU Incremental Hardening): Replaced the raw render pipeline with Three.js StorageBufferAttribute for positions and normals. MC compute writes directly into these GPU-resident buffers. The drawIndirect buffer controls how many vertices Three.js draws. This is the pattern all subsequent spikes use: compute stays as raw WebGPU, rendering goes through Three.js scene graph. The Three.js version across these spikes progressed from 0.170.0 to 0.172.0 as the WebGPU backend stabilized.

Spike 15-17: Transvoxel LOD Stitching

Spike 15 (Transvoxel Seam Scaffold): Added the three-zone architecture: MC chunk (volumetric center), transition strip (seam between MC boundary and heightmap), and terrain ring (surrounding heightmap). All three share a material pass. The transition strip is a placeholder mesh at this stage, not real Transvoxel cells.

Spike 16 (Transvoxel +X Face with Shared Heightmap): Two critical breakthroughs in one spike. First, replaced the flat terrain plane in the SDF with a shared Perlin heightmap: a 257x257 Float32Array uploaded to the GPU as a storage buffer, sampled via bilinear interpolation in the SDF compute shader. The MC surface and the heightmap mesh now agree on the same ground truth. Second, implemented real Transvoxel transition cells for the +X face by fetching Eric Lengyel's reference data tables from GitHub (transitionCellClass, transitionVertexData, transitionCellData) and the npm transvoxel-data package. The CPU evaluates 9-sample transition cells (512 configurations, 73 equivalence classes), places vertices by interpolating SDF values at grid points, and handles winding inversion for mirrored cases.

Spike 17 (Dual MC 1x/2x LOD): Two MC chunks side by side at different resolutions. High-res: 62 cells with cell_scale=1.0. Low-res: 31 cells with cell_scale=2.0. The MC shader gained cell_scale and grid_points uniforms. Introduced transition_shrink: the low-res chunk's face-0 boundary vertices are pulled inward by 15% of cell_scale, creating a thin gap that Transvoxel transition cells fill without z-fighting. This is the LOD model the production system needs: close chunks at full resolution, far chunks at half resolution, Transvoxel at every boundary.

Spikes 18-21: Transvoxel Corner Cases and GPU Acceleration

These four spikes each solved a specific failure case in the Transvoxel implementation. Grouping them obscures the distinct problems.

Spike 18 (Heightmap 2:1 Seam): Applied Transvoxel to a pure heightmap boundary where one side is twice the resolution of the other. No MC involved. The seam between a 62-cell and 31-cell heightmap chunk is generated from Transvoxel transition tables with 15% low-face shrink. This validated that Transvoxel works for the heightmap-only case, not just MC.

Spike 19 (64/32/32/16 Corner Grid): The hardest stitching case: four chunks of different resolutions meeting at a corner point (64, 32, 32, and 16 cells). The seam system must generate transition cells along four edges (A-B, A-C, B-D, C-D) with correct winding for each direction. This spike proved that the Transvoxel tables handle the multi-resolution corner without custom-case logic.

Spike 20 (GPU Transvoxel Corner): Moved the Transvoxel transition cell generation to the GPU for the 64/32/32/16 corner layout. The CPU was a bottleneck when regenerating transition cells every frame for animated terrain. GPU compute generates the seam vertices in the same pass as the MC extraction.

Spike 21 (GPU MC + Transvoxel Corner): Combined full GPU MC extraction with GPU Transvoxel seam generation in a single compute dispatch sequence. Both MC chunks and all four seams are generated on the GPU, with vertex counts managed via atomic counters and drawn with drawIndirect. This is the complete GPU pipeline for multi-resolution volumetric terrain with seamless LOD transitions.

Spikes 22-24: The Hybrid Architecture

Spike 22 (Hybrid MC/Heightmap Policy): The key architecture spike. Chunks are heightmaps by default. When the animated deformation sphere intersects a chunk's AABB, that chunk switches to MC mode. The rest stay as static heightmap meshes. Layout: 64, 32/32, and 16-cell chunks at different resolutions. Transvoxel seams handle every boundary, including MC-to-heightmap transitions. The spike tracks MC chunk count vs HM chunk count and vertex overflow per frame.

Spike 23 (Policy-Driven Chunk Modes): Loaded as a patch on top of Spike 22. Added camera-distance hysteresis (chunks don't flicker between modes when the camera is near a threshold) and an edit mask (chunks that have been deformed stay in MC mode even if the deformation source moves away). This is the "sticky edit" behavior sculpting needs: once a player carves a cave, that chunk stays volumetric forever.

Spike 24 (Policy + Clipmap Rings): The most advanced spike. Combines the near-field policy system from Spike 23 with far-field geometry clipmap rings from Spike 10. Upgraded to Three.js 0.183.1. Near-field uses HM/MC hybrid with Transvoxel seams at 64/32/16 resolution. Far-field uses static-center clipmap rings that follow the camera. This is the complete terrain rendering architecture: chunked volumetric sculpting where needed, cheap clipmap terrain everywhere else.

Why Marching Cubes, Not Surface Nets

The external research section of this guide suggests Surface Nets as the strongest candidate for browser terrain sculpting. But every spike in the pipeline uses Marching Cubes. That's not an accident.

MC's defining advantage is embarrassing parallelism: each cube is completely independent. The WGSL compute shaders in Spikes 12-24 dispatch one thread per cube with zero inter-cell communication. Atomic counters handle vertex allocation. This maps perfectly to GPU workgroups.

Surface Nets places one vertex per surface-containing cell, then connects neighbors. That neighbor connectivity is an inter-cell dependency. The fast-surface-nets crate handles it on the CPU with careful iteration order. On the GPU, it requires either a two-pass approach (find vertices, then connect) or shared memory within workgroups. Both are possible in WebGPU but add complexity.

The practical recommendation: stay with Marching Cubes for the sculpting pipeline. It's proven in our codebase, the WGSL shaders exist and are benchmarked, and the Transvoxel seam system is built around MC's edge-based vertex placement. Surface Nets is worth revisiting if MC's aliasing on binary data becomes a visible problem, but for SDF terrain where values are smooth gradients, MC produces clean results.

Practical Architecture for Sculpting

The spike sequence solved the rendering pipeline. What remains is the brush system, cascading side effects through the game systems, and multiplayer synchronization. Here's the plan, building on every spike.

Phase 1: Heightmap Sculpting (Minimal Change, Maximum Reach)

Add brush tools that modify chunk heightmaps in the production world/client/ code. This works within the existing WebGL renderer and doesn't require WebGPU.

Brush input: Follow the PlacementTool pattern in placement.ts. It already has a Raycaster hitting chunkManager.getChunkMeshes() and tracking a ghost mesh at the hit point. A TerrainBrushTool would do the same raycast but modify the chunk heightmap instead of placing an object. The World.onMouseDown handler already dispatches based on tool state.

Chunk modification (Chunk.applyBrush): Map the brush world position to heightmap grid coordinates. For each grid point within the brush radius, compute the falloff-weighted displacement and add/subtract from the heightmap value. Then update the mesh: set vertex Y positions from the modified heightmap, recompute normals via central differencing (the same terrainHeight(wx +/- eps, wz) pattern already used in chunk.ts line 155-158), and regenerate the splat map via createSplatMap() in terrain-material.ts for the affected region so slope-driven texture blending updates.

Character controller: CharacterController.update() calls getHeight() every frame for grounding. ChunkManager.getHeight() delegates to Chunk.sampleHeight(), which reads from the chunk's heightmap Float32Array. Since we're modifying that array directly, the character controller picks up the change on the next frame with zero additional wiring.

Object invalidation: Tree instances in chunk.ts are placed by sampling terrainHeight() at spawn time. After sculpting, trees in the affected area may be at the wrong height. Phase 1 can defer this (trees float slightly on small edits). Phase 2 needs a chunk.invalidateObjects() that re-samples heights and rebuilds the instance matrices. Same for the colliders used in resolveCollisions().

Rapier physics (if integrated): Spike 02 proved heightfield colliders work. If Rapier is active, the heightfield collider must be rebuilt or patched for the modified chunk. Rapier's ColliderDesc.heightfield() takes a flat Float32Array, so it's a straight replacement.

Network sync: Add MsgType.TerrainEdit = 10 to protocol.ts:

interface TerrainEditMsg {
  t: MsgType.TerrainEdit
  cx: number
  cz: number
  brush: {
    wx: number
    wz: number
    radius: number
    strength: number
    falloff: number
    operation: number
  }
}

The WorldChunkDO broadcasts this to all clients and appends it to a per-chunk edit log stored in Durable Object storage. Late-joining clients receive the edit log in the Snapshot message and replay it to reconstruct the terrain state. All clients apply the same deterministic brush function, so they converge on the same heightmap.

Chunk unload/reload: Spike 06 and Spike 11 established the streaming pattern. When a chunk is unloaded and later reloaded, the edit log for that chunk must be replayed against the base procedural heightmap. The edit log is stored server-side (Durable Object) and included in the Snapshot message.

Phase 2: Volumetric Sculpting with the Spike 22-24 Architecture

Port the Spike 24 pipeline into the production world. When a player sculpts below the surface (carving a cave, digging a tunnel), the affected chunk transitions from heightmap mode to MC mode.

WebGPU renderer migration: Spikes 13-14 proved that Three.js's WebGPURenderer can host custom compute alongside the scene graph. The production world moves from WebGLRenderer to WebGPURenderer with StorageBufferAttribute for MC chunks. Fallback to the Phase 1 heightmap-only path when WebGPU is unavailable.

Per-chunk SDF allocation: Follow Spike 22's hybrid pattern. Each chunk starts as a heightmap. On the first volumetric brush stroke, allocate a 64^3 Float32Array, initialize it by sampling the heightmap (the SDF value at each point is world.y - heightmap_value), and switch to MC rendering. The policy system from Spike 23 ensures the chunk stays in MC mode permanently (the "sticky edit" behavior from the edit mask).

Shared heightmap in the SDF shader: Spike 16's height_at() function. Upload the chunk's heightmap to a GPU storage buffer. The SDF compute shader evaluates max(height_sdf, edit_sdf) where height_sdf = world.y - height_at(world.xz) and edit_sdf contains the brush modifications. MC and heightmap chunks agree on ground truth at their boundaries.

Transvoxel seams: The full stack from Spikes 15-21. MC-to-heightmap boundaries use transition cells with the shrink gap. MC-to-MC boundaries at different resolutions use Spike 17's dual-LOD pattern. The corner case from Spike 19 handles 4-way intersections. Spike 21's GPU compute generates all seam geometry in the same dispatch.

Clipmap far-field: Spike 24's clipmap rings for terrain outside the sculpting range. Sculpting never touches these rings; they sample the base procedural heightmap.

Geomorphing: Spike 10's geomorphing eliminates pop-in at LOD transitions. For edited chunks, the geomorph target must include the edit. If a chunk is MC at LOD0 and its LOD1 neighbor is a heightmap, the geomorph blends between the two representations. This requires sampling the edit log even at lower LODs.

Material budget: Spike 08 benchmarked 4-layer triplanar + normals at 45+ FPS. MC chunks need the same material. The splat map can be generated from the SDF gradient (steep = rock, flat = grass) instead of the heightmap slope. This stays within the 4-layer budget.

Vegetation invalidation: Spike 07's density-map vegetation depends on terrain height and slope. When a chunk transitions to MC mode, tree instances must be regenerated by sampling the SDF surface. Trees on overhangs or inside caves must be culled. The instanced mesh matrices from chunk.ts are rebuilt from the new surface.

Phase 3: CSG Edit Trees for Undo/Redo and Network Sync

Replace raw SDF mutation with a CSG operation tree. Each brush stroke appends a primitive (sphere, capsule, box) with an operation (add, subtract, smooth blend). The SDF is recomputed from the tree.

Benefits:

• Non-destructive: any edit can be removed from the tree to undo it
• Network-efficient: broadcast the CSG operation, not the raw field values
• Deterministic: all clients build the same SDF from the same operation sequence
• ALICE-SDF's CSG tree diff/patch provides bandwidth-efficient synchronization and undo/redo across the network

Durable Object storage: The edit tree per chunk replaces the flat edit log from Phase 1. The WorldChunkDO stores the CSG tree structure, not raw heightmap deltas. Snapshot messages include the tree, and late-joining clients evaluate it to produce the local SDF.

Phase 4: Collaborative Sculpting

Add concurrent edit support with server-ordered operation replay. The Durable Object timestamps each edit and broadcasts in order. Late-joining clients receive the operation log and reconstruct the world state. The existing Snapshot message type extends to include terrain edit history per chunk.

Since brush strokes are small, localized, and additive/subtractive, the visual result of slightly reordered concurrent edits is usually indistinguishable from the "correct" order. Last-write-wins with server ordering is sufficient. The Durable Object's tick() function (currently running at 50ms intervals for player state) adds terrain edit broadcasts to the same loop.

Key References

Algorithms:

• Lorensen & Cline, "Marching Cubes" (1987)
• Eric Lengyel, "Transvoxel Algorithm" (2009), transvoxel.org
• Losasso & Hoppe, "Geometry Clipmaps" (SIGGRAPH 2004)
• "A High-Performance SurfaceNets Discrete Isocontouring Algorithm" (arxiv 2401.14906, 2024)
• MCHex (arxiv 2511.02064, 2025)

Implementations:

• bevy-sculpter v0.18.0 (Rust, Surface Nets + SDF brushes)
• fast-surface-nets (Rust, 20M tri/sec)
• ALICE-SDF v1.3.0 (Rust + WASM, CSG tree diff/patch)
• WebGPU SDF Editor (Nijhoff, January 2026)
• SculptingPro (Unity runtime sculpting API)
• TerraBrush (Godot terrain sculpting GDExtension)

Games:

• Dreams (Media Molecule, SDF + point cloud rendering, SIGGRAPH 2015)
• Teardown (Voxagon, voxel DDA ray tracing, deterministic multiplayer destruction)
• Mike Turitzin's SDF engine (brick maps + geometry clipmaps, January 2026)

Network:

• "Optimizing payload size for voxel state synchronization" (Oulu, 2024)
• Teardown multiplayer (semi-deterministic destruction sync, March 2026)
• cSculpt (collaborative mesh sculpting with multiresolution merge)

Last updated: June 2026.

Generating a 10-second clip is easy now. Every major model does it. The real question is: can you generate 5 or 10 minutes of coherent video where a character looks the same in minute one and minute eight? Where the scene holds together across hundreds of frames?

That's the hard problem. And it's where things are shifting fast. This guide covers every model we've found that either generates long video natively or supports the workflows needed to build long-form content with consistent characters through reference images. We split them into three tiers: models that generate minutes-long video directly, models with strong reference image support that you extend through continuation, and open-source options you can run yourself.

Tier 1: Native Long-Form Generation (Minutes+)

These models generate video measured in minutes, not seconds. They're built from the ground up for temporal consistency over long sequences.

LongCat Video

Meituan released LongCat Video in late 2025. It's a 13.6 billion parameter diffusion transformer and it's the first model that can reliably generate coherent video up to 15 minutes long.

The model supports text-to-video, image-to-video, and video continuation in a unified pipeline. In I2V mode, the input image becomes the literal first frame of the video. It's not a loose character reference you can place in any scene. The model animates forward from that starting frame while using "Cross-Chunk Latent Stitching" to keep referencing the original image throughout generation, preventing color drift and maintaining visual consistency over long sequences. An updated 2026 variant adds audio-driven avatar generation with lip-sync for 5+ minute talking head videos.

Under the hood, LongCat uses a coarse-to-fine generation approach with Block Sparse Attention to handle the massive sequence lengths. RLHF tuning improves motion quality. On the VBench 2.0 benchmark it ranks third overall, behind Google Veo 3 and Shengshu's Vidu Q1.

Availability: Open source under MIT license. Available through fal.ai API at $0.04 per generated second ($36 for a 15-minute video at 720p). Also available through LongCat's own platform with credit-based pricing.

Seaweed APT2

ByteDance's Seaweed APT2 takes a different approach. Instead of generating a complete video upfront, it produces frames autoregressively at 24fps with just 0.16 seconds latency per frame on a single H100. The result is stable video up to 5 minutes with temporal consistency that holds.

The technical trick is Autoregressive Adversarial Post-Training (AAPT), which converts a pretrained bidirectional video diffusion model into a unidirectional autoregressive generator. Single network forward evaluation per frame. That's what makes real-time generation possible.

What makes this model interesting beyond raw length is interactivity. You can control the camera, animate characters through pose detection, and manipulate scenes while the video renders. Think of it less as "generate a video" and more as "steer a video in real time."

Availability: Research phase only. Not publicly available yet. The 7B base model (Seaweed-7B) has a published paper but the APT2 weights haven't been released.

Helios

Helios comes from Peking University, built on top of Wan 2.1. It's a 14B parameter model that generates minute-scale video at 19.5 FPS on a single H100. The key innovation is how it handles long-video drifting. Instead of using conventional anti-drifting techniques like self-forcing or keyframe sampling, Helios simulates drifting during training so the model learns to correct for it.

It natively supports text-to-video, image-to-video, and video-to-video tasks. The I2V mode accepts reference images to seed the generation.

Availability: Fully open source under Apache 2.0. Released March 2026. Code and weights on GitHub (PKU-YuanGroup/Helios). Integrated into Diffusers, SGLang, and vLLM-Omni. Gradio demo on HuggingFace Spaces.

SkyReels V2 / V3

Skywork's SkyReels line aims for infinite-length video. V2 uses an AutoRegressive Diffusion-Forcing architecture that generates video without a fixed duration cap. V3, released January 2026, unifies reference image-to-video, video-to-video extension, and audio-guided avatar generation in a single model.

V3 accepts 1 to 4 reference images and preserves subject identity across the generated video. The video-to-video mode enables seamless single-shot continuation and multi-shot switching with cinematographic transitions.

Skywork shipped SkyReels V4 on February 25, 2026, the first open-source model to generate video and audio together in a single dual-stream pass at up to 1080p/32fps. It accepts text, images, video clips, masks, and audio as conditioning inputs and unifies generation, inpainting, and editing in one framework, extending the V2/V3 reference-and-extend line with native sound. It now sits near the top of the Artificial Analysis text-to-video arena.

Availability: Fully open source. Models from 1.3B to 14B parameters. Available at 540p and 720p. Code and weights on GitHub and HuggingFace.

Tier 2: Short Clips with Strong Reference + Extension

These models generate 8-60 second clips but offer strong reference image support and video extension features. For long-form content, you chain clips together using the model's continuation or extension endpoints. Character consistency comes from reference images that persist across generations.

This is the practical workflow most creators use today for content longer than a minute. The quality per-clip is often higher than the native long-form models.

Kling 3.0 Omni (Kuaishou)

Kling has the most complete reference image system of any video model. It separates reference inputs into three distinct categories, each serving a different purpose:

Reference Images (image_urls): Up to 4 images for style and appearance guidance. You tag them in your prompt as @Image1, @Image2, etc. These influence the overall look, scene style, and environment without being the first frame.

Elements (elements): Dedicated character/object inputs. Each element takes a frontal_image_url (clear front-facing photo) plus optional reference_image_urls (additional angles). You reference them as @Element1, @Element2 in your prompt. The model extracts the character's identity and places them in any scene you describe. This is the key feature for adventure-movie-style content: upload a character photo, then describe them walking through a forest, fighting a dragon, whatever you want.

Start/End Frames (start_image_url, end_image_url): Pin specific images as the first or last frame. These are literal frames, not style guides.

The total across all three categories is up to 7 reference inputs (drops to 4 when also using a reference video). A single prompt like "@Element1 and @Element2 are having dinner at this table on @Image1" can combine characters with scene references.

For long-form content, Kling offers two paths. Multi-shot mode generates up to 6 scenes in a single call, each with its own prompt and duration (3-15s each). Character elements persist across all shots automatically. The extend API continues from where a completed video left off, reaching roughly 3 minutes through chained extensions. The V2V editing mode takes existing video (3-10 seconds) and transforms it using element references and a text prompt, preserving camera motion and character staging from the source while restyling characters and environments based on your references. This makes Kling particularly useful for enhancing pre-existing footage, including low-fidelity 3D renders.

Kling 3.0 Omni unifies text-to-video, image-to-video, reference-to-video, and video editing in a single model with native audio generation and lip-sync.

Availability: Commercial API through Kuaishou, fal.ai ($0.084-0.112/sec), and Replicate. Web interface at klingai.com.

Grok Imagine (xAI)

xAI launched Grok Imagine's Reference-to-Video mode in early 2026 with support for 1-7 reference images. The documentation explicitly distinguishes this from image-to-video: "Unlike image-to-video where the source image becomes the starting frame, reference images influence what appears in the video without locking in the first frame."

You tag images in your prompt as <IMAGE_1>, <IMAGE_2>, etc. A prompt like "the model from <IMAGE_1> walks onto the runway wearing the shirt from <IMAGE_2>" combines a person reference with a clothing reference. The model handles virtual try-on, product placement, and character-consistent storytelling across scenes.

One constraint: you can't combine reference images with image-to-video in the same request. It's either first-frame mode or reference mode, not both.

Grok Imagine also has a video extension endpoint that adds new footage to the end of an existing video. The duration parameter controls only the new portion. You can chain extensions to build longer content.

In June 2026, xAI released Grok Imagine 1.5 as an API preview (June 3, 2026). It's an image-to-video model that animates a single still into cinematic motion with camera moves, atmosphere, physics, and natively generated synchronized audio in the same inference pass. It supports multi-shot sequencing to chain consistent scenes and currently ranks among the top image-to-video models on the Artificial Analysis arena. Preview pricing is $0.08/sec at 480p and $0.14/sec at 720p, plus $0.01 per input image.

Availability: xAI API (launched January 2026), fal.ai, and Replicate. Python SDK, JavaScript/AI SDK, and REST API. $0.05/sec at 720p with audio. Also available to X Premium subscribers.

Seedance 2.0 (ByteDance)

ByteDance's Seedance 2.0 accepts the most reference inputs of any model: up to 12 files simultaneously, including up to 9 images, 3 videos, and 3 audio files. The model supports native audio-video generation with phoneme-level lip-sync in 8+ languages.

Individual images can be up to 30MB each. Reference videos must be 2-15 seconds. The model uses the references for character appearance, scene styling, and motion guidance.

Availability: ByteDance official API (via Volcengine, launched February 2026) and third-party API providers. Output at 480p-720p via API, up to 2K cinema resolution through the platform.

Runway Gen-4.5

Runway Gen-4.5 launched at the top of the Artificial Analysis Text-to-Video leaderboard with 1,247 ELO, ahead of Veo 3 and Sora 2 Pro at the time. By mid-2026 the arena had reshuffled (Seedance 2.0 now leads), but Gen-4.5 remains a top-tier model for cinematic quality and controllable action. The model generates 2-10 second clips for text-to-video and supports character-consistent long-form video up to one minute through multi-shot sequencing.

Image-to-video was added in January 2026 and supports reference images for all aspect ratios. The model integrates neural radiance fields and Gaussian splatting within the diffusion architecture, giving it 3D geometric understanding rather than pixel-level prediction alone. This means better object permanence and physically plausible motion.

Availability: Commercial API and web interface. SDKs for Node and Python. Also available on Replicate.

Google Veo 3.1

Google's Veo 3.1 generates 4, 6, or 8 second clips natively. The "Extend Video" feature (currently in preview) chains clips to reach approximately 1-2.5 minutes, though coherence can drift on longer sequences.

The "Ingredients to Video" feature accepts up to 3 reference images as input. You can provide characters to animate, backgrounds, and material textures. When you use reference images, the model sticks closer to your visual references and makes fewer random alterations. One limitation: reference image mode only works with the 8-second duration option.

As of January 2026, Veo 3.1 added vertical video (9:16) for reference-based generation and 4K upscaling on Vertex AI.

Availability: Google Vertex AI API, Gemini API, and Google Flow. Requires Google Cloud account.

OpenAI Sora 2 / Sora 2 Pro

Sora 2 Pro generates clips up to 20 seconds. The Characters API uses a different approach from Kling or Grok: instead of uploading static images, you create a character_id by pointing the API at a video clip (with a 1-3 second timestamp range). Sora analyzes the video frames to extract facial structure, body proportions, clothing style, and other identifying features. That character_id persists indefinitely and can be reused across unlimited future generations.

You can reference up to 2 uploaded characters per generation. As of March 2026, character references work for objects and animals too, not just people. Video extension uses the full initial clip as context for continuation.

The character system requires video input (not static images) to create characters. If you only have photos, you'd need to generate a short video first, then extract the character from that.

Availability: OpenAI API with Batch API support for production workflows.

MiniMax Hailuo 02

Hailuo 02 ranks #2 globally on the Artificial Analysis benchmark, beating Veo 3. It generates 10-second clips at native 1080p with some of the best physics simulation in the field. The model handles extreme motion like gymnastics and acrobatics without breaking apart.

It supports image-to-video generation with strong character consistency through facial recognition and body tracking. The Noise-aware Compute Redistribution architecture dynamically allocates compute based on scene complexity.

MiniMax has since moved past Hailuo 02 with the Hailuo 2.3 family (Standard, Pro, Fast, Fast Pro), which improves physical action, stylization, and character micro-expressions. It outputs 1080p at 6 seconds or 768p at 10 seconds and is available through the MiniMax platform and fal.ai.

Availability: Commercial API. Available through MiniMax platform, fal.ai, and Replicate. $0.28 per video.

Luma Ray2

Ray2 generates 5-10 second clips at up to 1080p with 4K upscaling available. The Extend feature continues videos up to 30 seconds total. Image-to-video accepts reference images as start or end keyframes.

The model is trained on a multi-modal architecture with 10x the compute of Ray1. It handles photorealistic content well but the 30-second extension cap limits long-form use.

Availability: Luma API and web interface.

Pika 2.5

Pika takes a keyframe-based approach with Pikaframes. Upload 2-5 keyframes (reference images at key moments) and the model generates smooth transitions between them. Total duration reaches 20-25 seconds.

Pikascenes accepts up to 10 reference images and combines them into a single video. The model uses image recognition to figure out each reference's role (character, background, prop) automatically.

Availability: Pika web platform and API. Subscription plans from free to Pro.

Tier 3: Open-Source Models for Self-Hosted Workflows

These models generate shorter clips but they're fully open. You can run them on your own hardware, fine-tune them, and build custom extension pipelines without API dependencies.

Wan 2.1 (Alibaba)

Wan 2.1 is the foundation several other models build on (including Helios). The Wan-VAE architecture encodes and decodes 1080p video of any length while preserving temporal information. The model comes in I2V variants at 480p and 720p, plus a First-Last-Frame-to-Video model that generates video between two reference images.

Wan-Edit allows style and content transfer using reference images while maintaining specific structures or character poses.

HunyuanVideo (Tencent)

Tencent's 13B parameter model was the open-source video generation leader through most of 2025. HunyuanVideo-I2V uses a token replace technique with a pre-trained MLLM to incorporate reference image information. HunyuanVideo-1.5, released November 2025, improved efficiency. HunyuanCustom enables multimodal-driven customized video generation.

CogVideoX (Tsinghua/Zhipu AI)

CogVideoX uses a 3D causal VAE that reduces sequence length and prevents flickering. The adaptive LayerNorm transformer improves text-video alignment. Available in 2B (Apache 2.0) and 5B (research license) variants with native Diffusers integration.

Clips are 6-10 seconds at 720x480. Short, but the quality-to-compute ratio is good and it runs on a 12GB GPU.

First-Frame vs. True Reference: The Key Distinction

Not all "reference image" support is the same. Understanding the difference is critical for choosing the right model.

First-frame models (LongCat, Helios, Hailuo, Luma Ray2, HunyuanVideo) treat your image as the literal opening frame. The model animates forward from that exact visual. You can't upload a character headshot and describe them in a different scene. The image is the scene.

True reference models (Kling, Grok Imagine, Seedance, SkyReels V3) extract identity from your image and place that character/object into any scene you describe. Upload a photo of a person, then prompt "that person walks through a forest at sunset." The character appears in a completely new environment while maintaining their identity. This is what you need for multi-scene narrative content like an adventure movie.

Character ID models (Sora 2 Pro) extract identity from video clips rather than static images. You create a persistent character ID once and reuse it across unlimited future generations.

Style/ingredient models (Veo 3.1) use reference images to influence visual style, textures, and overall look rather than extracting specific character identities. Good for maintaining visual consistency across a project, less precise for individual character control.

The Real Workflow for 10-Minute Videos

Here's the honest take on where things stand in mid-2026. No single model reliably generates 10 minutes of consistent, high-quality video in one shot. LongCat Video gets closest with claims of 15 minutes, but quality and coherence vary significantly at those lengths. Helios and SkyReels V2 generate "minute-scale" and "infinite-length" video respectively, but the outputs need careful prompting and often multiple attempts.

The workflow that actually works for most creators building 5-15 minute videos combines multiple approaches:

For talking head / avatar content: LongCat Video's 2026 audio-driven mode or SkyReels V3's avatar generation can produce 5+ minutes of a consistent talking character. This is the closest thing to "press a button, get long video."

For narrative content with multiple scenes (adventure movie style): Use Kling 3.0, Grok Imagine, or Seedance 2.0 with true character reference images. Generate individual shots of 10-15 seconds each. Use the same @Element or <IMAGE> references across every generation to maintain character identity. Chain shots together using multi-shot mode (Kling supports 6 shots per call) or the extend API. Kling is the most battle-tested for this workflow. Grok Imagine's explicit separation between "reference mode" and "first-frame mode" makes it a strong alternative. Seedance 2.0 accepts the most reference inputs (12 files) but is newer and less proven.

For character consistency across many clips: Sora 2 Pro's persistent character_id system is the cleanest approach for very long projects. Extract the character once from a short video, then generate dozens of clips referencing that ID. The character identity doesn't degrade over time because it's stored as a persistent embedding, not re-interpreted from an image each time.

For style-transferred content: Lucy Restyle on fal.ai processes existing video up to 30 minutes, applying AI style transformations while preserving motion. If you have source footage, this sidesteps the generation length problem entirely. $0.01 per second of source video.

For open-source pipelines: Build on Wan 2.1 or Helios with a video continuation loop. Generate a clip, use the last frame as the start frame for the next clip, repeat. ComfyUI workflows automate this. Consistency degrades over many iterations but it's free and controllable.

The core challenge remains: even with true reference image support, character drift compounds across dozens of clips. Facial features, hair, clothing, and skin tone gradually shift. The workarounds (high-quality reference photos, consistent prompting, shot batching) are necessary. But models like Kling and Grok Imagine that separate character identity from scene composition make this dramatically easier than the first-frame-only models.

The 3D Scaffold Approach: Render Low, Transform High

There's a workflow gaining traction that sidesteps most long-form generation problems entirely. Instead of asking an AI model to generate 10 minutes of video from scratch, you render a low-fidelity 3D cutscene with correct camera work, character blocking, and timing, then run it through a video-to-video model with reference images and an enhancement prompt. The 3D engine handles structure. The AI handles aesthetics.

This works because V2V transformation is a narrower problem than full generation. The model doesn't need to invent camera motion, character placement, or scene composition. It just needs to make existing footage look photorealistic while following your visual references. That's far more tractable, and it scales to any length your 3D engine can render.

Why This Works

Your 3D engine gives you everything AI video models still struggle with: precise camera control, exact character placement across the frame, correct physics interactions, and consistent timing across minutes of footage. Dolly zooms, tracking shots, characters entering and exiting frame on cue, all trivial in a 3D engine, all unreliable in text-prompted generation. The V2V model's only job is transforming materials, lighting, and textures into photorealistic output while preserving the geometry and motion you already defined.

Character consistency gets easier too. Instead of fighting identity drift across 50 separate AI generations, you're showing the model the same 3D character in every frame. The reference images tell the model what that character should look like in the final output. That's a simpler problem than generating a consistent character from scratch each time.

And length is no longer a constraint. Lucy Restyle handles 30 minutes in a single call. Wan 2.1 in ComfyUI can process any length in chunks. You're not fighting the "how do I generate 10 minutes" problem at all because the footage already exists.

RealMaster (Meta / Tel Aviv University)

RealMaster is a research system built specifically for this workflow. Published March 2026 by Meta Reality Labs and Tel Aviv University, it transforms rendered 3D video into photorealistic video while maintaining full geometric alignment with the source.

The method extracts edge maps from the 3D render to preserve structure and motion, then applies a video diffusion model (built on the VACE/Wan architecture) to transform everything else into photorealistic output. A lightweight IC-LoRA adapter distills the pipeline into a single inference pass that doesn't need anchor frames and handles objects appearing mid-sequence.

Tested on GTA-V and CARLA simulator sequences, RealMaster significantly outperforms general-purpose video editing baselines. It can also layer weather effects through the text prompt ("Make it rain", "Make it snow") on top of the realism transformation. The model generalizes across simulators without retraining. Weights trained on GTA-V data work on CARLA output with no additional tuning.

Availability: Research only. No public weights or API yet.

Production V2V Tools Available Today

Kling 3.0 V2V with Element References is the most complete production option. The Edit Video and Reference V2V endpoints on fal.ai accept 3-10 second source video clips alongside element references (@Element1, @Element2 with frontal and multi-angle photos) and an enhancement prompt. The model analyzes motion trajectories and camera patterns in the source, then regenerates the footage with your specified character appearances and visual style while preserving the original staging and camera work. Up to 7 reference inputs. Output at 1080p. Process your cutscene in 10-15 second chunks with the same element references across all chunks to maintain character consistency.

Lucy Restyle 2 handles up to 30 minutes of source video in a single API call at $0.01 per second of input. It accepts a text prompt and an optional reference image for style guidance. No per-character element references like Kling, but for overall cinematic style transfer of a full-length 3D render it's the simplest and cheapest path. Feed it your complete render and a prompt describing the target look. Output at 720p with temporal consistency across thousands of frames.

Wan 2.1 VACE in ComfyUI is the open-source route. The 14B VACE model does reference-driven V2V: input a source video plus a style reference image, output a restyled version that preserves structure and motion. Edge map conditioning improves structural fidelity. You can build a processing loop that handles any length in chunks with consistent style references. Free, runs locally on your own hardware.

Grok Imagine V2V accepts source video plus 1-7 reference images in reference mode. $0.05 per second at 720p. The explicit separation between reference mode and first-frame mode means your references guide character appearance without overriding the source video's structure.

What Your 3D Render Needs

The render quality floor matters. A bare wireframe won't give the V2V model enough to work with. But you don't need production-quality materials or lighting either.

Correct proportions and geometry. Character models need roughly correct body proportions and facial structure for the reference images to map properly. Basic humanoid geometry with correct proportions is enough. A stick figure won't produce a recognizable character.

Basic lighting direction. A single directional light establishing the scene's overall illumination helps the model understand the intended mood. The AI will enhance and add detail, but it needs to know whether the scene is bright daylight or dark interior.

Smooth camera motion. Stable, deliberate camera moves translate well. Erratic or extremely fast motion can confuse V2V models. Keep your virtual camera behaving like a real camera would.

Flat shading over wireframe. Simple flat-shaded or low-poly geometry gives better results than wireframes or untextured models. Even basic solid colors on surfaces help the model understand material boundaries.

Cost and Scale

Processing a 10-minute cutscene through different tools:

Lucy Restyle is the cheapest for full-length processing. Kling is the most precise for character-specific enhancement with element references. Wan 2.1 is free if you have the hardware (needs about 60GB VRAM for the 14B model at 720p, or 8GB for the 1.3B variant at lower quality).

Comparison Table

What's Coming

The trajectory through 2026 is clear. LongCat Video proved that minute-scale generation with consistency is possible in an open model. Helios showed it can happen in real-time. Seaweed APT2 demonstrated interactive long-form generation. And the true-reference models (Kling, Grok, Seedance) proved that character identity can persist across arbitrary scenes.

The next step is combining these capabilities: native long-form generation with true character reference support. Right now you pick one or the other. When a model can generate 5 minutes of video while maintaining characters from reference images across dozens of scene changes, the chained-clips workflow becomes obsolete.

As of mid-2026, the Artificial Analysis text-to-video arena (with audio) is led by Seedance 2.0 (~1,215 Elo), followed by HappyHorse-1.0, SkyReels V4, and several Kling 3.0 variants, with Veo 3.1 and Sora 2 close behind. The leaderboard reshuffles frequently, so treat any single ranking as a snapshot rather than a fixed order.

The 3D scaffold approach offers a parallel trajectory. As V2V models improve their structural preservation and photorealism, enhancing low-fidelity 3D renders becomes increasingly viable for full production. RealMaster from Meta already achieves research-quality sim-to-real transformation on game engine output. When this capability hits production APIs with reference image support, anyone with basic 3D skills will be able to produce photorealistic long-form video with complete control over camera, staging, and character placement at any duration.

For now, the practical answer depends on your use case:

Best for multi-character reference: Kling 3.0 (up to 7 refs with separate element + style system) or Seedance 2.0 (up to 12 multimodal inputs).

Best API for reference-to-video: Grok Imagine (clean API, explicit reference mode, $0.05/sec) or Kling via fal.ai ($0.084-0.112/sec).

Best for persistent characters across many clips: Sora 2 Pro (character ID system, no drift over time).

Best open source: SkyReels V3 (1-4 true reference images, unlimited length) or Helios (real-time, Apache 2.0).

Best for raw duration: LongCat Video (~15 min, but first-frame only).

Best for 3D render enhancement: Kling 3.0 V2V (per-character element refs, 1080p) or Lucy Restyle 2 (30 min input, $0.01/sec).

Common Questions

What is the best AI video model for long videos?

For raw duration, LongCat Video generates around 15 minutes natively, though it's first-frame only. For long video with consistent characters, the practical answer in 2026 is a reference-and-extend workflow: generate clips with a model that has strong reference support (Kling 3.0, Runway Gen-4.5, or open-source SkyReels V3), then chain them. No single model both runs long and holds character identity perfectly, so most production work combines them.

Which AI video models support reference images?

Kling 3.0 Omni, Runway Gen-4.5, Seedance 2.0, and Google Veo 3.1 all support reference images among the commercial options. On the open-source side, SkyReels V2/V3 and Wan 2.1 accept reference inputs you can run yourself. Support quality varies a lot, which is why the guide above splits them into tiers.

Can AI generate a consistent character across a long video?

Yes, but not in one shot. The reliable approach is to lock a character with one or more reference images, generate short clips, and extend or stitch them while feeding the same references back in. True reference support (the model keeps identity across new generations) matters far more here than first-frame conditioning, which only seeds the opening frame.

What is the difference between first-frame and true reference image support?

First-frame conditioning uses your image as the literal opening frame of the clip, then drifts as the video progresses. True reference support treats the image as an identity anchor the model honors throughout the generation, so a character or style stays consistent across the whole clip and across separate clips. The section above breaks down which models do which.

More Reading

• Frontier Open-Source Gen AI Models — practical guide to open-source generative AI for video, image, 3D, audio, and more
• Video Generator — our video generation tool powered by Kling 3.0 Pro
• How to go from sketch to animated 3D character — using image and video generation for character animation

Heightmaps with Perlin noise are the default starting point for procedural terrain. Layer some octaves of fBm, apply a color gradient, and you have something that looks terrain-ish. Every tutorial ends here. But real landscapes don't look like stacked noise. They have river valleys carved by water, cliff faces shaped by stress fractures, caves and arches formed by erosion over millions of years. They have correlated textures (grass on flat ground, rock on steep slopes, snow above the treeline) that emerge from the same physical processes that shaped the geometry.

This guide covers what comes after noise. Physically grounded generation methods, volumetric representations that handle caves and overhangs, diffusion-based neural terrain synthesis, and the GPU-driven LOD and streaming pipelines needed to render it all in a browser tab at 60fps.

Everything here targets our specific constraint: a multiplayer open world running in WebGL 2 / WebGPU, streamed over the network, editable by creators.

Why Heightmaps Aren't Enough

A heightmap stores one height value per grid point. It's a 2D function: given (x, z), return y. This representation is compact, GPU-friendly, and fast to render. But it has fundamental limitations that matter for a creator world.

No caves or overhangs. A heightmap can't represent terrain where one point has two different heights. Caves, arches, cliff overhangs, tunnels, and floating islands are all impossible. Minecraft, No Man's Sky, and Deep Rock Galactic all need volumetric terrain for this reason.

No vertical features. A vertical cliff face in a heightmap is a near-infinite slope, which creates extreme texture stretching and collision artifacts. Real cliffs have horizontal features (ledges, crevices) that a heightmap can't represent.

Noise looks like noise. Even with 8 octaves of fBm and domain warping, the terrain has a synthetic quality. It lacks the directional structure of real geology: ridge lines, drainage networks, sediment deposits, tectonic folds. These patterns emerge from physical processes, not from noise functions.

Creator edits are limited. If creators can only modify heights, they can't dig tunnels, create caves, or build underground spaces. For a world that creators can truly shape, the terrain representation needs to support subtraction as well as addition.

The solution isn't to abandon heightmaps entirely. They remain the best representation for the 90% of terrain that is a simple surface. The solution is a hybrid approach: heightmap base terrain with volumetric overlays where complex geometry is needed, physically correct generation for realistic landforms, and neural synthesis for the diversity that noise can't achieve.

The Quick Answer: What We'd Actually Build

Before the deep dive, here's the decision framework. The rest of the article explains each piece.

Terrain representation

Use a hybrid heightmap + SDF system. The heightmap covers the entire world (cheap, compact, proven). SDF volumes exist only where caves, overhangs, or creator-carved features require them (maybe 5-10% of chunks). This keeps 90% of the world at heightmap cost while supporting arbitrary geometry where needed.

Generation pipeline

Run server-side as a chained process:

1. Terrain Diffusion (or MESA for text prompts) generates the base heightmap from a seed. This replaces noise with geologically realistic landforms trained on real elevation data.
2. Analytical erosion (stream power law) refines the heightmap with river networks and ridgelines in milliseconds.
3. TerraFusion or Geodiffussr generates correlated textures from the heightmap (or use procedural slope/altitude rules for the WebGL 2 fallback).
4. Ecosystem simulation produces vegetation density maps.
5. Arenite-style erosion generates SDF volumes for cliff faces, arches, and caves where the terrain calls for them.
6. Chunk, compress, upload to CDN. Average chunk: 2-8 KB. Complex chunk with volumetric data: 20-100 KB.

Creators interact with this pipeline by adjusting parameters ("wetter," "more mountainous," "add caves"), sketching intent, or directly sculpting with brushes and SDF tools.

LOD strategy

Both paths use geomorphing in the vertex shader for pop-free transitions. Both use billboard impostors for distant vegetation. The WebGPU path is faster (3.5ms terrain budget) but the WebGL 2 path is viable (6.5ms).

Streaming

Progressive loading: terrain geometry first (<100ms), textures second (<300ms), vegetation third (<1s), volumetric data fourth (<3s). Pre-fetch based on player velocity. Memory budget: 256 MB total terrain.

Editing

Heightmap brushes for surface sculpting (raise, lower, smooth, erode). SDF primitives for volumetric editing (carve caves, add arches). Both are instant locally, sync to server in 100-300ms, broadcast to other players via delta updates.

What to build in what order

Month 1-2: Heightmap terrain with geometry clipmaps. WebGL 2 only. Streaming from CDN. Procedural slope/altitude materials. This gets terrain on screen in every browser. Use noise + analytical erosion for generation (Terrain Diffusion can wait).

Month 3-4: Vegetation and atmosphere. GPU-instanced grass and trees from density maps. Procedural sky with day/night cycle. Atmospheric fog. Cascaded shadow maps. The world starts to feel like a place.

Month 5-6: Creator editing. Heightmap brush tools. Delta-based sync for multiplayer terrain edits. Spatial locking for concurrent editing. This is when creators start shaping the world.

Month 7-9: Volumetric terrain and WebGPU path. SDF overlays for caves and overhangs. Marching cubes in WebGPU compute. Transvoxel for LOD boundaries. SDF sculpting tools. This unlocks the full creation toolkit.

Month 10-12: Neural generation and polish. Terrain Diffusion or MESA for the base heightmap. TerraFusion/Geodiffussr for textures. Phasor noise for micro-detail. Hex-tiling for anti-repetition. Virtual texturing. Laplacian blending. The terrain reaches production quality.

This ordering means something playable exists after 2 months, something beautiful after 4, something editable after 6, and something state-of-the-art after 12.

The rest of this article is the research behind each decision.

Terrain Representations Beyond Heightmaps

Signed Distance Fields (SDFs)

A signed distance field stores, at every point in 3D space, the distance to the nearest surface. Positive values are outside, negative values are inside, and the zero-crossing is the surface itself. SDFs represent arbitrary 3D shapes including caves, arches, and floating geometry.

To render an SDF as a mesh, you run marching cubes (or a variant) to extract the zero-crossing as triangles. The mesh resolution depends on the grid resolution: a 256x256x256 SDF grid produces terrain covering roughly one chunk at 1-meter resolution.

Browser implementation: WebGPU marching cubes runs entirely on the GPU via compute shaders. Will Usher's webgpu-marching-cubes implementation processes a 256^3 grid in real time in the browser, achieving native-speed performance. The algorithm is embarrassingly parallel (each cell processes independently), making it ideal for GPU compute.

@compute @workgroup_size(4, 4, 4)
fn marchingCubes(@builtin(global_invocation_id) id: vec3<u32>) {
    let sdfValues = sampleSDF(id);
    let caseIndex = classifyCell(sdfValues);
    if (caseIndex == 0u || caseIndex == 255u) { return; }
    let triangles = lookupTriangulation(caseIndex);
    let vertices = interpolateEdges(sdfValues, triangles);
    appendToMeshBuffer(vertices);
}

Storage cost: A 256^3 SDF at 8-bit quantized precision is 16 MB uncompressed. But most of the volume is empty (far from the surface). Run-length encoding or sparse octree storage typically reduces this to 100-500 KB per chunk, comparable to heightmap terrain.

Editing: SDF terrain is naturally editable. Adding material is a min() operation on the distance field. Removing material (digging) is a max() with a negated shape. Smooth blending between shapes uses smoothMin(). These operations run in a compute shader at interactive rates.

Dual Contouring

Marching cubes places vertices on grid edges, which produces smooth surfaces but loses sharp features (cliff edges, rock corners). Dual contouring places one vertex per cell at the position that best represents the surface within that cell, preserving sharp edges and corners.

The algorithm needs both the distance field values and the surface normals (the gradient of the SDF) at each grid point. It solves a small least-squares problem per cell to find the optimal vertex position. The result is a mesh that captures both smooth terrain and sharp rocky features.

Neural Dual Contouring (Chen et al., 2022, arXiv:2202.01999) replaces the least-squares solver with a neural network that predicts optimal vertex positions and edge crossings. It achieves better surface reconstruction accuracy and feature preservation, particularly for complex natural rock formations.

The Transvoxel Algorithm

The hardest problem in volumetric terrain isn't generating the mesh. It's LOD transitions. When a high-resolution chunk sits next to a low-resolution chunk, the meshes don't line up at the boundary, creating visible cracks.

The Transvoxel algorithm, designed by Eric Lengyel (transvoxel.org), solves this by inserting special transition cells along boundaries between resolutions. These cells bridge the resolution difference with additional triangles that exactly match both sides. The algorithm reduces the complex boundary problem to 73 equivalence classes (compared to ~1.2 million cases for a brute-force approach).

Transvoxel is specifically designed for real-time applications where voxel data changes dynamically (creator edits, erosion, mining). It's patent-free and has been used in shipped games (Space Engineers, Astroneer). For a browser world with editable volumetric terrain, Transvoxel is the LOD solution.

Hybrid: Heightmap Base + Volumetric Overlays

The practical approach for a browser open world is a layered system:

Layer 1: Heightmap terrain covers the entire world. This is the cheap, compact representation for rolling hills, valleys, and mountains. It streams as small heightmap patches per chunk (2-4 KB each). Rendering uses geometry clipmaps with constant GPU cost.

Layer 2: Volumetric overlays exist only in chunks where complex geometry is needed. Caves, cliff faces, arches, creator-carved tunnels, and underground spaces are stored as sparse SDF volumes. Only chunks with volumetric data incur the SDF storage and marching cubes cost.

Layer 3: Creator modifications are stored as SDF edits on top of the base layers. A creator who digs a tunnel stores the tunnel's SDF shape. The rendering system combines the heightmap surface with volumetric subtractions and additions to produce the final mesh.

This hybrid costs almost nothing for flat terrain (just the heightmap), and scales only where complexity exists. In a typical world, maybe 5-10% of chunks need volumetric data.

Physically Correct Terrain Generation

Noise produces random terrain. Physics produces realistic terrain. The difference is visible: noise terrain has no structure (it's randomly bumpy everywhere), while physical terrain has river networks, ridge lines, alluvial fans, and cliff bands that emerge from erosion and tectonics.

Hydraulic Erosion: The Foundation

Water flows downhill, picks up sediment, and deposits it when it slows down. This single process, simulated over millions of virtual years, transforms featureless noise into terrain with recognizable geological features.

The particle-based approach drops simulated raindrops on the heightmap. Each drop flows downhill (following the gradient), erodes material based on speed and slope, carries sediment as dissolved load, and deposits sediment when velocity decreases or capacity is exceeded. After 200,000-500,000 particles, the terrain develops:

• River valleys that follow the path of maximum water flow
• Ridge lines that separate drainage basins
• Alluvial fans where steep valleys open into flat plains
• V-shaped valleys in mountainous terrain, U-shaped in glaciated terrain

GPU implementation: The particle simulation is parallelizable. Each particle is independent (the approximation ignores particle-particle interaction, which is fine for erosion). A WebGPU compute shader processes 10,000 particles per frame at 60fps, completing 200,000 particles in about 3 seconds of real time.

Sebastian Lague's open-source implementation (GitHub) is the standard starting point. It runs on a single thread in C# and processes a 1024x1024 heightmap in a few seconds. The GPU version is 50-100x faster.

Thermal Erosion

Water isn't the only erosion force. Temperature changes cause rock to crack and crumble (thermal weathering). When the slope between two terrain points exceeds a material's angle of repose, material falls from the higher point to the lower one. This produces:

• Talus slopes at the base of cliffs (piles of fallen rock)
• Softened ridge lines over time
• Material-dependent profiles (hard rock maintains steep slopes, soft soil collapses to gentle angles)

Thermal erosion is simpler than hydraulic erosion. It's a local operation: for each cell, compare the height difference with neighbors. If the slope exceeds the threshold, move material downhill. It runs as a single compute shader pass per iteration and converges in 50-100 iterations.

Combining hydraulic and thermal erosion produces terrain that looks dramatically more natural than either alone. Water carves valleys; thermal erosion softens the ridges between them and fills the valley floors with debris.

The Stream Power Law: Analytical Erosion

Recent research offers an alternative to particle-based simulation. The stream power law (a geomorphological equation relating erosion rate to drainage area and slope) can be solved analytically rather than simulated iteratively.

Cordonnier et al. (2024, HAL) combine the analytical stream power law with landslide and hillslope diffusion processes. The result is terrain generation that's physically grounded but runs as a mathematical function rather than a temporal simulation. You input a noise-based heightmap and parameters (rainfall rate, rock hardness, tectonic uplift rate) and get an eroded terrain in milliseconds.

This analytical approach is ideal for a browser world because it runs once server-side during terrain generation, not iteratively. The parameters are tunable by creators ("make this region more mountainous" adjusts the uplift rate; "make it wetter" increases rainfall and deepens valleys).

Arenite: Multi-Physics Erosion (SIGGRAPH 2025)

Arenite (Project page) is a physics-based sandstone simulator that generates arches, alcoves, hoodoos, and buttes from simple initial conditions. Users paint erodability maps (soft rock vs. hard rock layers) and vegetation, then the system simulates:

• Stress distribution through the rock column
• Wind erosion that preferentially removes exposed soft material
• Fluvial erosion from water flow
• Particle deposition that creates new formations

The GPU implementation runs in under 5 minutes for complex formations on a desktop GPU. While this is too slow for real-time browser use, it's fast enough for server-side generation. A creator could define a cliff face with soft/hard rock layers and get a realistic arch formation within minutes.

The output is a 3D voxel field, which fits directly into our SDF/marching cubes pipeline.

Flexible Erosion Across Representations

A 2024 paper from IRIT-STORM ("Flexible Terrain Erosion," Springer) solves a practical problem: most erosion methods only work on heightfields. If your terrain uses voxels, SDFs, or layered materials, you need separate erosion code for each.

The flexible erosion method decomposes erosion into two independent processes: terrain alteration (removing material from the surface) and material transport (moving sediment with particles governed by basic physics). Each particle has configurable size, density, restitution, and sediment capacity. An optional vector field controls particle motion for realistic fluid dynamics.

Because the particles interact with the terrain through a unified material alteration interface, the same simulation works on heightfields, voxel grids, implicit surfaces, and layered material stacks. For our hybrid terrain (heightmap base + SDF overlays), this means one erosion system handles both representations. The particle simulation runs in parallel on the GPU.

River Networks and Drainage Basins

Erosion carves rivers, but generating convincing river networks requires more than just running water downhill. Two approaches produce better results:

Drainage-first generation (Amit Patel, Red Blob Games, Project) builds the river network before assigning elevation. Start with a graph (Voronoi or triangle mesh). Classify edges as ridges (no flow), entries (water flows in), or outlets (water flows out). This creates realistic drainage hierarchies where rivers converge from small tributaries into major waterways, following the Rosgen classification system for different river types (braided channels in flat terrain, narrow gorges in mountains).

Flow accumulation tracks how much water passes through each terrain cell. Drop simulated rain uniformly, flow it downhill, and count visits per cell. Cells with high accumulation are river channels. Cells with moderate accumulation are seasonal streams. The accumulation map also drives erosion intensity (more water = more erosion) and vegetation distribution (river banks are wetter, supporting different plant species).

For a creator world, the river network is generated server-side during terrain creation and stored as a 2D flow direction map plus a water accumulation map per chunk. The browser client uses these maps to render water surfaces (flat planes at the correct height in river channels) and to drive the vegetation distribution (lusher near water).

Coastal and Shoreline Generation

Coastlines are where terrain meets water, and they have distinctive features that standard erosion doesn't produce: sea cliffs, beach deposits, tidal flats, sea stacks, and wave-cut platforms.

NEWTS1.0 (2024, MIT) models rocky coastline evolution using two erosion mechanisms: uniform retreat (constant erosion rate) and wave-driven erosion (erosion rate as a function of fetch distance and incident wave angle). The model runs over thousands of simulated years and produces headlands, bays, sea stacks, and arches that match real coastal geomorphology.

For a browser world, coastal features would be pre-generated during world creation. The parameters (dominant wave direction, rock hardness variation along the coast) let creators control the character of their coastline. A "Norwegian fjord" setting produces steep-sided inlets. A "tropical atoll" setting produces low-lying sandy shores with lagoons.

Cave and Underground Generation

Caves require fully volumetric terrain (SDFs or voxels) since heightmaps can't represent enclosed spaces. The generation approaches:

3D noise thresholding is the simplest method. Sample 3D Perlin or simplex noise at each voxel. Values below a threshold are solid, above are empty. Adjust the threshold and noise parameters to control tunnel diameter, connectivity, and chamber size. This produces organic, worm-like cave systems reminiscent of Minecraft.

PLUME (Procedural Layer Underground Modeling Engine) (2024, arXiv:2508.20926) generates realistic cave and lava tube environments using layered procedural rules. Originally built for space exploration research (simulating Martian lava tubes), it produces geologically plausible underground structures with stalactites, columns, and chamber systems.

L-system tunnels with metaball carving use an L-system grammar to grow branching tunnel paths through the terrain, then carve the actual tunnel geometry using metaball implicit surfaces. The metaballs produce smooth, rounded cave walls. Multiple passes with different parameters create primary passages, side chambers, and narrow connecting tunnels.

For a creator world, cave generation ties into the SDF overlay system. The base terrain is a heightmap (no caves). When a chunk needs caves (either from procedural generation or creator design), an SDF volume is generated that subtracts cave geometry from the base terrain. The marching cubes pipeline renders the combined surface.

Vegetation as a Physical Process

Vegetation in games is typically placed procedurally with rules like "grass below 2000m, trees below 1500m, snow above 3000m." This is fast but produces uniform, unrealistic distribution.

Physically grounded vegetation simulation models each plant as competing for resources (light, water, soil nutrients). The simulation:

1. Seeds are scattered across the terrain
2. Each plant grows based on available resources (water flows from the hydraulic erosion simulation, sunlight depends on slope and aspect, soil depth depends on erosion history)
3. Plants compete: trees shade out grass, dense canopy prevents new seedlings
4. Over simulated time, biomes emerge naturally: forests in valleys with water, sparse vegetation on windswept ridges, wetlands where water pools

Deussen et al.'s ecosystem simulation (Paper) generates forest distributions that match real-world ecological patterns. The simulation runs on a 2D grid (one cell per terrain chunk) and produces density maps and species assignments that the rendering system uses for GPU instanced vegetation placement.

For a browser world, the vegetation simulation runs once during world generation (server-side). The output is a set of density maps per chunk: tree density, grass density, flower density, rock debris density. The browser client uses these maps with GPU instancing to scatter vegetation at runtime.

Diffusion-Based Terrain Synthesis

This is where the field is moving fastest. Diffusion models (the same technology behind Stable Diffusion for images) are being applied to terrain generation, and the results are substantially more realistic than noise-based methods.

Terrain Diffusion: A Successor to Perlin Noise

Terrain Diffusion (Goslin, 2025, arXiv:2512.08309, Project page) is the most significant advance in procedural terrain generation since Perlin noise in 1985. The work has since been accepted to SIGGRAPH 2026, and the canonical arXiv title is now "InfiniteDiffusion: Bridging Learned Fidelity and Procedural Utility for Open-World Terrain Generation." The InfiniteDiffusion algorithm and the Terrain Diffusion framework are the two halves of the same paper.

The key innovation is InfiniteDiffusion, an algorithm that reformulates diffusion sampling for unbounded domains. Traditional diffusion models generate fixed-size outputs (e.g., a 512x512 heightmap). InfiniteDiffusion generates terrain of infinite extent with:

• Seed consistency: Same seed always produces the same terrain, like Perlin noise
• Constant-time random access: You can query the height at any point without generating neighboring regions first
• No boundary artifacts: Infinite generation without visible seams or repetition

The system uses a hierarchical stack of diffusion models. The top level captures planetary-scale features (continents, mountain ranges). Each subsequent level adds finer detail (individual peaks, valleys, small-scale roughness). A compact Laplacian encoding stabilizes outputs across the enormous dynamic range from sea level to Himalayan peaks.

Performance: Generation keeps pace with real-time exploration. The paper reports that even at the theoretical extreme of orbital velocity (about 7,700 m/s), terrain synthesis runs 9x faster than traversal on a consumer GPU. The project includes a Minecraft integration demonstrating real-time terrain synthesis.

Why this matters for us: Terrain Diffusion produces landscapes trained on real-world elevation data (Earth's actual topography). The output has river networks, mountain ranges, coastal features, and plateau structures that emerge from the training data, not from hand-tuned noise parameters. A creator could say "generate terrain like the Scottish Highlands" and the diffusion model would produce something with the right geological character.

The model runs server-side during world generation. The output is a standard heightmap that streams to the browser like any other terrain data. The generation method is invisible to the client.

TerraFusion: Joint Geometry and Texture

TerraFusion (2025, arXiv:2505.04050) goes further by jointly generating heightmaps and terrain textures. The key insight: terrain geometry and surface appearance are correlated (river beds are sandy, cliff faces are rocky, flat areas are grassy). Generating them separately produces mismatches.

TerraFusion uses a latent diffusion model with separate VAEs for heightmaps and textures, trained to model their joint distribution. The system supports:

• Unconditional generation: Random plausible terrain with matching textures
• Sketch-conditioned generation: A creator draws a rough map (valleys here, ridges there, cliffs along this edge) and the model generates detailed geometry and textures that match the sketch

For a creator world, this is powerful. A creator sketches the general layout of their plot, and the system fills in geologically plausible terrain with appropriate surface materials. No heightmap painting, no texture splatting, no manual material assignment.

MESA: Text-to-Terrain

MESA (2025, arXiv:2504.07210, CVPR 2025 Workshop) generates terrain from text descriptions. It's trained on global remote sensing data from the Copernicus program, so it has exposure to every type of terrestrial landscape.

A prompt like "a fjord with steep granite walls opening to a rocky coastline" produces a heightmap with the appropriate geological structure. "Rolling farmland with gentle hills and a wide river valley" produces something completely different.

MESA introduces the Major TOM Core-DEM extension dataset, which pairs satellite imagery with digital elevation models globally. This training data gives the model understanding of how real terrain looks at every scale and in every climate zone.

Geodiffussr: Text-Guided Terrain Texturing

Geodiffussr (2025, arXiv:2511.23029) takes an existing heightmap and generates textures guided by text descriptions while respecting the elevation data. "Autumn forest" produces orange and gold foliage on moderate slopes with bare rock on steep faces. "Tropical coast" produces palm vegetation on low ground with coral sand at sea level.

The system uses multi-scale content aggregation to ensure texture assignments respect elevation: snow only appears above a physically plausible altitude, water features sit in depressions, vegetation thins with altitude.

For a creator world, this means terrain textures can be regenerated from a text prompt without changing the geometry. A creator sculpts the terrain they want, then describes the mood ("dark volcanic wasteland" or "lush temperate forest") and the system generates appropriate surface materials.

Dynamic LOD for Browser Terrain

Rendering large terrain at full resolution everywhere is impossible in a browser. A 4 km x 4 km world at 1-meter resolution is 16 million vertices of terrain alone. Dynamic LOD reduces this to a constant, manageable vertex count regardless of world size.

Geometry Clipmaps

Geometry clipmaps (Losasso and Hoppe, SIGGRAPH 2004, Paper) remain the gold standard for heightmap terrain LOD. The idea: render terrain as a set of concentric square rings centered on the camera. Each ring is twice the area of the previous one but at half the resolution.

Close to the camera (innermost ring): full resolution, 1-meter grid spacing. One ring out: 2-meter spacing, covering 4x the area. Next ring: 4-meter spacing, covering 16x the area. And so on for 6-8 levels, until the outermost ring covers the entire visible distance.

The total vertex count is constant: roughly N^2 * levels, where N is the ring width in vertices. With N=256 and 8 levels, that's about 500K vertices total. This renders the same whether the world is 1 km or 100 km wide.

Browser implementation: Geometry clipmaps work in WebGL 2 because they only require standard vertex buffer updates (no compute shaders). As the camera moves, the CPU updates the heightmap data for each ring by sampling the terrain at the appropriate resolution. The vertex shader reads height values from a texture and displaces the flat grid.

Morphing: The transition between LOD levels produces visible "popping" if handled naively. Geomorphing (detailed in its own section below) blends vertex positions between levels over a transition zone in the vertex shader, producing smooth, pop-free transitions at no extra draw call cost.

CDLOD: Quadtree-Adaptive Clipmaps

CDLOD (Strugar, 2014, Paper) improves on geometry clipmaps by using a quadtree instead of fixed concentric rings. The quadtree adapts to the terrain: flat areas use coarse nodes, while areas with high detail (cliffs, ridges) get finer subdivision.

This matters for a creator world because different chunks have different complexity. A flat meadow needs minimal resolution. A mountainous region with cliffs and caves needs maximum detail. CDLOD allocates resolution where it matters.

The CPU-side quadtree traversal is lightweight (a few hundred nodes) and determines which terrain patches to draw at which resolution. The GPU renders each patch as an instanced grid with per-patch LOD uniforms.

Concurrent Binary Trees: Planetary-Scale Tessellation

Concurrent Binary Trees (CBT) are a GPU-friendly data structure for adaptive terrain tessellation, presented by Benyoub and Dupuy (Intel, HPG 2024, Paper, GitHub).

The core idea: represent the terrain as a binary tree where each node is a triangle. Adaptive subdivision splits triangles that are close to the camera and merges triangles that are far away. The binary tree lives entirely in GPU memory as a 1D array (a binary heap), and subdivision/merge operations run as compute shaders.

The 2024 paper extends CBT from square heightmap domains to arbitrary polygon meshes. This means you can tessellate a sphere (for planetary rendering) or an arbitrary base mesh (for a game world with non-rectangular boundaries). The key improvement: using CBT as a memory pool manager instead of implicit encoding allows much higher subdivision levels.

Performance: planetary-scale terrain tessellation in under 0.2ms on console-level hardware. The algorithm scales linearly with processor count. For WebGPU, this provides a path to planet-scale terrain rendering in a browser, though the implementation complexity is high.

GPU-Driven LOD with WebGPU

WebGPU enables a fully GPU-driven terrain pipeline that eliminates CPU involvement in LOD decisions:

1. Compute pass 1: Frustum and occlusion culling. A compute shader tests each terrain patch's bounding box against the view frustum and an occlusion buffer (the depth buffer from the previous frame, downsampled). Invisible patches are discarded entirely.

2. Compute pass 2: LOD selection. For visible patches, compute the screen-space size and select the appropriate LOD level. Write the LOD level and patch ID into an indirect draw buffer.

3. Compute pass 3: Mesh generation (for volumetric terrain). For chunks with SDF data, run marching cubes to generate the mesh at the selected LOD resolution.

4. Indirect draw. A single drawIndexedIndirect() call renders all terrain patches. The GPU decides everything: what to draw, at what resolution, in what order.

This pipeline has constant CPU cost (dispatching compute shaders and the indirect draw call) regardless of world size or complexity. The GPU handles all the per-patch decisions.

@compute @workgroup_size(64)
fn lodSelection(@builtin(global_invocation_id) id: vec3<u32>) {
    let patchIdx = id.x;
    let bounds = patchBounds[patchIdx];

    if (!frustumTest(bounds, viewProjection)) { return; }
    if (occlusionTest(bounds, depthPyramid) == OCCLUDED) { return; }

    let screenSize = projectedSize(bounds, viewProjection, screenDimensions);
    let lod = clamp(u32(log2(maxScreenSize / screenSize)), 0u, MAX_LOD);

    let drawIdx = atomicAdd(&drawCount, 1u);
    drawArgs[drawIdx] = DrawArgs(patchIdx, lod, indexCount[lod], indexOffset[lod]);
}

Geomorphing: Pop-Free LOD Transitions

The biggest visual artifact in terrain LOD is popping: vertices suddenly jump to new positions when a patch switches LOD level. Geomorphing eliminates this by smoothly interpolating vertex positions between LOD levels over a transition zone.

The implementation lives entirely in the vertex shader. Each vertex stores both its current-LOD position and its next-coarser-LOD position. As the camera distance crosses the transition threshold, a morph factor blends between the two:

float morphFactor = smoothstep(lodNear, lodFar, distanceToCamera);
float morphedHeight = mix(fineLodHeight, coarseLodHeight, morphFactor);
gl_Position = viewProjection * vec4(worldPos.x, morphedHeight, worldPos.z, 1.0);

Hoppe's geometry clipmap paper (GPU Gems 2, Chapter 2) describes the full implementation for clipmap rings. The morph zone is the outer 20% of each ring. Within this zone, vertices smoothly converge to the next ring's resolution. The visual effect: terrain geometry "melts" between detail levels rather than snapping. At typical camera speeds, the transition is invisible.

Image-space blending (Scherzer et al., Paper) is an alternative that blends rendered images of two LOD levels in screen space. This handles more extreme LOD differences (e.g., mesh-to-billboard transitions) but costs an extra render pass for the transition zone.

GPU-Driven Vegetation Culling

Vegetation (trees, grass, rocks) is often the largest source of draw calls in an open world. A naive approach draws every vegetation instance every frame. GPU-driven culling, now available in Three.js via WebGPU, eliminates invisible instances before they reach the rasterizer.

Three.js's ComputeInstanceCulling (Docs) provides frustum and LOD culling for instanced meshes with 10-100x performance gains for large instance counts. The pipeline:

1. A compute shader reads all instance bounding spheres
2. Tests each against the camera frustum (6-plane test)
3. Applies distance-based LOD: instances beyond a threshold switch to lower detail or are culled entirely
4. Surviving instances are compacted into a buffer and drawn via drawIndirect

The CPU does zero per-instance work. After initial setup, the cost is one compute dispatch plus one indirect draw call per vegetation type, regardless of instance count.

For denser vegetation, Three.js's IndirectBatchedMesh (Docs) packs multiple geometry types (trees, bushes, rocks) into a single buffer and draws them with multi-draw indirect. One draw call for all vegetation in a chunk.

Combined with the density-map-based scattering from our vegetation system, this means: the density map generates 50,000 grass blade positions in a compute shader, the culling pass eliminates the 70% that are off-screen or too far away, and one indirect draw call renders the surviving 15,000 blades. Total CPU cost: negligible.

LOD for Volumetric Terrain

Volumetric terrain (SDF + marching cubes) needs its own LOD system because the mesh is generated, not pre-authored. The approach:

Multi-resolution SDF storage. Store the SDF at multiple resolutions in a mipmap-like hierarchy. Level 0 is full resolution (1-meter voxels). Level 1 is 2-meter voxels (8x less data). Level 2 is 4-meter voxels. At each level, the SDF is downsampled by taking the minimum absolute distance.

LOD-selected marching cubes. Run marching cubes on the SDF level that matches the desired LOD. Close-up chunks use level 0. Distant chunks use level 2 or 3. The Transvoxel algorithm handles the boundary between different levels.

Caching. Generated meshes are cached until the SDF changes (creator edit) or the LOD level changes (camera moved significantly). For static terrain, the mesh is generated once and reused.

Streaming Architecture for Terrain

Chunk Data Format

Each terrain chunk (64x64 meters) streams as a compact binary package:

ChunkPacket {
  header: {
    chunkX: i16, chunkZ: i16,
    version: u32,
    flags: u8  // hasHeightmap | hasVolumetric | hasVegetation
  }
  heightmap: {
    resolution: u8,      // 65x65 for full, 33x33 for half, 17x17 for quarter
    quantizedHeights: u16[resolution * resolution],  // delta-encoded, zlib compressed
    splatMap: u8[4 * resolution * resolution]         // RGBA blend weights, LZ4 compressed
  }
  volumetric?: {         // only present if flags.hasVolumetric
    sdfResolution: u8,   // typically 32 or 64
    sparseOctree: bytes  // run-length encoded sparse SDF
  }
  vegetation?: {         // only present if flags.hasVegetation
    treeDensityMap: u8[16 * 16],    // 4m resolution density grid
    grassDensityMap: u8[32 * 32],   // 2m resolution density grid
    rockDensityMap: u8[16 * 16]
  }
  creatorObjects: {
    count: u16,
    objects: PlacedObject[]   // assetId + transform + properties, ~40 bytes each
  }
}

Typical sizes:

• Heightmap-only chunk (flat terrain): 2-4 KB compressed
• Heightmap + vegetation: 4-8 KB
• Heightmap + volumetric + vegetation (complex chunk): 20-100 KB
• 5x5 full-detail neighborhood: 50-500 KB total

Progressive Chunk Loading

Chunks load in priority order based on distance, direction of movement, and data type:

Priority 1 (immediate, <100ms): Heightmap geometry for chunks the player is about to enter. The terrain surface appears first. Even at the lowest resolution (17x17 per chunk), the ground is there.

Priority 2 (fast, <300ms): Splat maps and terrain textures. The ground gets color.

Priority 3 (streaming, <1s): Full-resolution heightmap upgrade. Vegetation density maps. GPU instancing generates trees and grass.

Priority 4 (background, <3s): Volumetric SDF data for chunks with caves/overhangs. Marching cubes generates the mesh in a Web Worker, transfers the buffer to the main thread.

Priority 5 (lazy, <10s): Creator-placed objects. High-resolution textures for structures. Detail objects like flowers, small rocks, debris.

Predictive Pre-fetching

Don't wait for the player to enter a chunk before loading it. Predict where they're going based on velocity and load ahead:

• Walking speed (5 km/h): Pre-fetch 2 chunks ahead (128m). At typical broadband latency, this is 200-400ms of lead time.
• Running/riding (15 km/h): Pre-fetch 4 chunks ahead. The load ring shifts with velocity direction.
• Flying/fast travel: Pause rendering during the transition. Stream the destination chunks at highest priority. Resume rendering when enough data exists for a first frame.

The pre-fetch system tracks which chunks are in the cache, which are in-flight (requested but not yet arrived), and which are needed. A priority queue sorts pending requests by urgency. Cancel requests for chunks the player has moved away from.

Memory Budget and Eviction

A browser tab gets 2-4 GB on desktop. The terrain system needs to live within a fraction of that (the rest is for rendering, physics, networking, and the JavaScript heap).

Target budget: 256 MB for all terrain data.

At our typical chunk sizes:

• Full-detail cached chunks: ~100 (10x10 neighborhood) at 5-100 KB each = 5-10 MB for raw data
• GPU terrain geometry: ~50 MB (vertex buffers, index buffers for visible terrain)
• Terrain textures: ~100 MB (KTX2 compressed, atlas-packed)
• Vegetation instance buffers: ~50 MB (positions, rotations, scales for GPU instancing)
• SDF volumes and cached marching cubes meshes: ~50 MB

Chunks beyond the visible range are evicted from GPU memory first (textures, vertex buffers), then from CPU cache. The raw heightmap data is the last to go because it's the cheapest to keep and the most important to have when the player turns around.

Biome Transitions and Anti-Tiling

Smooth Biome Boundaries

Real landscapes don't have hard edges between biomes. A forest doesn't stop at a line and become desert. There's a gradient: dense forest thins into scattered trees, then scrubland, then sparse desert vegetation. Getting this right makes the world feel continuous rather than tiled.

AutoBiomes (Kötter et al., Paper) combines procedural terrain generation with simplified climate simulation. Temperature, humidity, and elevation determine biome type at each point. Between biomes, material weights and vegetation density interpolate over a transition zone (typically 50-100 meters wide). The transition width varies by biome pair: forest-to-grassland is wide and gradual, cliff-to-water is narrow and abrupt.

For a creator world, biome assignment runs on a coarse grid (one biome sample per 16x16 meter area). The terrain shader reads the biome values for the current fragment and its neighbors, interpolates material weights in the transition zone, and blends textures accordingly. Vegetation scattering uses the same interpolated density values, so tree density fades gradually at the forest edge.

Creator control: let creators paint biome overrides on their plots. The system generates default biomes from terrain properties, but creators can override them. Paint "swamp" on a low-lying area and the material shifts to murky water, moss, and dead trees. The biome paint map is a per-chunk 16x16 grid of biome IDs (256 bytes) that overrides the procedural assignment.

Hex-Tiling: Eliminating Texture Repetition

The most common visual artifact in terrain rendering is texture repetition. A 1-meter grass texture tiled across a 100-meter meadow produces visible grid patterns. Two techniques fix this:

Hex-tiling (Mikkelsen, Demo) replaces the square tiling grid with a hexagonal one. Each hexagonal tile samples the texture at a random offset and rotation. The hexagonal boundaries are blended to hide seams. The result is a surface that looks uniformly random rather than tiled. Cost: about 3 extra texture samples per fragment. The technique is widely used in production games and runs in any fragment shader.

Stochastic texture filtering (Pharr et al., NVIDIA, 2024, Paper) applies filtering after shading rather than before, using stochastic sampling. The error from stochastic sampling is minimal and handled well by spatiotemporal denoising. This produces more accurate filtered results and works with compressed/sparse textures. For terrain, it eliminates both tiling artifacts and the filtering artifacts that hex-tiling can sometimes introduce at transitions.

For a browser world, hex-tiling is the practical choice (it works in any shader). Stochastic filtering requires more infrastructure but produces better results if temporal denoising is available (which it would be in a WebGPU path with TAA).

Terrain Material System

Triplanar Mapping

Standard UV-mapped textures stretch horribly on steep slopes because the UV coordinates compress. Triplanar mapping projects textures along all three axes (X, Y, Z) and blends based on the surface normal:

vec3 blending = abs(normal);
blending = normalize(max(blending, 0.00001));
blending /= (blending.x + blending.y + blending.z);

vec4 xaxis = texture(material, worldPos.yz * scale);
vec4 yaxis = texture(material, worldPos.xz * scale);
vec4 zaxis = texture(material, worldPos.xy * scale);

vec4 color = xaxis * blending.x + yaxis * blending.y + zaxis * blending.z;

Cliff faces get the X or Z projection (no stretching). Flat ground gets the Y projection. The blending is smooth and automatic. No UV unwrapping required.

Babylon.js has a built-in TriPlanar Material. Three.js requires a custom shader, but the implementation is about 30 lines of GLSL.

For PBR terrain, apply triplanar mapping to all channels: albedo, normal, roughness, and ambient occlusion. The same blending weights apply to each channel.

Slope and Altitude-Based Material Assignment

Rather than hand-painting splat maps, assign materials procedurally based on terrain properties:

float slope = acos(dot(normal, vec3(0, 1, 0)));
float altitude = worldPos.y;

float grassWeight = smoothstep(0.3, 0.0, slope) * smoothstep(2000.0, 1500.0, altitude);
float rockWeight = smoothstep(0.2, 0.5, slope);
float snowWeight = smoothstep(2500.0, 3000.0, altitude) * smoothstep(0.4, 0.1, slope);
float sandWeight = smoothstep(5.0, 0.0, altitude) * smoothstep(0.15, 0.0, slope);

Flat ground at low altitude gets grass. Steep slopes get rock. High altitude gets snow (but only on surfaces flat enough for snow to accumulate). Near sea level gets sand. The transitions are smooth and physically motivated.

For a creator world, expose the altitude thresholds and blend zones as paintable parameters per chunk. Creators can move the treeline up or down, extend the snow coverage, or turn a grassy hill into a sandy desert by adjusting the material rules for their plot.

GPU-Friendly Laplacian Texture Blending

Standard texture blending (linear interpolation between layers) produces either visible seams or washed-out, low-contrast results. Laplacian pyramid blending solves this but traditionally requires expensive precomputation.

Wronski (NVIDIA, 2025, JCGT) presents a GPU-friendly variant that works in real-time shaders with no precomputation and no extra memory. The technique uses the standard mipmap chain as an approximation of the Laplacian pyramid: sample the texture at both the current mip level and a coarser level, compute the difference (the Laplacian), and blend the Laplacian contributions from each layer.

The result preserves sharp local features (individual grass blades, rock cracks) while blending smoothly at larger scales. The cost is a few extra texture taps per fragment. For terrain where you're blending 4+ material layers per pixel, this produces noticeably better results than linear blending, especially at the transitions between grass, rock, and sand.

Phasor Noise for Erosion Detail

Standard terrain often looks flat at close range because the erosion simulation operates at the heightmap resolution (1-meter grid). Real terrain has fine-scale erosion patterns (rills, gullies, weathering cracks) at centimeter scale.

Grenier et al. (2024, CGF) use phasor noise to add terrain micro-detail in real time. Phasor noise synthesizes structured patterns by defining a stochastic phase field fed into periodic functions. Applied to terrain, it creates spatially varying erosion patterns that:

• Cascade narrow rills into larger gullies across scales
• Automatically align with the terrain slope (erosion patterns follow the fall line)
• Achieve up to 32x amplification (adding detail at 32x the heightmap resolution)
• Run entirely in a fragment shader at interactive frame rates

For a browser world, phasor noise runs in the terrain shader as a detail layer. The heightmap provides the large-scale shape. Phasor noise adds convincing micro-erosion in the fragment shader without increasing geometry complexity. The parameters (pattern frequency, amplitude, orientation) can vary per biome: deep rills on bare rock, gentle ripples on sand dunes, rough bark-like texture on dried mud.

Virtual Texturing for Terrain

At large scales, the terrain texture atlas becomes unwieldy. A 4 km x 4 km world at 1 texel per centimeter needs a 400,000 x 400,000 pixel texture. Obviously impossible.

Virtual texturing (also called megatexture, from id Software's Rage) solves this by treating the terrain texture as a paged structure. The full texture exists conceptually but only the tiles visible on screen are loaded into GPU memory.

The pipeline:

1. Feedback pass: Render the terrain with a shader that outputs which texture tile each pixel needs (tile ID and mip level). Read this back to the CPU (or process with a compute shader).
2. Tile loading: Load requested tiles from the CDN or generate them procedurally from the splat map data.
3. Indirection texture: A small texture maps virtual tile coordinates to physical tile coordinates in a texture atlas.
4. Render pass: The terrain shader looks up the indirection texture to find the correct physical tile, then samples the material texture from the atlas.

WebGPU's compute shaders can handle the feedback analysis and page table management entirely on the GPU. The CPU only manages tile I/O.

The result: terrain with unique texturing at any resolution without a texture memory explosion. Tiles far from the camera load at low resolution. Close-up tiles load at high resolution. The total memory usage stays within a fixed budget (typically 128-256 MB of texture atlas).

Dynamic Terrain Effects

Puddles and Wetness

Rain doesn't just fall. It accumulates. In depressions, it forms puddles. On surfaces, it creates a wet sheen. On steep slopes, it runs off. Simulating this makes weather feel connected to the terrain rather than a purely visual overlay.

The shader-based approach doesn't simulate fluid dynamics. It uses the terrain heightmap to determine where water pools:

float concavity = heightCenter * 4.0 - heightLeft - heightRight - heightUp - heightDown;
float puddleDepth = max(0.0, concavity * rainIntensity - evaporationRate * timeSinceRain);
float wetness = smoothstep(0.0, 0.02, puddleDepth);

vec3 wetColor = baseColor * 0.7;
float wetRoughness = baseRoughness * 0.3;
vec3 finalColor = mix(baseColor, wetColor, wetness);
float finalRoughness = mix(baseRoughness, wetRoughness, wetness);

Concave areas (negative Laplacian of the heightmap) collect water. The deeper the concavity, the larger the puddle. Wet surfaces darken and become more reflective (lower roughness). The effect fades over time after rain stops.

For full puddle reflections, add a planar reflection pass at the puddle surface. Or use screen-space reflections (SSR) which are cheaper and already available in both Three.js and Babylon.js post-processing stacks.

Raindrops on puddle surfaces use a simple 2D wave equation in a feedback texture (Saurel, 2026, Blog). Each raindrop creates a ripple that propagates outward and decays. The wave texture modulates the puddle normal map, creating convincing ripple patterns at 60fps.

Footprints and Terrain Deformation

When a player walks on soft terrain (sand, snow, mud), footprints add to the sense of physical presence. The technique: maintain a small deformation texture per chunk (64x64 pixels = 4 KB) that stores height offsets. When a character steps on soft terrain, stamp a footprint shape into the deformation texture.

The terrain vertex shader reads the deformation texture and subtracts the offset from the height. The terrain fragment shader darkens the footprint area (compressed soil is darker) and increases roughness (disturbed surface).

Footprints fade over time (snow fills in, rain washes out mud prints) by gradually zeroing the deformation texture. The fade rate depends on weather: fast in rain, slow in dry conditions.

For a multiplayer world, footprint data is ephemeral and local. Each client generates footprints for visible players. The deformation texture doesn't need to sync across clients (everyone sees their own version of transient footprints). This avoids the networking cost of broadcasting every step.

Procedural Sky and Day/Night Cycle

The sky is the largest visible surface in any open world. It sets the mood for the entire terrain.

Three.js has a complete sky system that includes procedural sun/moon, day/night cycle, clouds, stars, and lens flares. The built-in Three.js Sky example (also available in WebGPU) implements Preetham's analytical sky model.

For more physically accurate results, webgpu-sky-atmosphere implements Hillaire's atmosphere model as a WebGPU post-process. It supports multiple scattering phase functions and produces correct aerial perspective (distant terrain appears hazier), sun/sunset colors, and sky gradients from first principles.

TerrainView7 demonstrates full-scale planet rendering in WebGPU with precomputed atmospheric scattering, proving that physically accurate atmosphere rendering runs in a browser.

For a creator world, the sky parameters (sun position, cloud coverage, haze density) are synchronized across all clients from the server's world clock. The sky shader runs locally on each client, producing consistent lighting across all players.

Terrain Lighting: Indirect Illumination

Direct sunlight is handled by shadow maps. But the color and brightness of terrain in shadow (ambient/indirect light) is equally important for visual quality. Pure black shadows look wrong. Shadows should be tinted by sky color (blue on a clear day, grey on an overcast day).

For a browser world, the practical approach:

Screen-space indirect lighting with visibility bitmask (Jimenez et al., 2023, Paper) improves on standard SSAO by tracking 32 directional visibility sectors per pixel. This captures not just how occluded a point is, but the directionality of the occlusion. Thin surfaces correctly allow light through from the other side. The result is indirect illumination that reacts to nearby geometry without global illumination infrastructure.

The cost is comparable to SSAO (1-2ms). The visual improvement is significant: terrain in canyons picks up bounce light from the canyon walls. The undersides of overhangs are illuminated by ground reflection. The effect combines with the atmospheric scattering from the sky shader to produce physically motivated ambient lighting.

Terrain Physics Integration

Rapier Heightfield Colliders

Rapier's WASM physics engine provides native heightfield colliders optimized for terrain. Creating a terrain collider is straightforward:

const heights = new Float32Array(65 * 65);
// Fill with heightmap data...
const groundCollider = RAPIER.ColliderDesc.heightfield(
  64, 64, heights, new RAPIER.Vector3(64.0, 100.0, 64.0)
);
world.createCollider(groundCollider);

The heightfield collider uses the structured grid for efficient broadphase queries. Raycasting against a heightfield is O(log n) rather than O(n) for a triangle mesh. For a 65x65 chunk, collision queries resolve in microseconds.

For chunks with volumetric terrain (SDF overlays), generate a triangle mesh from the marching cubes output and use a trimesh collider. This is more expensive than a heightfield collider but handles arbitrary geometry. Only generate trimesh colliders for the 3-5 chunks closest to the player. Distant chunks don't need physics.

Performance: Rapier's heightfield collider for a 65x65 chunk adds approximately 0.1ms per physics step for character controller queries. 5 active chunks with heightfield colliders: 0.5ms. One trimesh collider for a volumetric chunk: 0.2-0.5ms. Total terrain physics budget: under 1ms, well within the 2-3ms physics budget for the entire world.

Terrain-Aware Character Controller

The character controller needs to respond to terrain properties:

• Slope limiting: The character can walk on slopes up to 45 degrees. Steeper slopes cause sliding. This uses the terrain normal at the character's position (cheaply computed from the heightmap gradient).
• Surface material response: Walking on rock produces different footstep sounds and movement speed than walking on sand or mud. The terrain's splat map provides the surface material at any point.
• Step climbing: The character can step up ledges up to 0.5 meters. Rapier's KinematicCharacterController handles this automatically with configurable step height.

Interactive Terrain Editing in the Browser

For a creator world, terrain isn't just generated. It's sculpted, modified, and reshaped by players. The editing tools need to be responsive (instant visual feedback) and networked (other players see changes within seconds).

WebGPU SDF Editor

Reinder Nijhoff's WebGPU SDF Editor demonstrates that full-featured SDF modeling works in the browser today. The editor supports:

• Six primitive shapes (sphere, box, cone, cylinder, capsule, torus) with position, rotation, and scale
• Boolean operations (union, subtraction, intersection) with smooth blending and configurable blend radii
• Hierarchical scene graphs with groups and nested operations
• Real-time rendering using multiple GPU compute shader stages, octree-based space partitioning across 16,384 grid cells, and surface extraction via marching cubes or surface nets
• Temporal anti-aliasing and ambient occlusion via shadow maps

Each primitive is stored as 28 floats (112 bytes) in a single GPU buffer. This compact representation means a complex terrain edit (dozens of SDF primitives defining a cave entrance, an arch, or a carved cliff face) weighs under 5 KB and syncs to other players instantly.

For a creator world, the SDF editing workflow looks like:

1. Creator selects a sculpting tool (add sphere, subtract box, smooth blend)
2. Click/drag in the world to place and size the SDF primitive
3. The client immediately runs marching cubes on the modified SDF to update the local mesh (feedback in <16ms)
4. The SDF edit (primitive type + transform + blend mode, ~100 bytes) is sent to the server
5. The server validates the edit (within the creator's plot, doesn't intersect protected areas) and broadcasts to nearby players
6. Other players' clients apply the SDF edit and regenerate their local mesh

The total round-trip for an edit to become visible to others: 100-300ms depending on network latency. The creator sees their edit instantly because it's applied locally before server confirmation.

Brush-Based Heightmap Editing

For the heightmap layer (the 90% of terrain that doesn't need volumetric features), a simpler editing model works. The creator paints height modifications with a brush:

• Raise/lower: Add or subtract height in a radius with falloff
• Smooth: Average heights in a radius, removing sharp features
• Flatten: Set all heights in a radius to a target value
• Erosion brush: Apply a few steps of hydraulic erosion locally in the brush radius

The heightmap edit is a delta: a small patch of height changes that overlays the base terrain. The delta patch is tiny (a 32x32 grid of 16-bit height offsets = 2 KB) and syncs to other players as a single message. Multiple delta patches accumulate per chunk and are periodically merged server-side into the chunk's persistent heightmap.

Collaborative Editing Constraints

When multiple creators edit the same chunk simultaneously, the system needs rules:

• Spatial locking: Only one creator can edit a given 8x8 meter sub-region at a time. The lock is acquired when the creator starts an edit stroke and released when they lift the brush. Locks timeout after 10 seconds of inactivity.
• Non-overlapping edits: If two creators edit different parts of the same chunk, both edits apply without conflict (they modify different heightmap cells or SDF regions).
• Overlapping edits: If two creators edit the same location, the server serializes the edits in arrival order. Both clients see the same final result after reconciliation.

This is simpler than the full CRDT approach used for placed objects because terrain edits are additive operations on a continuous field (heights, SDF distances) rather than discrete object state.

Nanite-Style Virtual Geometry in WebGPU

Unreal Engine 5's Nanite renders billions of triangles by building a cluster DAG (directed acyclic graph) at build time and selecting the right LOD per-cluster at runtime based on screen-space error. The entire pipeline runs on the GPU. This approach has been ported to WebGPU.

Nanite WebGPU

Nanite WebGPU by Scthe (1.1k+ GitHub stars) is a complete browser implementation of Nanite's core architecture:

• Meshlet LOD hierarchy built offline using meshoptimizer's cluster generation
• Software rasterizer implemented in WGSL compute shaders (working within WebGPU's constraints where hardware rasterization can't do per-cluster draws efficiently)
• Per-instance and per-meshlet culling using frustum and occlusion tests
• Billboard impostors for extremely distant objects
• Texture and per-vertex normal support

The pipeline: meshes are split into clusters of ~128 triangles. Neighboring clusters are grouped, and each group is simplified (using meshoptimizer) while preserving shared boundaries. This recurses until the entire mesh collapses to a single cluster. At runtime, a compute shader walks the DAG and selects the coarsest cluster per-group that produces less than 1 pixel of error at the current screen resolution.

THREE-Nanite is an emerging Three.js implementation achieving 20-40fps on integrated graphics hardware while handling hundreds of thousands of triangles. It demonstrates that Nanite-style rendering is viable even on low-end browser hardware.

meshoptimizer: The LOD Pipeline Foundation

meshoptimizer (by Arseny Kapoulkine) is the library behind most browser-compatible LOD pipelines. Version 1.0 (2025) provides:

• Mesh simplification with error metrics (how much the shape changed, used for LOD selection)
• Cluster generation for Nanite-style meshlet hierarchies
• Vertex cache optimization for GPU-friendly triangle ordering
• Overdraw optimization to reduce pixel shader cost
• Vertex quantization and compression for smaller downloads

meshoptimizer 1.0 (released December 2025) ships a new single-header clusterlod.h that implements Nanite-style continuous LOD directly. It builds a hierarchy of clusters that are progressively grouped and simplified, usable as-is or as a reference for a custom pipeline. This is the exact cluster-DAG primitive the terrain pipeline needs for SDF/marching-cubes mesh LOD.

For our terrain pipeline, meshoptimizer processes the marching cubes output from SDF terrain into optimized, clustered meshes with LOD hierarchies. The offline processing runs server-side. The browser receives pre-clustered meshes and performs GPU-driven LOD selection at runtime.

The combination of meshoptimizer for LOD generation and WebGPU compute for runtime selection gives browser terrain the same architectural pattern as Nanite, adapted for web constraints.

Terrain-Aware Level Design

Terrain isn't just a surface to walk on. Its shape guides player movement, directs attention, and creates the emotional rhythm of exploration. The best open worlds use terrain as a design tool.

Sightlines and Landmarks

The Level Design Book's wayfinding chapter documents how terrain elevation controls what players see and where they go. A ridge hides what's beyond it, creating curiosity. A valley funnels movement toward its lowest point. A tall landmark (tower, mountain peak, unusual tree) visible from a distance gives players a goal to walk toward.

For a creator world, this means terrain generation should produce natural wayfinding features. Ridge lines should break sightlines, creating "reveal moments" when a player crests a hill and sees a new area. Valleys should converge toward interesting locations. Elevated points should exist where creators can place landmarks visible from far away.

Curiosity-Driven Exploration

Research from Purdue University (Paper) identifies four spatial exploration triggers:

1. Reaching extreme points (highest peak, furthest edge, deepest cave). Terrain should have clear extremes that reward reaching them.
2. Resolving visual obstructions (what's behind that cliff? inside that cave?). Terrain that blocks the view motivates movement to find what's hidden.
3. Out-of-place objects (a structure in the wilderness, a light in the darkness). Creator-placed objects against natural terrain create contrast that draws attention.
4. Understanding spatial connections (how does this valley connect to that coast?). Terrain that creates legible geography encourages map-reading and route-planning.

PlotMap: AI-Assisted POI Placement

PlotMap (arXiv:2309.15242) automates point-of-interest layout by taking narrative requirements (this quest needs a village near a river, that quest needs a ruin on a hilltop) and finding terrain locations that satisfy the spatial constraints. For a creator world, a similar system could suggest where to place structures based on terrain properties: "this hilltop has good sightlines for a watchtower," "this sheltered valley would suit a village."

Flowing Water and Waterfalls

Rivers and waterfalls are terrain features that combine visual appeal with ambient sound and gameplay affordance (water as a barrier, a resource, or a path).

River Rendering

Rivers in open worlds are typically rendered as textured strips that follow the terrain surface. The strip mesh is generated from the river spline (stored as control points) and projected onto the terrain heightmap. The river shader applies:

• Flow-aligned UVs that scroll in the river's direction, creating the appearance of flowing water
• Foam at edges where the river meets the bank (depth-based, similar to shoreline foam)
• Speed variation based on channel width (narrow sections flow faster, wide sections slow down)
• Transparency with depth-based color (shallow is clear, deep is dark)

For a browser world, river data is compact: a spline (20-50 control points per river segment, ~400 bytes) plus width and flow speed parameters. The client generates the river mesh locally by projecting the spline onto its terrain surface.

Waterfall Rendering

Where a river drops off a cliff face, a waterfall particle system replaces the flat river surface. The hybrid approach from real-time water simulation research (EG): regions that can't be represented by a height field (waterfalls, splashes) convert to spray, splash, and foam particles that exchange mass and momentum with the fluid simulation.

For a browser world, waterfalls are simpler: detect where the river spline crosses a terrain height discontinuity, spawn a particle system at that point with downward velocity, and add a foam splash at the base. The particles are GPU-instanced quads with scrolling alpha textures. 500 particles per waterfall, one draw call. The sound of falling water uses the Web Audio API with distance attenuation.

Impostors for Distant Terrain Features

Trees, rocks, buildings, and other terrain features become tiny at distance. Rendering them as full 3D meshes wastes GPU cycles. Impostors replace distant objects with pre-rendered flat images that face the camera.

Octahedral Impostor Atlases

An octahedral impostor captures the appearance of a 3D object from multiple viewing angles and stores them in a texture atlas. At runtime, the shader samples the atlas based on the current view direction, interpolating between the two closest captured angles.

A hemi-octahedral atlas (views from the upper hemisphere only, since you rarely look at trees from below) provides twice the angular resolution of a full octahedral atlas with the same texture size. Unity's impostor system reports frame time dropping from 111ms to 5.78ms when switching 1,600 tree instances from real meshes (140K triangles each) to impostors.

For a browser world, the impostor pipeline:

1. Server-side: render each asset from 16-32 viewing angles, capturing color, normal, and depth
2. Pack into an atlas texture (one atlas per asset, ~256x256 pixels, <100 KB as KTX2)
3. At runtime: instances beyond the impostor distance (typically 100-200m) render as billboards sampling the atlas
4. Crossfade between mesh and impostor over a 20m transition zone to hide the switch

Billboard Splatting (BBSplat)

Billboard Splatting (2024, arXiv:2411.08508) takes this further by using learnable textured planar primitives. Instead of pre-rendered views, BBSplat optimizes billboard positions and textures to best represent the 3D object from any angle. This achieves up to 17x compression compared to 3D Gaussian Splatting while maintaining view-dependent appearance. For distant terrain features in a browser world, BBSplat could reduce the per-asset impostor storage while improving angular coverage.

Professional Terrain Tools and What They Teach

Before building a terrain pipeline from scratch, it's worth understanding what professional offline tools do. These tools represent decades of terrain generation research distilled into production workflows.

Gaea (QuadSpinner) is GPU-accelerated and provides near-instant feedback on changes. It supports tiled builds up to 2 million pixels per side, automatic LOD mesh export, and a node-based graph where each node is a physical process (erosion, sedimentation, uplift, thermal weathering). Gaea's erosion nodes produce terrain that looks hand-sculpted because they model specific physical processes rather than generic noise. The key insight: Gaea doesn't use a single erosion algorithm. It offers separate nodes for fluvial erosion (river carving), thermal erosion (cliff crumbling), coastal erosion (wave action), and wind erosion (sand dune formation). Combining them in a graph produces terrain with the geological character of a specific climate.

World Machine takes a similar graph approach with a focus on terrain macro-structure. Its "layout generator" lets artists sketch the rough shape of terrain features (mountain here, valley there, coastline along this edge) and the system fills in physically plausible detail. This is exactly the creator workflow we want: sketch intent, get geology.

World Creator differentiates itself with real-time preview during editing and built-in river generation that analyzes the terrain and auto-calculates flow paths based on drainage analysis.

What we take from these tools: The node-graph approach to combining physical processes is more powerful than any single algorithm. Our server-side generation pipeline should support chaining: noise base > tectonic uplift > hydraulic erosion > thermal weathering > coastal erosion > vegetation. Creators control parameters at each stage. The pipeline runs server-side in seconds and produces heightmaps, splat maps, and vegetation density maps.

SoilMachine: Open-Source Geomorphology

SoilMachine is an open-source modular geomorphology simulator that couples multiple erosion systems (hydraulic, thermal, wind) with sediment transport and deposition. Built in C++ with GPU compute, it provides a reference implementation of the multi-process erosion approach that professional tools use.

The related soillib (C++20, MIT license) provides the underlying geomorphology simulation primitives as a reusable library. And hydro-gen implements both grid-based (shallow water) and particle-based (raindrop) hydraulic erosion in OpenGL compute shaders with real-time parameter adjustment.

These open-source tools could be adapted for our server-side generation pipeline. The compute shader implementations translate directly to WebGPU if we ever want to run erosion in the browser for real-time creator feedback.

Multi-Layer Terrain Materials

Real terrain isn't a single surface. It's layers: bedrock at the bottom, soil on top, snow or sand accumulating on surfaces. Dynamic layering changes the visual character of terrain with seasons, weather, and creator actions.

Layered Heightfield Representation

Instead of a single heightmap, use multiple height layers per grid cell:

Cell {
  bedrock_height: f16,    // permanent rock surface
  soil_height: f16,       // accumulated soil/sediment above bedrock
  snow_height: f16,       // dynamic snow accumulation
  water_height: f16       // standing water depth
}

Total: 8 bytes per cell (vs 2 bytes for a single heightmap). For a 65x65 chunk, that's 34 KB before compression. Still compact.

The visual surface is bedrock + soil + snow. The terrain shader reads all layers and blends materials accordingly: where soil is thin, rock shows through. Where snow has accumulated, the surface is white. Where water sits, you get puddles or lakes.

Dynamic Accumulation

Snow accumulates on flat, upward-facing surfaces during snowfall. The accumulation rate depends on surface normal (steep slopes don't hold snow), temperature (altitude-dependent), and shelter (areas below overhangs stay clear). A compute shader pass updates the snow layer once per weather tick (every few seconds).

Sand accumulation works similarly with wind-driven deposition. Wind carries particles from exposed surfaces and deposits them behind obstacles and in sheltered areas.

For a creator world, dynamic accumulation means the terrain looks different in different weather. Snow covers the world during a blizzard and melts during a clear spell. Rain fills depressions with water. This makes the world feel responsive without creators doing anything.

Multi-Layered Erosion

The 2024 paper "3D Real-Time Hydraulic Erosion Simulation using Multi-Layered Heightmaps" (EG) extends erosion to work across layers. Water erodes soil faster than bedrock. Sediment deposits as a new soil layer. The simulation maintains layer integrity (bedrock stays below soil) while enabling complex features like overhangs (where bedrock overhangs eroded soil below).

Performance: approximately 6ms per simulation step on an RTX 3070 at 2048x2048 resolution. This is fast enough for server-side generation but too slow for per-frame browser simulation. The layered representation works for static terrain generation, and the dynamic snow/water accumulation runs as a cheaper per-frame shader.

Grass, Rock, and Detail Rendering

The landscape needs more than terrain geometry and textures. It needs grass blades that sway in the wind, rocks scattered on slopes, and small details like flowers, pebbles, and fallen branches that make close-up views look natural.

GPU-Instanced Grass

Browser-based grass rendering is well-proven in Three.js and works through GPU instancing. The approach from al-ro's grass demo renders 100,000 grass blades with a single draw call using InstancedBufferGeometry.

Each grass blade is a simple quad (4-8 triangles). Per-instance attributes define position, height, bend direction, color variation, and wind phase. The vertex shader:

1. Reads the per-instance transform
2. Applies wind animation using sine waves keyed to world position and time
3. Bends the blade based on wind strength (more bend at the tip, none at the base)
4. Applies color gradient (darker at the base, lighter at the tip for subsurface scattering)

Codrops' fluffy grass tutorial (2025, Tutorial) demonstrates a shell texturing approach: render the ground plane multiple times at increasing offsets, with each layer sampling a noise texture to create the appearance of dense grass volume. This is cheaper than individual blade instancing for very dense coverage but less realistic at close range.

For a creator world, grass density comes from the vegetation density map per chunk. The GPU scatters blade positions from the density map at render time. No per-blade data is stored or streamed. The density map is a 32x32 grid per chunk (1 KB), and the GPU generates thousands of blade instances from it.

Procedural Rock and Cliff Detail

Cliff faces and rocky terrain need geometric detail that the base heightmap or SDF can't provide at reasonable resolution. Two approaches complement each other:

GPU mesh shader resurfacing (Raad et al., Eurographics 2025, Paper) generates procedural geometry at render time from a coarse control mesh. The mesh shader reads a base terrain surface and adds displacement, cracks, and protrusions without storing the detail geometry in memory. This reduces VRAM usage and enables dynamic LOD.

Instanced rock scattering places pre-made rock meshes on steep slopes and cliff edges using GPU instancing. A compute shader reads the terrain normal and slope, and scatters rock instances where the slope exceeds a threshold. Each instance is a small mesh (200-500 triangles) with random rotation and scale. 10,000 scattered rocks add negligible rendering cost with instancing.

Roads and Paths

Creator-placed roads, trails, and paths need to conform to terrain and modify the surface material (replacing grass with dirt or stone).

The approach used by every major game engine: define the path as a spline (a series of control points). Project the spline onto the terrain surface. Generate a strip mesh that follows the spline and sits slightly above the terrain. Apply a road texture to the strip. In the terrain shader, blend the terrain material toward the road material within the spline's width using a projected texture or decal.

For a browser world, a creator draws a path on the terrain. The client generates control points and sends them to the server (a few dozen vec3 values). The server stores the spline. All clients render the road strip locally by projecting the spline onto their terrain mesh. The road data is tiny (path spline points, maybe 200 bytes) but the visual impact is large: paths connecting creator builds make the world feel inhabited.

Shadows for Terrain

Terrain shadows are critical for readability (understanding terrain shape) and atmosphere (time-of-day mood). In an open world, the sun casts shadows across the entire visible terrain.

Cascaded Shadow Maps (CSM)

CSM splits the view frustum into 3-4 distance ranges (cascades). Each cascade renders a shadow map from the sun's perspective at a resolution appropriate for its distance. Close cascade: high resolution (detailed shadows under trees and buildings). Far cascade: low resolution (broad mountain shadows).

Both Three.js and Babylon.js support CSM. The key optimization for terrain: only render terrain in the shadow map, not individual grass blades or small details. Grass self-shadows using the terrain's shadow map, not its own.

Performance budget: 3-4 shadow cascades at 1024x1024 each. Rendering terrain into shadow maps costs 0.5-1ms (the terrain geometry is already in GPU memory). Sampling 4 cascades in the terrain shader adds 0.2-0.3ms.

Terrain Self-Shadowing from Heightmap

For very large terrain where CSM becomes expensive, pre-compute a horizon map: for each terrain cell, store the maximum elevation angle in 8 compass directions. At render time, compare the sun angle to the horizon map to determine if a point is in shadow. This is how Skyrim handles terrain self-shadows for distant terrain (beyond CSM range).

The horizon map is computed server-side from the heightmap (a few seconds of processing) and streams as a 128x128 per-chunk texture (16 KB compressed). The visual impact is significant: mountain valleys darken realistically even at extreme view distances.

Terrain Data Compression for Streaming

The network is the bottleneck for a browser world. Every byte saved in terrain data is a faster load time.

Heightmap Compression

Raw 16-bit heightmaps compress well because adjacent cells have similar values. The pipeline:

1. Delta encoding: Store the difference between each cell and its predicted value (average of neighbors). Delta values are small, clustering near zero.
2. Quantization: For distant chunks, reduce precision from 16-bit to 12-bit or 8-bit. At 500 meters away, 8-bit height precision (0.4m resolution on a 100m height range) is indistinguishable from 16-bit.
3. Entropy coding: Apply zlib or brotli compression to the delta-encoded stream. Typical compression ratio: 4-8x.

Result: a 65x65 chunk at 16-bit goes from 8.4 KB raw to 1-2 KB compressed. At 8-bit reduced precision: 0.5-1 KB.

Progressive Heightmap Streaming

Send terrain at low resolution first, then refine. A 17x17 heightmap (the minimum for a 64m chunk with 4m cell spacing) is 578 bytes raw, under 200 bytes compressed. The terrain is visible instantly. Then stream the 33x33 refinement (adds odd-row/column samples). Then the 65x65 full resolution. Each level adds detail without replacing previous data.

This maps to the geometry clipmap LOD rings: distant terrain uses the low-resolution version (17x17), mid-range uses medium (33x33), close-up uses full (65x65). The streaming priority matches the rendering LOD.

SDF Volume Compression

Sparse SDF volumes compress dramatically because most voxels are far from the surface (empty space). Options:

Run-length encoding: Encode runs of identical values (empty voxels). Typical SDF volumes are 95%+ empty, so RLE achieves 10-50x compression.

Sparse octree: Only store octree nodes that contain surface-crossing voxels. Empty space has no nodes. A 64^3 SDF volume with a single cave tunnel might have only 2,000-5,000 occupied nodes (vs 262,144 total voxels), each stored as 1-2 bytes.

Entropy-driven progressive compression (2024, HAL) applies to 3D spatial data by recursively partitioning space with entropy-optimized planes and adaptive quantization. This produces a stream of refinements optimized for rate-distortion trade-offs, particularly beneficial at the low bitrates of network streaming.

Putting It All Together: The Browser Terrain Pipeline

The strategic overview at the top of this article gives the quick decision framework and phased build plan. This section provides the full technical detail for both the server-side generation pipeline and the browser rendering pipeline.

Generation Pipeline (Server-Side)

The generation pipeline runs as a directed graph of physical processes, inspired by Gaea and World Machine's node-graph approach. Each stage takes the previous stage's output and refines it. Creators control parameters at each stage.

A new 4x4 km world generates in 15-90 seconds. Creator edits (sculpting, parameter changes) re-run only the affected stages for the affected chunks, typically completing in under 5 seconds.

Rendering Pipeline (Browser)

At 60fps (16.6ms per frame), the terrain system uses 21% of the frame budget on WebGPU and 39% on WebGL 2. The remainder is available for player avatars, creator objects, UI, networking, and post-processing.

Why This Works in a Browser

The entire pipeline is designed around three browser constraints:

Memory (2-4 GB max): The 256 MB terrain budget fits because heightmap chunks are 2-8 KB each (delta-encoded), SDF volumes are sparse (100-500 KB per volumetric chunk), vegetation is generated at runtime from 1 KB density maps, and textures use KTX2 compression (150 KB per 1024x1024). At any moment, the world visible in the browser weighs 50-200 MB total.

No disk access: Everything streams over the network. Progressive loading means the player sees terrain in <100ms (low-res heightmap), textured terrain in <300ms, and full detail in <3s. Pre-fetching based on velocity hides load times during normal exploration.

GPU varies wildly: The WebGPU path handles high-end desktops. The WebGL 2 fallback handles everything else, including mobile. The same chunk data drives both paths. The difference is rendering technique, not data format. A Chromebook running WebGL 2 sees the same world as an RTX 4090 running WebGPU, just at lower detail and shorter view distance.

Research Papers

Terrain Representation and Mesh Generation

"Marching Cubes: A High Resolution 3D Surface Construction Algorithm" -- Lorensen and Cline (SIGGRAPH 1987). DOI. The foundational algorithm for extracting triangle meshes from volumetric data. Still the most widely used isosurface extraction method 38 years later. GPU-parallel implementations run in real time in WebGPU compute shaders.

"Dual Contouring of Hermite Data" -- Ju, Losasso, Schaefer, Warren (SIGGRAPH 2002). DOI. Produces meshes that preserve sharp features (cliff edges, rock corners) that marching cubes rounds off. Requires surface normals in addition to distance values.

"Neural Dual Contouring" -- Chen et al. (2022). arXiv:2202.01999. Replaces the least-squares vertex placement in dual contouring with a learned predictor. Better surface quality for complex natural features.

"The Transvoxel Algorithm" -- Lengyel (2009, updated 2024). transvoxel.org. Seamless LOD transitions for voxel terrain. Eliminates cracks at resolution boundaries using 73 transition cell types. Patent-free, designed for real-time applications.

Terrain LOD and Rendering

"Geometry Clipmaps: Terrain Rendering Using Nested Regular Grids" -- Losasso and Hoppe (SIGGRAPH 2004). Paper. Constant-cost terrain rendering with concentric LOD rings. Handles 40 GB terrains at interactive rates. The foundation for most browser terrain renderers.

"CDLOD: Hybrid LOD for Terrain Rendering" -- Strugar (2014). Paper. Quadtree-adaptive improvement on geometry clipmaps. Allocates resolution based on terrain complexity rather than distance alone.

"GPU-Driven Rendering Pipelines" -- Ubisoft (SIGGRAPH 2015), Wihlidal and Hoppe. Formalized the GPU-driven approach where compute shaders handle culling, LOD selection, and draw call generation. The architectural pattern for our WebGPU terrain pipeline.

Physical Terrain Generation

"Physically-Based Analytical Erosion for Fast Terrain Generation" -- Cordonnier et al. (2024). HAL. Analytical stream power law erosion that avoids iterative simulation. Generates physically plausible terrain in milliseconds.

"Fast Hydraulic Erosion Simulation and Visualization on GPU" -- Mei, Decaudin, Hu (2007). HAL. GPU-parallel hydraulic erosion using shallow-water simulation. The basis for most game engine erosion implementations.

"Arenite: A Physics-Based Sandstone Simulator" -- SIGGRAPH 2025. Project. Multi-physics erosion generating arches, hoodoos, and alcoves from stress and erosion simulation. Runs in under 5 minutes on desktop GPUs.

"Efficient Debris-flow Simulation for Steep Terrain Erosion" -- Purdue CGVLAB (2024). Paper. GPU-accelerated debris flow and steep-slope erosion producing realistic mountain terrain features.

"Flexible Terrain Erosion" -- IRIT-STORM (2024). Springer. Particle-based erosion that works across heightfields, voxel grids, implicit surfaces, and layered materials with a unified interface. Enables one erosion system for hybrid terrain representations.

Terrain Detail and Texturing

"GPU-Friendly Laplacian Texture Blending" -- Wronski (NVIDIA, 2025). JCGT. Real-time Laplacian pyramid blending for terrain materials without precomputation. Preserves sharp features while eliminating seam artifacts. A few extra texture taps per fragment.

"Real-time Terrain Enhancement with Controlled Procedural Patterns" -- Grenier et al. (2024). CGF. Phasor noise-based micro-erosion detail at up to 32x heightmap resolution. Slope-aligned patterns run entirely in a fragment shader.

Adaptive Tessellation

"Concurrent Binary Trees for Large-Scale Game Components" -- Benyoub and Dupuy (Intel, HPG 2024). Paper. GPU-friendly binary tree data structure for adaptive terrain tessellation. Renders planetary-scale geometry in under 0.2ms. Extended from square domains to arbitrary polygon meshes.

Cave and Underground Generation

"PLUME: Procedural Layer Underground Modeling Engine" -- 2024. arXiv:2508.20926. Open-source framework for generating realistic cave and lava tube environments using layered procedural rules. Originally built for space exploration robotics.

Neural Terrain Synthesis

"InfiniteDiffusion: Bridging Learned Fidelity and Procedural Utility for Open-World Terrain Generation" -- Goslin (2025, SIGGRAPH 2026). arXiv:2512.08309. Infinite, seed-consistent terrain generation using hierarchical diffusion models with Laplacian encoding. 9x faster than baseline on consumer GPUs.

"TerraFusion: Joint Generation of Terrain Geometry and Texture" -- 2025. arXiv:2505.04050. Latent diffusion for simultaneous heightmap and texture synthesis with sketch conditioning.

"MESA: Text-Driven Terrain Generation" -- CVPR 2025 Workshop. arXiv:2504.07210. Text-to-terrain using Copernicus remote sensing training data.

"Geodiffussr: Generative Terrain Texturing with Elevation Fidelity" -- 2025. arXiv:2511.23029. Text-guided terrain texture generation that respects elevation data using flow-matching.

"Sketch2Terrain: AI-Driven Real-Time Terrain Sketch Mapping" -- 2025. Project. Sketch-to-terrain in augmented reality. 38% efficiency improvement over manual mapping.

Vegetation and Ecosystem Simulation

"GPU-Based Real-Time Procedural Distribution of Vegetation on Large-Scale Virtual Terrains" -- SBGames 2018. Paper. Quadtree-based vegetation scattering using biotic/abiotic factors on GPU.

"Procedural Generation and Rendering of Forests" -- 2022. Paper. L-system tree generation combined with ecosystem competition simulation for realistic forest distribution.

"Real-Time Procedural Generation with GPU Work Graphs" -- AMD GPUOpen 2024. Paper. GPU work graphs generating 79K+ vegetation instances in under 4ms.

Browser GPU Technology

"GSWT Renderer" -- SIGGRAPH Asia 2025. GitHub. WebGPU + Rust/Wasm renderer using Gaussian Splatting Wang Tiles for infinite 3D terrain with dynamic LOD and streaming.

"GPU Compute in the Browser at the Speed of Native: WebGPU Marching Cubes" -- Usher (2024). Blog. Demonstrates that WebGPU compute achieves native-speed performance for parallel mesh generation algorithms.

Voxel Rendering and Large-Scale Scenes

"Aokana: A GPU-Driven Voxel Rendering Framework for Open World Games" -- 2025. arXiv:2505.02017. Sparse Voxel DAG with LOD and streaming for scenes with tens of billions of voxels. 9x memory reduction and 4.8x faster rendering than prior state-of-the-art. Designed for integration with game engines.

Procedural Geometry and Resurfacing

"Real-time Procedural Resurfacing using GPU Mesh Shaders" -- Raad et al. (Eurographics 2025). Paper. Generates detailed geometric surfaces from coarse control meshes at render time using mesh shaders. Enables dynamic LOD without storing high-resolution geometry in memory.

Terrain Shadows

"Optimizing Terrain Shadows" -- AMD GPUOpen. Blog. Practical CSM optimization for large-scale terrain. Covers cascade splitting, GPU-efficient shadow map rendering, and terrain-specific optimizations.

Virtual Geometry and Mesh Optimization

"Nanite WebGPU" -- Scthe (2024). GitHub, Demo. Complete browser implementation of UE5 Nanite architecture: meshlet LOD hierarchy, software rasterizer in WGSL, per-meshlet culling, billboard impostors.

"Billions of Triangles in Minutes" -- Kapoulkine (2025). Blog. Meshoptimizer v1.0 for hierarchical clustered LOD generation. Processes massive meshes into Nanite-style cluster DAGs efficiently.

"Billboard Splatting (BBSplat)" -- 2024. arXiv:2411.08508. Learnable textured planar primitives for novel view synthesis achieving 17x compression vs 3D Gaussian Splatting.

Terrain Lighting and Global Illumination

"Global Illumination in Once Human" -- GDC 2025. Session. Hybrid GI for 16km open world: neural network compressed probes (69:1 ratio), ML-based indoor/outdoor leak resolution, dynamic probe reactions.

"GI with AMD FidelityFX Brixelizer" -- GDC 2024. Paper. Compute-based sparse distance field cascades with screen-space probes. No hardware ray-tracing required.

"Radiance Cascades" -- 2024. Blog. Noiseless real-time global illumination using cascaded radiance structures without temporal accumulation.

Level Design and Exploration

"PlotMap: Automated Layout Design for Building Game Worlds" -- 2023. arXiv:2309.15242. AI-assisted POI placement satisfying narrative spatial constraints on terrain.

"Spatial Exploration Triggers" -- Purdue University (FDG 2022). Paper. Four design patterns that drive player exploration: extreme points, visual obstructions, out-of-place objects, spatial connections.

Data Compression and Streaming

"Entropy-driven Progressive Compression of 3D Point Clouds" -- SGP 2024. Paper. Rate-distortion optimized progressive compression using recursive space partitioning with adaptive quantization. Produces refinement streams suited to variable-bandwidth network streaming.

Interactive Editing

"WebGPU SDF Editor" -- Nijhoff (2026). Project. Full-featured SDF modeling in the browser with real-time marching cubes, boolean operations, smooth blending, and octree space partitioning. 112 bytes per primitive.

River and Coastal Generation

"Procedural River Drainage Basins" -- Patel (Red Blob Games). Project. Drainage-first river generation using Voronoi/triangle mesh edge classification. Builds river hierarchies before assigning terrain elevation.

"NEWTS1.0: Numerical Model of Coastal Erosion by Waves and Transgressive Scarps" -- MIT (2024). Paper. Simplified coastal erosion model using uniform retreat and wave-driven erosion. Generates headlands, bays, sea stacks, and arches matching real geomorphology.

Further Reading

• Browser 3D Open World Tech covers the full architecture for a multiplayer creator world in the browser
• Web Games Tech Stack in 2026 covers WebGL, WebGPU, and WebAssembly fundamentals
• Three.js + USDC Tech Report on loading 3D assets in the browser
• Frontier Open-Source Gen AI Models for the AI generation pipeline
• WebGPU getting started — compute shaders for GPU-driven terrain
• Web Workers for game logic — offloading terrain generation to Workers
• Game physics libraries — physics for terrain interaction and heightfield colliders
• Where to Find Free Game Assets — texture and material sources for terrain rendering

We want to put our creators in a shared open world. Not a lobby. Not a gallery. A living 3D space where people can explore, build, and stumble onto each other's work. Something that loads in a browser tab and feels like a place worth being in.

That's a hard problem. Skyrim and The Witcher 3 spent hundreds of millions of dollars building their worlds, and they still ship on dedicated hardware with tens of gigabytes on disk. We're targeting a browser tab, shared state across hundreds of players, and a world that creators can actually reshape.

This guide documents what we found researching the tech. What's possible today, what's coming, and what architecture would get us there.

What We Can Learn from Skyrim and The Witcher

Before picking technology, it helps to understand how the two most successful open worlds actually work. Their techniques map surprisingly well to browser constraints.

Skyrim's Cell System

Skyrim divides its world into a grid of 57x37 exterior cells, each 4096x4096 game units (roughly 59 meters). The engine loads a 5x5 grid of cells around the player at any time. As you walk, cells on the trailing edge unload while cells on the leading edge stream in. Interior spaces (dungeons, buildings) are separate worldspaces loaded through door triggers.

This is directly applicable to a browser world. You don't load the entire map. You load a neighborhood around the player and swap chunks as they move. The key insight: Skyrim never lets you see the full detail of distant terrain. It uses a multi-resolution approach.

LOD (Level of Detail) tiers in Skyrim:

• Full detail within 1-2 cells (player's immediate area)
• Simplified meshes at medium range (objects lose small geometry)
• Billboard impostors for trees and rocks beyond medium range
• Terrain LOD uses pre-baked low-resolution meshes for distant landscape
• Object fade distances tuned per-object (grass fades first, buildings last)

Skyrim's landscape uses a heightmap-based terrain with 32x32 vertex patches per cell. Each patch can have different textures blended with alpha maps. This is cheap to store and render. A single cell's terrain data is measured in kilobytes, not megabytes.

What we can apply: Chunk-based streaming, heightmap terrain, aggressive LOD, interior/exterior separation, and the idea that distant terrain doesn't need to be geometrically accurate, just visually convincing.

The Witcher 3's Streaming Architecture

The Witcher 3's world is larger than Skyrim (roughly 136 km2 across all regions) and uses a more sophisticated streaming system. CD Projekt RED built a custom engine (REDengine 3) where the world is divided into streaming sectors, not a strict grid.

Each sector contains a hierarchy of content layers:

• Terrain streams as patches with 4 LOD levels
• Foliage uses GPU instancing with distance-based culling
• Static geometry (buildings, rocks) streams independently from terrain
• Gameplay objects (NPCs, items, triggers) load based on quest state and proximity

The renderer uses deferred shading with physically-based materials. One trick that's particularly relevant: The Witcher 3 aggressively uses impostors. A tree that's 200 meters away isn't a 3D model with branches and leaves. It's a flat textured quad that always faces the camera. The engine pre-renders these impostors from multiple angles and swaps them as the camera moves. You never notice because they're far enough away.

World composition: The Witcher 3's open world was built in layers by separate teams working simultaneously. The terrain team sculpted the landscape. The environment art team placed structures. The quest team wired up triggers and NPC paths. This layered composition system let a large team work without stepping on each other.

What we can apply: Layered world composition (critical for a creator platform), impostor rendering for distant objects, deferred rendering for many light sources, and the insight that streaming should be content-aware (don't load interiors you can't see, don't spawn NPCs you're not near).

Breath of the Wild's Chemistry Engine

Nintendo's approach to open world design inverted the typical formula. Instead of filling the world with scripted content, they built systemic rules and let players discover emergent behavior. Fire spreads to grass. Metal conducts electricity in thunderstorms. Wind carries objects. Everything interacts with everything else through a consistent physics/chemistry layer.

This matters for a creator world because it suggests that the world itself can be more interesting than any individual placed object. If creators can define material properties and interaction rules (not just place static meshes), the world gains emergent behavior that makes exploration worthwhile even in areas nobody explicitly designed.

Technical takeaways from BotW:

• Relatively low polygon count and resolution (900p docked on Switch). The stylized cel-shaded look hides the low fidelity. A browser world could achieve similar visual quality today.
• Distance rendering uses a toon-shaded fog that fades into watercolor-style skyboxes. Cheap to render, beautiful in practice. This kind of atmospheric perspective is essentially free in a fragment shader.
• The world is roughly 84 km2 but sparse. Most of the map is traversable terrain with points of interest spread across it. Dense content is reserved for towns and shrines. This "sparse but interesting" approach reduces asset requirements dramatically compared to The Witcher 3's densely populated Novigrad.
• Physics objects have material tags (wood, metal, food, explosive) that determine interactions. The number of actual rules is small (maybe 20-30 interaction types) but they combine in ways that feel infinite.

GTA V's World Layering

Rockstar's approach to Los Santos teaches a different lesson. GTA V's world feels alive because of layered ambient systems, not because every NPC has a quest.

The city has traffic systems that simulate hundreds of vehicles on a road network. Pedestrians walk routes, react to the player, and interact with each other. Time of day changes the population density, the types of NPCs present, and the ambient activity. Weather affects driving physics and NPC behavior.

For a creator world, the insight is that ambient life makes the difference between a world that feels like a museum and one that feels like a place. Even simple behaviors (birds flying overhead, waves lapping at a shore, NPCs walking between buildings) create the illusion of a living world.

Technical takeaways from GTA V:

• The map is 75 km2 and uses aggressive LOD combined with a streaming system that loads based on velocity (driving fast loads further ahead than walking).
• GTA Online puts 30 players in this world simultaneously. Even at 30, Rockstar found they needed spatial interest management: you get detailed updates about nearby players and less frequent updates about distant ones. The same principle applies at our 200-player target.
• GTA V's asset pipeline is one of the most efficient ever built. The entire game fits in roughly 80 GB because of extreme texture compression, mesh sharing, and procedural detail. Their building interiors reuse modular pieces extensively.
• The game uses a sophisticated imposter system where distant buildings are actually flat photographs composited together. The player never notices because the transition distances are tuned to specific viewing angles.

Elden Ring's Seamless Open World

FromSoftware built Elden Ring on the same engine as Dark Souls but scaled it to an open world. The result is interesting because it shows how a relatively small team (compared to Rockstar or CD Projekt) can build a large open world.

The trick is density variation. Elden Ring's map is huge (roughly 79 km2) but large sections are open terrain with scattered enemies and points of interest. The dense, hand-crafted content (Legacy Dungeons like Stormveil Castle) is embedded in the open world and uses the same interior/exterior separation that Skyrim uses.

Technical takeaways from Elden Ring:

• The world streams in large tiles. On horseback you can see very far, but the distant terrain is extremely simplified. Close up, the detail is comparable to Dark Souls 3.
• Loading screens only appear when fast-traveling or entering certain dungeons. The open world itself streams without interruption.
• The game reuses environmental assets extensively. The same tree models, rock formations, and ruin pieces appear across the map in different arrangements. With enough variety in arrangement and lighting, repetition isn't noticeable. This is directly applicable to a creator world where a library of modular pieces can generate infinite variety.
• Multiplayer is session-based (summoning other players into your world), not persistent. But the asynchronous features (messages, bloodstains, ghosts of other players) create a sense of shared existence without the networking overhead of a persistent server. These ambient multiplayer features would be cheap to add to a browser world.

No Man's Sky: Procedural Everything

No Man's Sky is the extreme case of procedural generation. 18 quintillion planets generated from a shared seed, so every player sees the same universe without any of it being stored on a server.

The generation pipeline works in stages: galaxy-level seeds determine star positions, star seeds determine planet count and types, planet seeds drive terrain generation (multi-octave noise), biome assignment, flora/fauna species, color palettes, and resource distribution. Everything computes deterministically from the seed, so two players visiting the same coordinates see the same planet without exchanging any planet data.

Technical takeaways from No Man's Sky:

• Terrain generation uses voxel-based marching cubes with a stack of noise functions. This allows caves, overhangs, and floating islands that heightmap terrain can't represent. The trade-off is higher compute cost per chunk, but the result is more visually interesting terrain.
• The base-building system is closest to what we're envisioning. Players place structures that persist on the server and are visible to other players. The base data is compact (a list of parts with positions and rotations) and streams on demand when someone visits the planet.
• The game was initially criticized for repetitive content despite infinite variety. Seed-based generation can produce infinite terrain but limited surprises. The lesson: procedural generation works for the canvas, but creator-placed content is what makes a place feel designed and intentional.
• Hello Games added multiplayer years after launch. Up to 32 players share a session with full building and exploration. The network model is simple: one player hosts, others connect. For a browser world with persistent state, a server-authoritative model works better, but No Man's Sky proves that shared procedural worlds are tractable.

Minecraft: The Blueprint for Creator Worlds

Minecraft is the most important reference point for what we're building, more than any graphical AAA title. 300 million copies sold. The world is infinitely generated, fully destructible, and multiplayer. Creators don't just place objects in Minecraft. They reshape the terrain itself.

Technical takeaways from Minecraft:

• The world is divided into 16x16x384 block chunks. Only chunks near the player are loaded (render distance is configurable). This is the same chunk streaming pattern as Skyrim but with fully editable terrain.
• Each chunk is stored as a palette-compressed array of block IDs. A chunk with only 5 different block types stores a 4-bit palette index per block instead of a full block ID. This makes chunk data very compact (typically 10-50 KB per chunk after compression).
• Minecraft's multiplayer protocol is well-documented and relatively simple. The server sends chunk data as the player moves. Block changes are broadcast as small delta updates (position + new block type). This is exactly the model we'd use for creator edits.
• The game runs in Java and now has a Bedrock Edition in C++. There are browser-based Minecraft clones (ClassiCube, eaglercraft) that prove the core concept works in WebGL. They typically handle 8-12 chunk render distance at 60fps, which gives a view of about 200-400 meters.
• Minecraft's modding ecosystem is its real moat. Mods add new blocks, entities, biomes, and gameplay systems. For a creator world, this suggests that extensibility matters as much as the base experience. If creators can define new interaction types (not just place objects), the world gets richer over time.
• Redstone (Minecraft's in-game wiring system) shows that creators will build complex systems if you give them simple, composable primitives. Logic gates, automatic farms, calculators. The simple rule set generates extraordinary complexity. This is the same lesson as BotW's chemistry engine.

Common Patterns Across AAA Open Worlds

Looking across all these titles, the same patterns keep showing up:

Spatial partitioning is universal. Whether it's cells (Bethesda), sectors (CD Projekt), chunks (Mojang/Rockstar), or tiles (FromSoftware), every open world divides space into loadable units. No engine tries to hold the entire world in memory.

Multi-resolution everything. Terrain, meshes, textures, and even audio all have multiple quality levels. The resolution you get depends on how close you are and how important the object is.

Occlusion culling matters more than raw triangle throughput. Not rendering what you can't see saves more performance than optimizing what you can. Skyrim uses a simple distance-based system. The Witcher 3 uses software occlusion with large occluders (buildings, cliffs). Modern engines like UE5's Nanite take this further with hardware occlusion queries.

Async loading hides transitions. These games don't show loading screens for the open world (only for fast-travel or interior transitions). They load content on background threads, decompress in parallel, and swap in new content gradually.

Art direction over polygon count. Skyrim shipped in 2011 with graphics that were modest even then. BotW runs on a tablet-class chip and looks beautiful. Minecraft uses 16x16 pixel textures and is one of the most visually recognizable games ever made. For a browser world, this matters enormously. A well-directed art style at lower fidelity will always beat a technically advanced but artistically flat world.

Systemic design beats scripted content. BotW's chemistry engine, Minecraft's block interactions, and GTA V's traffic systems all create emergent behavior from simple rules. This is cheaper to build, cheaper to run, and generates more player stories than hand-crafted scripted sequences. For a creator world, systemic design means the world stays interesting even in areas nobody explicitly designed.

Creator-placed content needs persistence and visibility. Minecraft bases, No Man's Sky bases, and GTA Online properties all persist across sessions and are visible to other players. The data model for creator content is always compact (part IDs + transforms) while the visual representation is rich (the client expands the data into full 3D scenes).

Rendering: What Can a Browser Actually Do?

Three.js

Three.js is the foundation. It has the largest community (over 100K GitHub stars), the most examples, and the broadest compatibility. It abstracts WebGL 2 and has experimental WebGPU support via WebGPURenderer.

For an open world, Three.js gives you:

• Instanced rendering for foliage, rocks, and repeated geometry (InstancedMesh)
• LOD system built in (THREE.LOD swaps meshes by distance)
• Frustum culling automatic per-object
• Terrain via custom BufferGeometry or heightmap-based PlaneGeometry
• PBR materials through MeshStandardMaterial and MeshPhysicalMaterial
• Post-processing via EffectComposer (bloom, SSAO, tone mapping)
• Shadow maps with cascaded shadow mapping possible through custom code
• glTF/GLB as the primary asset format (compact, GPU-ready)

Three.js also has an active ecosystem of tools that matter for open worlds. three-mesh-bvh accelerates raycasting and spatial queries over complex meshes. postprocessing (by vanruesc) provides a more performant post-processing stack than Three's built-in one. three-gpu-pathtracing enables reference-quality rendering.

Limitations for open worlds: Three.js doesn't have a built-in scene graph that handles streaming, LOD management, or spatial partitioning at scale. You build those yourself. There's no built-in entity component system (ECS), no physics, and no terrain system. It's a renderer, not an engine. That's actually an advantage for a custom open world because you control memory layout and loading strategy, but it means more up-front work.

const lod = new THREE.LOD();
lod.addLevel(highDetailMesh, 0);
lod.addLevel(mediumDetailMesh, 50);
lod.addLevel(lowDetailMesh, 200);
lod.addLevel(impostorSprite, 500);
scene.add(lod);

Babylon.js

Babylon.js is the other major contender. Microsoft-backed, it has deeper built-in features than Three.js for the specific problem of open worlds.

Relevant built-in features:

• Octree-based scene partitioning for efficient culling of large scenes
• Solid Particle System for massive instanced rendering
• Terrain from heightmaps with built-in LOD and multi-texture splatting
• Node Material Editor for visual shader creation
• WebGPU support (more mature than Three.js, as Babylon invested earlier)
• Havok physics integration (Wasm-compiled, production-quality)
• glTF streaming with progressive loading

Babylon's DynamicTerrain extension generates terrain chunks on-the-fly from heightmap data, handles LOD automatically, and supports texture splatting. This is much closer to what Skyrim does than anything Three.js offers out of the box.

const terrain = new BABYLON.DynamicTerrain("terrain", {
  terrainSub: 100,
  mapData: heightmapData,
  mapSubX: 1000,
  mapSubZ: 1000,
}, scene);
terrain.LODLimits = [4, 3, 2, 1];

The trade-off: Babylon.js is a larger library (the full build is ~1-2MB minified vs Three.js at ~600KB). But for an open world, you'll likely add enough custom code to Three.js that the size difference becomes negligible. Babylon also has its own Node Material system, inspector, and dev tools, which speed up iteration.

Babylon.js 8.0 (released March 2025) hardened the case for it as the engine core here. All core engine shaders now ship in both GLSL and WGSL, so WebGPU works with no conversion layer and the WebGPU bundle is roughly half the size it used to be. The same release brought Havok's full character controller into the engine, area lights, a rebuilt audio engine, and Gaussian splatting improvements (SPZ and compressed PLY formats, spherical harmonics, and lower memory/CPU footprint).

PlayCanvas

PlayCanvas is worth mentioning because it's the most production-proven web-first 3D engine. Snap, Facebook, and numerous advertising clients have shipped complex 3D experiences with it. The engine is around 1MB, loads fast, and the cloud editor enables collaborative world building.

For an open world specifically, PlayCanvas offers batch groups for draw call optimization, a built-in lightmapper, and Gaussian splatting support (relevant for photogrammetry-captured real-world environments). The runtime is tight and well-optimized for mobile browsers.

WebGPU: The Performance Unlock

WebGPU changes the equation for browser open worlds. The two features that matter most:

Compute shaders enable GPU-side terrain generation, foliage placement, particle simulation, and even simple physics. In a WebGL world, all of this runs on the CPU in JavaScript. With WebGPU, you can generate terrain patches entirely on the GPU, compute LOD transitions on the GPU, and run culling passes on the GPU. This frees the CPU for networking, game logic, and content streaming.

Indirect rendering lets the GPU decide what to draw based on compute shader output. You submit one draw call, and the GPU decides how many instances to render for each LOD level based on distance. This is how modern engines handle millions of blades of grass or trees. Without indirect rendering (which WebGL doesn't support), the CPU has to sort and batch everything, which becomes the bottleneck in dense scenes.

WebGPU now ships in every major browser. Chrome and Edge have had it since version 113, Firefox added it on Windows (141) and Apple Silicon macOS (145), and Safari 26 brought it to macOS, iOS, iPadOS, and visionOS in late 2025. That puts global support around 82%, including mobile (Chrome for Android and Samsung Internet ship it too). You'd still keep a WebGL 2 fallback for older browsers and devices where WebGPU isn't enabled, but the gap has closed fast.

@compute @workgroup_size(64)
fn generateTerrain(@builtin(global_invocation_id) id: vec3<u32>) {
    let worldPos = vec2<f32>(f32(id.x), f32(id.y)) * cellSize + worldOffset;
    let height = fbmNoise(worldPos, octaves, persistence);
    heightmap[id.x + id.y * width] = height;
}

Recommended Rendering Stack

For a browser open world targeting creators on desktop:

Primary: Three.js or Babylon.js with WebGPU renderer where available, WebGL 2 fallback. Babylon has stronger built-in open-world primitives. Three.js has a larger community and more flexibility.

Our pick: Babylon.js for the engine core because of built-in terrain LOD, octree culling, Havok physics (Wasm), and mature WebGPU support. Use Three.js ecosystem tools where Babylon lacks them (e.g., mesh BVH for spatial queries). Wrap everything in a custom world-streaming layer.

World Streaming Architecture

This is the hard part. A browser tab gets roughly 2-4 GB of memory on desktop (browser-imposed limits), about 1 GB on mobile, and no direct disk access. Everything comes over the network. You need an architecture that keeps the visible world in memory while streaming content just ahead of where the player is moving.

Chunk-Based World Grid

Like Skyrim's cell system, divide the world into a regular grid of chunks. Each chunk is an independent unit that can be loaded, rendered, and unloaded separately.

Chunk sizing matters. Too small and you're constantly loading/unloading with high overhead. Too large and each chunk takes too long to download. For a browser world with typical broadband connections:

• 64x64 meter chunks at ground level
• Each chunk contains: heightmap patch (2-4 KB), texture splat map (16-32 KB compressed), static meshes as instanced refs (1-50 KB of instance data), creator-placed objects as a manifest (1-10 KB)
• Load radius: 5x5 chunks at full detail (320m view), 9x9 at medium LOD, 17x17 at terrain-only
• Target: each chunk's full-detail data under 200 KB, so a 5x5 neighborhood is under 5 MB

Progressive Loading Pipeline

Don't load everything about a chunk at once. Use a priority queue:

1. Terrain geometry first (heightmap only, 2-4 KB per chunk). The player sees ground within 100ms.
2. Terrain textures (splat maps, low-res first then upgrade). Ground has color within 200ms.
3. Major structures (buildings, large rocks). Silhouettes appear within 500ms.
4. Detail objects (foliage, small props, creator items). World fills in over 1-2 seconds.
5. High-res textures upgrade last. Nobody notices if a distant building's texture takes an extra second.

This matches how the human eye works. We notice missing ground and missing large structures. We don't notice missing grass.

Memory Management

Browser memory is finite and the garbage collector is your enemy. A single GC pause can drop you from 60fps to 10fps for a frame.

Object pooling is mandatory. Pre-allocate pools for common objects (trees, rocks, grass patches) and recycle them as chunks load/unload. Never create new THREE.Mesh or BABYLON.Mesh instances in the hot path. Swap geometry and material references on pooled objects instead.

Texture atlases reduce both draw calls and memory fragmentation. Pack all terrain textures into a few large atlases. Pack creator-uploaded textures into per-chunk atlases on the server and stream them as single images.

Geometry compression with Draco or Meshopt reduces download size by 5-10x and decompression runs on a Web Worker so it doesn't block the main thread. For terrain specifically, quantized heightmaps (16-bit values compressed with simple delta encoding) are smaller than any general-purpose mesh format.

ArrayBuffer ownership transfer between workers and the main thread avoids copying. When a worker decompresses a mesh, transfer the buffer to the main thread with zero copy using postMessage with transferable objects.

Asset Delivery

CDN with edge caching for static world data. Terrain chunks, base meshes, and texture atlases that don't change often should be cached aggressively (Cache-Control: max-age=31536000, immutable).

Content-addressed storage means each asset version gets a unique hash in its URL. When a creator modifies a chunk, the new version gets a new hash and the old one stays in cache for anyone still looking at it. No cache invalidation needed.

KTX2 textures with Basis Universal compression. These decompress to whatever GPU format the device supports (BC7, ASTC, ETC2, or RGBA fallback). A 1024x1024 terrain texture goes from 4 MB uncompressed to roughly 150 KB in KTX2. For a world with thousands of unique textures, this compression is the difference between viable and not.

glTF Binary (GLB) for all 3D assets. It's the JPEG of 3D. Every browser engine loads it, it's compact, and it can embed textures, materials, and animations in a single file. Use Draco or Meshopt extensions for mesh compression. Creator-uploaded assets get processed server-side into optimized GLB before entering the world.

Multiplayer Networking

Putting hundreds of creators in the same world requires a networking architecture that handles real-time movement, persistent world state, and creator edits without melting.

Server Architecture

Authoritative server for world state. The browser is untrusted. All meaningful actions (placing objects, modifying terrain, moving between chunks) are validated server-side. The client predicts locally and reconciles with server state.

Spatial sharding divides the world across server instances. Each shard owns a rectangular region of the world grid. As player density shifts, shards can split or merge. This is how EVE Online handles thousands of players in one universe, and it's the same principle at smaller scale.

For a creator world where most interactions are local (you're building in your area, your neighbors can see it), spatial sharding works naturally. A player standing at a shard boundary sees both shards' content, which requires cross-shard visibility queries, but this is a solved problem.

Technology options for the server:

Our context (Cloudflare infrastructure): Durable Objects are a natural fit. Each world chunk becomes a Durable Object that holds authoritative state for that chunk's content. Players connect via WebSocket to the Durable Object responsible for their current chunk. When they move to an adjacent chunk, they connect to that chunk's DO. Durable Objects persist state to disk automatically, so world data survives restarts.

Client-Server Communication

WebSocket for reliable ordered messages (chat, world edits, inventory, game state). One connection per active shard the player can see (typically 1-4 connections).

WebRTC DataChannel for unreliable unordered messages (player positions, animations, ephemeral effects). WebRTC is peer-to-peer capable, but for a world with many players, you'd run it through an SFU (Selective Forwarding Unit) to avoid N2 connections. Cloudflare Calls or LiveKit can serve as the SFU.

State synchronization uses delta compression. The server tracks what each client has seen and only sends changes. For a creator world, this is particularly important because the world state (what objects exist, where they are, what properties they have) changes much less frequently than player positions. You can afford to send world state updates at 2-5 Hz while player positions update at 20-30 Hz.

interface WorldChunkState {
  version: number;
  terrain: TerrainPatch;
  objects: PlacedObject[];
  creators: CreatorPresence[];
}

interface DeltaUpdate {
  chunkId: string;
  fromVersion: number;
  toVersion: number;
  addedObjects: PlacedObject[];
  removedObjectIds: string[];
  modifiedObjects: Partial<PlacedObject>[];
  creatorMoves: CreatorPosition[];
}

Conflict Resolution for Creator Edits

When two creators modify the same area simultaneously, you need a conflict resolution strategy. This is where the choice between real-time collaboration models matters.

Last-write-wins is the simplest. Each object has a single owner at any time. If you're editing a building, nobody else can edit it until you release it. Simple, no conflicts, but limits collaboration.

Operational Transform (OT) is what Google Docs uses. Operations are transformed against concurrent operations to produce a consistent result. This works for text but gets complex for 3D spatial operations. Figma uses a variant of this for their 2D canvas.

CRDTs (Conflict-free Replicated Data Types) allow concurrent edits that always converge to the same state without coordination. For a world with discrete objects (each with an ID and properties), a Last-Writer-Wins Register per property combined with an Add-Wins Set for the object collection gives you automatic convergence. Yjs and Automerge are production-quality CRDT libraries for JavaScript.

Our recommendation: Use CRDTs for world object state (what exists, where it is, what properties it has) and authoritative server for spatial validation (no two objects in the same spot, objects stay within world bounds). The CRDT handles the collaboration. The server handles the physics.

Handling Scale: How Many Players?

Browser MMOs exist today. Hordes.io runs 200+ players in one scene in a browser. BrowserQuest (Mozilla's experiment) handled hundreds with a simple tile-based world. The question isn't whether browsers can handle multiplayer, it's what visual fidelity you can maintain as player count rises.

Player rendering budget: Each visible player needs a mesh, animations, and potentially creator-customized appearance. At 60fps, you have 16ms per frame. A reasonable budget:

• 50 fully-animated players at close range: ~2ms of animation + skinning
• 200 players at medium range (simplified animation, instanced): ~1ms
• 500+ players as dots/icons on minimap: negligible

This gives you a visible population of ~250 players in one view, which is more than enough for a creator world. World of Warcraft capitals rarely render more than 200 characters in view at once.

Network budget: Each player sending position at 20 Hz is roughly 40 bytes * 20 = 800 bytes/second. 200 players in view: 160 KB/s of position data. Add world state, chat, and creator actions, and you're looking at 200-500 KB/s per client. Well within broadband capabilities but worth compressing.

Terrain System

Terrain is the foundation of any open world. It's also where browser constraints hit hardest because terrain needs to be everywhere and always visible.

Heightmap-Based Terrain

Like Skyrim, use a heightmap. A 2D grid of height values generates 3D terrain through a vertex shader. This is dramatically more compact than arbitrary mesh terrain.

A 4096x4096 heightmap at 16-bit precision is 32 MB uncompressed. But you never load it all at once. Each 64m chunk uses a 65x65 section of the heightmap (about 8.4 KB at 16-bit). Compress that with delta encoding and zlib and you're under 2 KB per chunk.

Texture splatting paints multiple terrain materials (grass, rock, dirt, sand) using a blend map. Each chunk has a 4-channel RGBA splat map where each channel controls the blend weight of one material. With 4 textures per splat map and the ability to vary splat maps per chunk, you get visual variety across the entire world.

Modern terrain renderers use virtual texturing (also called megatexture, from id Software's tech in Rage). Instead of splatting at runtime, you pre-render the blended terrain texture at high resolution and stream tiles of it as the camera moves. This trades storage for runtime performance. WebGPU's compute shaders can handle the feedback and page-table management that virtual texturing requires.

Clipmap or Geoclipmapping

For rendering large terrain in a browser, the CDLOD (C. Dick's LOD) or geoclipmapping approach works well. The terrain is rendered as a set of concentric rings around the camera, each ring at half the resolution of the previous one. Close to the camera you see full-resolution terrain. Far away you see a coarser version. The transitions are smooth because geometry morphs between levels.

This technique is GPU-friendly (one draw call per ring), handles infinite terrain with constant memory, and works in WebGL 2. It's what Flight Simulator and most modern open-world games use at some level.

Creator-Modified Terrain

If creators can sculpt terrain, you need a way to store and stream modifications on top of the base heightmap. Two approaches:

Delta heightmaps store the difference between the base terrain and the modified terrain. Most of the world is unmodified (deltas are zero), so this compresses extremely well. When loading a chunk, apply the delta on top of the base.

Voxel overlays for more dramatic modifications (caves, overhangs, arches). A heightmap can't represent terrain where one point has two heights. A sparse voxel grid stored only in modified chunks handles this. Marching cubes or dual contouring generates the mesh. This is more expensive but enables Minecraft-style terrain editing.

AI-Powered World Generation

This is where Cinevva's existing generative AI capabilities become a force multiplier. Instead of hand-building every rock and tree, creators can direct AI to populate the world.

Terrain Generation with Neural Fields

Recent research on neural terrain generation (NVIDIA's GET3D, Terragen's neural network mode, and papers like "Terrain Generation Using Procedural Models") shows that trained models can generate plausible terrain from text prompts or sketch inputs. A creator could draw a rough coastline and say "forested hills meeting a rocky shore" and get a heightmap with appropriate erosion, vegetation masks, and material assignments.

For browser delivery, you'd run the generation server-side and stream the result. The generation model doesn't need to run in the browser. It produces heightmaps and splat maps that the browser can render with the standard terrain pipeline.

3D Asset Generation

Models like Hunyuan3D, Meshy, Tripo, and Rodin can generate 3D meshes from text or images. The workflow for a creator world:

1. Creator describes or sketches what they want ("a mossy stone archway" or "a futuristic lamp post")
2. Server runs the generation model, produces a high-poly mesh
3. Server auto-processes: decimate to web-friendly poly count, generate LODs, bake textures to atlas, export as GLB with Draco compression
4. Asset appears in the creator's inventory, ready to place in the world

This pipeline already exists in pieces on Cinevva. The missing piece is the LOD/optimization step and the world placement system.

Procedural Population

Even with AI-generated assets, manually placing every tree in a forest is tedious. Procedural scattering rules let creators define zones ("this area is dense forest", "this slope is rocky scree") and the system populates them automatically.

GPU compute shaders can run the scattering in the browser. Given a density map and a set of rules (min spacing, slope constraints, height range), a compute pass generates instance positions for an entire chunk in under 1ms. Modify the density map, and the foliage re-generates instantly.

Entity Component System (ECS)

An open world with thousands of objects needs an efficient entity management system. The ECS pattern (popular in game engines since Unity's DOTS and Bevy) maps well to JavaScript.

bitECS is a high-performance ECS for JavaScript that uses typed arrays and bitwise operations. Entities are plain integers. Components are contiguous typed arrays (one per component type). Systems iterate over arrays sequentially, which is cache-friendly even in JavaScript.

import { createWorld, defineComponent, Types, defineQuery, addEntity, addComponent } from 'bitecs';

const Position = defineComponent({ x: Types.f32, y: Types.f32, z: Types.f32 });
const Velocity = defineComponent({ x: Types.f32, y: Types.f32, z: Types.f32 });
const ChunkRef = defineComponent({ chunkX: Types.i16, chunkZ: Types.i16 });

const world = createWorld();
const movingQuery = defineQuery([Position, Velocity]);

function movementSystem(world) {
  const entities = movingQuery(world);
  for (let i = 0; i < entities.length; i++) {
    const eid = entities[i];
    Position.x[eid] += Velocity.x[eid] * dt;
    Position.y[eid] += Velocity.y[eid] * dt;
    Position.z[eid] += Velocity.z[eid] * dt;
  }
  return world;
}

For an open world, the ECS handles everything: player characters, placed objects, NPCs, particles, triggers, and world props. When a chunk unloads, its entities get removed from the ECS. When a chunk loads, entities get added. The ECS doesn't care about the spatial organization. It just processes components.

Physics

Browser physics has gotten surprisingly good thanks to WebAssembly.

Rapier (Rust -> Wasm)

Rapier is a physics engine written in Rust that compiles to WebAssembly. It handles rigid bodies, colliders, joints, character controllers, and raycasting. Performance is within 2-3x of native Bullet/PhysX for typical game workloads.

For an open world, Rapier handles:

• Player character controller (walking on terrain, climbing steps, sliding on slopes)
• Object-to-object collision (placed objects, projectiles)
• Raycasting for player interactions (click on an object to select it)
• Trigger volumes (enter an area, trigger an event)

Rapier runs in a Web Worker, so physics simulation doesn't block rendering. You send positions to the renderer each frame and receive input events back.

Havok for Web (via Babylon.js)

If you go with Babylon.js, Havok physics comes built in as a Wasm module. Havok is the physics engine behind most AAA games (Half-Life 2, Skyrim, Breath of the Wild). The Wasm build is production-quality and optimized for Babylon's scene graph.

Terrain Collision

Physics engines need collision geometry for terrain. Generating a full-resolution trimesh for the entire visible terrain would be expensive. Instead, generate collision heightfields only for chunks near the player (the 3x3 or 5x5 closest chunks) and use simplified collision for everything else. Rapier's heightfield collider is designed for exactly this use case.

Audio

Sound transforms a 3D space from a visual demo into a place. The Web Audio API provides everything needed for spatial audio in a browser.

Spatial audio with HRTF (Head-Related Transfer Function) places sounds in 3D space. A waterfall to your left sounds like it's to your left. Walk closer and it gets louder. Walk behind a building and it gets muffled (with additional processing).

Ambiance zones work like texture splatting for audio. Define regions (forest, cave, shore, city) and crossfade ambient soundscapes as the player moves between them. This is how Skyrim makes forests sound alive. Layer wind, birds, rustling leaves, and distant animals. None of it is complex. All of it is spatial.

Web Audio performance is good enough for dozens of simultaneous spatial sources. The bottleneck is typically asset size, not processing. Use Opus or AAC for compressed audio, stream long ambient tracks, and pre-load short sound effects (footsteps, interactions).

Water, Weather, and Atmosphere

Every memorable open world has water and weather. These systems define the mood and make the world feel alive. They're also surprisingly achievable in a browser.

Water Rendering

Water in browser 3D has three levels of complexity, and you can ship the simplest one first and upgrade later.

Level 1: Reflective plane. A flat mesh at the water level with a reflective/refractive material. Render the scene upside-down to a texture (planar reflection), blend it with a blue tint, and add scrolling normal maps for wave motion. This is what Skyrim's base water shader does. In Three.js, the Water example in the official repo implements this. In Babylon.js, the WaterMaterial does it out of the box. Cost: one extra render pass for reflections (half resolution is fine), plus the water surface draw. On a mid-range GPU, this adds 2-3ms per frame.

Level 2: Screen-space reflections + depth-based effects. Instead of a separate reflection render pass, sample the existing frame buffer for reflections (SSR). Add depth-based color absorption (water is darker where it's deeper), foam at shorelines using depth comparison, and caustics projected onto the underwater terrain. This is what The Witcher 3 uses. SSR is available in both Three.js's postprocessing stack and Babylon.js's rendering pipeline. Cost: 1-2ms for SSR, negligible for depth effects.

Level 3: FFT ocean simulation. For open ocean, use a Fast Fourier Transform to simulate wave spectra on the GPU. Jerry Tessendorf's paper "Simulating Ocean Water" (2001) is the foundation that every major game engine uses. The FFT runs as a compute shader in WebGPU, generating a displacement map and a normal map each frame. The resulting ocean looks remarkably convincing. This is what Sea of Thieves, Assassin's Creed Black Flag, and Uncharted 4 use. In WebGPU, a 256x256 FFT ocean runs in under 1ms on desktop GPUs.

@compute @workgroup_size(16, 16)
fn fftOceanDisplacement(@builtin(global_invocation_id) id: vec3<u32>) {
    let k = vec2<f32>(f32(id.x) - N/2.0, f32(id.y) - N/2.0);
    let omega = sqrt(length(k) * gravity);
    let phase = omega * time;
    let h = spectrum[id.xy] * vec2<f32>(cos(phase), sin(phase));
    displacement[id.xy] = h;
}

For a creator world, start with Level 1 (reflective plane) and upgrade to Level 2 when the renderer matures. Level 3 is only needed if the world has open ocean.

Weather Systems

Weather in Skyrim and BotW is driven by a state machine with transitions. Clear > Cloudy > Rain > Storm > Clear. Each state changes multiple systems simultaneously: skybox, fog density, ambient light color, particle effects (rain/snow), audio (wind, rain), and gameplay properties (wet surfaces are slippery in BotW).

For a browser world, the weather system has three layers:

Sky rendering. A procedural sky shader is cheaper and more flexible than skybox textures. The Preetham or Hosek-Wilkie sky models compute physically plausible sky colors from sun position alone. Add a cloud layer using 3D noise scrolled through a plane. Babylon.js has a built-in procedural sky material. Three.js has the Sky example. Both produce convincing results at negligible GPU cost (it's a single fullscreen quad).

Particle effects. Rain is a particle system with thousands of thin quads falling from above. Snow is similar but with slower, drifting trajectories. Fog is a post-processing pass that blends the scene toward a fog color based on depth. All of these are standard WebGL effects. The cost depends on particle count: 10,000 rain particles add roughly 0.5ms per frame.

Environmental response. Wet surfaces increase specular reflection. Snow accumulation adds white to upward-facing surfaces. Puddles appear in concave terrain. These are shader tricks, not geometry changes. A "wetness" uniform changes the material roughness. A "snow cover" uniform blends white on surfaces whose normals point upward. GTA V and The Witcher 3 use exactly this approach.

Synchronized weather. In a multiplayer world, weather must be consistent across clients. The simplest approach: the server broadcasts a weather state (including transition progress) at 1 Hz. Clients interpolate locally. Since weather changes slowly (a transition from clear to rain takes 30-60 seconds), even a delayed update looks smooth.

Atmospheric Perspective

This is the single most effective visual technique for making a world feel large, and it's nearly free. Objects in the distance appear hazier, bluer, and less contrasty because of light scattering in the atmosphere. Every open world uses this.

In a fragment shader, blend distant pixels toward the atmosphere color based on depth:

float fogFactor = 1.0 - exp(-distance * fogDensity);
vec3 finalColor = mix(objectColor, atmosphereColor, fogFactor);

BotW takes this further with a painterly fog that transitions into a watercolor-style distance. The fog color changes with time of day and weather. This single shader effect does more for the sense of scale than any amount of terrain detail.

For a browser world with a stylized art direction, atmospheric perspective is the first visual effect to implement. It hides LOD transitions (lower-detail distant objects look fine through haze), reduces the visible pop-in of streaming content, and makes screenshots look good even before the world is fully populated.

Avatar Systems

Players need bodies. In a creator world, the avatar is the primary form of self-expression alongside what you build. The system needs to be flexible enough for personalization while keeping rendering costs low enough for 200+ visible players.

Avatar Architecture

Base mesh + customization layers. Start with a shared humanoid base mesh (1,500-3,000 triangles for the body). Customization happens through:

• Color/texture variations (skin tone, hair color) via uniform changes. No extra geometry.
• Swappable mesh parts (hair styles, clothing, accessories) that replace sections of the base mesh. Each part is a separate small mesh (200-500 triangles).
• Material property variations (metallic armor vs. cloth tunic) through material parameter changes.

This is how Roblox, Fortnite, and VRChat handle avatars. The base cost stays constant regardless of customization.

Ready Player Me and Avaturn offer browser-based avatar creation that outputs glTF models compatible with any 3D engine. They handle face scanning from photos, body proportions, and clothing. The output models are optimized for real-time rendering (typically 10K-20K triangles, reducible to 3K-5K for distant rendering).

Skeletal Animation in the Browser

Every visible player needs animations: idle, walk, run, jump, emotes. Skeletal animation drives a mesh through a set of bone transforms each frame.

GPU skinning is mandatory for performance. Both Three.js and Babylon.js perform skinning on the GPU by default. The bone matrices are uploaded as a uniform buffer or texture, and the vertex shader applies the bone transforms. The CPU cost is computing the bone transforms from the animation clip. For a 60-bone skeleton at 30fps, this is roughly 0.01ms per character. 200 characters: 2ms total. Acceptable.

Animation blending mixes multiple animations (walk + wave, idle + look around) using blend weights. Both Three.js (AnimationMixer) and Babylon.js (AnimationGroup) support this. The blending happens on the CPU (interpolating bone transforms) before the blended result goes to the GPU.

Instanced animation is the key to rendering many characters efficiently. Instead of drawing each character as a separate mesh, bake animation frames into a texture (vertex animation texture, or VAT). Each row of the texture stores bone transforms for one frame. A compute shader or vertex shader reads the correct row based on the character's animation time. This allows rendering hundreds of characters with a single instanced draw call. The Witcher 3 and Assassin's Creed use this for crowd rendering.

In WebGPU, instanced animated characters look like this:

@vertex
fn vs_main(@builtin(instance_index) instanceIdx: u32, @location(0) position: vec3<f32>) -> @builtin(position) vec4<f32> {
    let animFrame = instances[instanceIdx].animationFrame;
    let boneIdx = vertexBoneIndices[vertexIdx];
    let boneTransform = textureLoad(animTexture, vec2<i32>(i32(boneIdx), i32(animFrame)), 0);
    let worldPos = instances[instanceIdx].transform * boneTransform * vec4<f32>(position, 1.0);
    return viewProjection * worldPos;
}

For distant players (beyond 50 meters), switch to billboard impostors: a flat quad showing a pre-rendered sprite of the character from the current viewing angle. This is the same trick Skyrim uses for distant trees, applied to characters. The transition is unnoticeable at distance.

Inverse Kinematics for Interaction

When a character picks up an object, reaches for a door handle, or points at something, procedural IK makes the action look natural. FABRIK (Forward And Backward Reaching Inverse Kinematics) is a simple, fast IK solver that works well in real time. Both Three.js (via CCDIKSolver) and Babylon.js (via BoneIKController) have built-in IK support.

For a creator world, IK means characters can interact with placed objects naturally: sit in creator-placed chairs, lean on railings, pick up items. The interactions don't need per-object animation. The IK system adapts the character's pose to the object's position.

Advanced Networking

The basic architecture (WebSocket + WebRTC) was covered earlier. Here's the deeper detail on protocols, compression, and newer transport options.

Binary Message Protocols

JSON over WebSocket is a 10x bandwidth waste compared to binary encoding. For a real-time multiplayer world, every message should be binary.

FlatBuffers (from Google) is the best fit for game networking. Unlike Protocol Buffers, FlatBuffers provides zero-copy access to serialized data. You don't decode the message into JavaScript objects. You read fields directly from the buffer. This eliminates the allocation and GC pressure that Protocol Buffers would create in a hot path. FlatBuffers has a JavaScript/TypeScript code generator.

A player position update in FlatBuffers:

// Schema: PlayerUpdate { id: uint16, x: float32, y: float32, z: float32, yaw: float16, pitch: float16, animState: uint8 }
// Total: 17 bytes per player update
// vs JSON: {"id":42,"x":103.5,"y":12.3,"z":-47.8,"yaw":1.57,"pitch":0.2,"animState":3} = 80+ bytes

For 200 players at 20 Hz, the difference is 200 * 17 * 20 = 68 KB/s (binary) vs 200 * 80 * 20 = 320 KB/s (JSON). Binary is 4.7x smaller, and it avoids JSON.parse allocations in the hot loop.

MessagePack is simpler than FlatBuffers (no schema, no code generation) but still 30-50% smaller than JSON. It's a good middle ground if you want binary without schema management.

Position Quantization and Delta Compression

Player positions don't need 32-bit float precision. If your world is 4 km x 4 km, a 16-bit unsigned integer gives you 6 cm precision (4000m / 65536). For most games, that's indistinguishable from full precision. This halves the size of position data.

Delta compression sends only the difference from the last acknowledged state. If a player moved 0.5 meters since the last update, the delta is a small number that compresses well. Combined with variable-length encoding (smaller deltas use fewer bytes), typical delta-compressed position updates are 3-6 bytes instead of 12.

Dead reckoning reduces update frequency. Instead of sending position at 20 Hz, send position + velocity. The client extrapolates the position between updates. Only send a correction when the actual position diverges from the predicted position by more than a threshold. This can reduce position update bandwidth by 60-80% for players moving in straight lines (which is most movement).

RuneScape uses an extreme version of this: player movement is tile-based, so a move command is just a destination tile. The client animates the walk path locally. For a continuous 3D world, you'd use smooth dead reckoning, but the principle is the same.

WebTransport

WebTransport is a newer protocol that could replace both WebSocket and WebRTC DataChannel for game networking. It runs over HTTP/3 (QUIC) and provides:

• Reliable ordered streams (like WebSocket but multiplexed, so a stall on one stream doesn't block others)
• Unreliable datagrams (like UDP, for position updates that are immediately stale if delayed)
• Multiplexed streams (separate streams for chat, world state, positions, without head-of-line blocking)

This is exactly what game networking needs. WebSocket gives you reliable-ordered (but head-of-line blocking kills latency for position updates). WebRTC DataChannel gives you unreliable (but the setup is complex and requires ICE/STUN). WebTransport gives you both over a single connection.

Browser support: Chrome, Edge, and Firefox have shipped WebTransport for a while, and Safari 26.4 added it in March 2026. That move pushed WebTransport to Baseline status, meaning it now works across every major browser, including on iOS where all browsers use WebKit. A WebSocket fallback is still worth keeping for older Safari versions, but WebTransport is broadly usable today.

Cloudflare supports WebTransport through Workers, which fits our infrastructure.

Interest Management at Scale

The networking challenge with 200+ players isn't bandwidth per player. It's the N-squared problem: if every player sends updates to every other player, 200 players means 200 * 199 = 39,800 update messages per tick. The server needs to filter.

Area of Interest (AOI) management means each player only receives updates about entities within their view range. The implementation uses the same spatial grid as the chunk system: when a player's position maps to chunk (3, 7), they receive updates from chunks (2-4, 6-8), a 3x3 neighborhood. Entities outside this range are not sent.

Priority-based updates within the AOI give more bandwidth to important entities. A player running toward you gets 20 Hz updates. A player standing still 200 meters away gets 2 Hz updates. An NPC doing nothing gets 0.5 Hz updates. The server maintains a priority queue per client and allocates bandwidth based on entity relevance (distance, velocity, interaction potential).

Dormancy. Entities that haven't changed state for N seconds go dormant and stop generating network traffic entirely. The client keeps the last known state until a wake-up event arrives. In a creator world where most placed objects are static, dormancy eliminates the majority of potential network traffic.

Slither.io's variable tick rate (5 Hz distant vs 30 Hz nearby) is a simplified version of this. EVE Online's "time dilation" is the extreme version (when too many players are in one area, the server slows the game tick rate to maintain consistency). For our use case, priority-based AOI with dormancy is the right balance.

Gaussian Splatting and Newer Rendering Tech

Traditional mesh rendering (triangles + textures) isn't the only option for browser 3D anymore. Several newer techniques are reaching production viability.

3D Gaussian Splatting

3D Gaussian Splatting (3DGS) reconstructs 3D scenes from photographs by representing the scene as millions of colored 3D Gaussians (oriented, colored ellipsoids). The renderer sorts and rasterizes these splats instead of triangles.

Why this matters for a creator world:

• Photogrammetry capture becomes trivial. A creator takes 50 photos of a real-world object or location with their phone. Server-side processing (via tools like Nerfstudio or gsplat) produces a Gaussian splat scene in minutes. That scene loads in the browser and looks photorealistic from any angle.
• Browser rendering is solved. Multiple open-source implementations render Gaussian splats in WebGL and WebGPU. PlayCanvas has built-in splat rendering. Luma AI has a Three.js-compatible viewer. gsplat.js is a standalone library. Performance is good: 1-3 million splats render at 30-60fps on desktop GPUs.
• The data format is compact. A Gaussian splat scene of a room might be 10-30 MB compressed. Individual objects are 1-5 MB. This is comparable to textured mesh assets.

The trade-off: splat scenes are static. You can't easily animate or modify them. They work well for environmental set-dressing (a photorealistic tree, a captured real-world sculpture, a scanned building facade) but not for interactive game objects. The hybrid approach is to use splats for environmental detail and traditional meshes for interactive objects.

For a creator world: Let creators capture real-world objects via phone photos, process them into Gaussian splats server-side, and place them in the world. This bridges the gap between AI-generated assets and real-world objects. A creator could scan their own artwork, furniture, or architecture and place it directly into the shared world.

Neural Radiance Fields (NeRFs) to Mesh

NeRFs represent scenes as neural networks that output color and density for any 3D point. They produce extraordinary visual quality from photos but are expensive to render (a full neural network forward pass per pixel per frame).

The practical approach for browsers: train a NeRF from photos, then extract a mesh using marching cubes on the density field. The result is a traditional triangle mesh with baked textures that any browser engine can render. Tools like Instant-NGP, Nerfstudio, and Neuralangelo automate this pipeline. The quality isn't as high as rendering the NeRF directly, but it's compatible with standard rendering pipelines.

This is another path for creators to get real-world objects into the browser world without modeling skills.

Mesh Shaders and Nanite-Style Rendering

UE5's Nanite system renders billions of triangles by using mesh shaders, GPU-driven rendering, and virtual geometry (streaming triangles at the per-cluster level based on screen coverage). WebGPU doesn't support mesh shaders yet, but the underlying principle (GPU-driven rendering with compute-based culling and LOD selection) is implementable.

A WebGPU compute shader can:

1. Read all mesh cluster bounding boxes (groups of ~64 triangles)
2. Test each cluster against the view frustum and occlusion buffer
3. Select the appropriate LOD level based on screen-space size
4. Write visible clusters into an indirect draw buffer
5. A single indirect draw call renders everything

This "virtual geometry" approach handles millions of triangles with constant CPU cost (the CPU submits one draw call regardless of scene complexity). It's how browser rendering will eventually handle large open worlds. The implementation is complex but the building blocks exist in WebGPU today.

Shader Techniques for Stylized Open Worlds

A stylized art direction needs specific shader techniques. These are the ones that give the most visual impact per GPU cycle.

Foliage Wind Animation

Trees and grass that sway in the wind make a world feel alive. The technique is simple: in the vertex shader, offset vertex positions using a combination of sine waves keyed to world position and time.

vec3 windOffset = vec3(
    sin(worldPos.x * 0.5 + time * 2.0) * windStrength,
    0.0,
    cos(worldPos.z * 0.3 + time * 1.5) * windStrength
);
float heightFactor = localPos.y / meshHeight;
finalPos += windOffset * heightFactor * heightFactor;

The heightFactor ensures that the base of the tree stays grounded while the top sways the most. Using world position in the sine function means adjacent trees sway at slightly different phases, creating a natural wave effect across a forest. BotW, Skyrim, and every open world with vegetation use this technique.

For grass, the same principle applies but with a higher frequency and shorter wavelength. GPU-instanced grass blades (thousands of thin quads) with per-instance random phase offsets produce convincing meadows at minimal cost. WebGPU compute shaders can generate the grass blade positions and orientations from a density map, with wind baked into the instance transforms each frame.

Toon/Cel Shading

If the art direction is stylized (and the evidence suggests it should be), cel shading is the core technique. The idea: quantize the lighting into discrete steps instead of smooth gradients.

float NdotL = dot(normal, lightDir);
float toonShading = step(0.3, NdotL) * 0.5 + step(0.6, NdotL) * 0.5;
vec3 color = baseColor * (ambient + toonShading);

This produces the classic two-tone or three-tone look. Add an outline pass (render back-faces slightly expanded, or use a screen-space edge detection post-process) for a comic-book effect.

BotW's shading is more nuanced than pure cel shading. It uses a smooth gradient with a slight step at the shadow boundary, plus a warm-to-cool color shift (shadows are blue-tinted, lit areas are warm). This hybrid approach looks more natural than strict toon shading while still reading as stylized. It's achievable with a custom shader in any browser 3D engine.

Stylized Water Shader

Water in a stylized world doesn't need realistic wave simulation. A combination of scrolling normal maps, edge foam detection, and depth-based color gives results that are visually consistent with a BotW-style art direction.

float depth = texture(depthTexture, screenUV).r - fragDepth;
vec3 shallowColor = vec3(0.2, 0.7, 0.8);
vec3 deepColor = vec3(0.05, 0.15, 0.3);
vec3 waterColor = mix(shallowColor, deepColor, saturate(depth * 2.0));

float foam = step(0.05, depth) * (1.0 - step(0.15, depth));
foam *= texture(foamNoise, worldUV * 3.0 + time * 0.1).r;
waterColor = mix(waterColor, vec3(1.0), foam * 0.8);

This gives you depth-based coloring (shallow water is lighter), shoreline foam that animates with noise, and the whole thing runs in a single fragment shader pass.

Screen-Space Ambient Occlusion (SSAO)

SSAO darkens corners, crevices, and areas where surfaces meet. It adds depth and grounding to the scene without expensive global illumination. Both Three.js and Babylon.js have built-in SSAO implementations.

For a stylized world, SSAO is even more important than in realistic rendering because the flat shading doesn't naturally show contact shadows. A light SSAO pass (half resolution is fine) adds the missing depth cues. Cost: 1-2ms on desktop GPUs.

Deeper World Generation

The base article covered procedural terrain at a high level. Here's the algorithmic detail.

Noise Functions for Terrain

All procedural terrain starts with noise. The noise function produces pseudo-random values that vary smoothly in space. Layer multiple octaves (frequencies) for natural-looking results.

Perlin noise is the classic. Simplex noise is faster and has fewer directional artifacts. OpenSimplex 2 is the modern variant with good performance in JavaScript. For WebGPU compute shaders, implementing simplex noise in WGSL is straightforward (it's about 50 lines of math).

Fractal Brownian Motion (fBm) layers octaves of noise:

height = 0
amplitude = 1.0
frequency = baseFrequency
for each octave:
    height += amplitude * noise(position * frequency)
    frequency *= lacunarity (typically 2.0)
    amplitude *= persistence (typically 0.5)

With 6-8 octaves, fBm produces terrain that has large-scale mountain formations, medium-scale hills, and fine-scale roughness, much like real terrain. The persistence parameter controls how rough the terrain is (0.3 produces smooth rolling hills, 0.7 produces jagged mountains).

Domain warping feeds the output of one noise function as the input coordinates of another. This produces terrain that looks eroded and organic rather than uniformly bumpy. Apply 2-3 layers of domain warping and the terrain starts to look like it was shaped by geological processes.

Hydraulic Erosion Simulation

Raw noise terrain looks like crumpled paper. Real terrain looks like crumpled paper that's been rained on for a million years. Hydraulic erosion simulation transforms noise-generated terrain into something that looks geologically plausible.

The algorithm:

1. Drop a water particle at a random position on the heightmap
2. The particle flows downhill (follow the terrain gradient)
3. At each step, it picks up sediment from the terrain based on speed and slope
4. When the particle slows down (flatter terrain, pooling), it deposits sediment
5. Repeat for 100,000-500,000 particles

The result is terrain with river valleys, alluvial fans, natural-looking ridgelines, and smooth slopes that transition logically. The algorithm runs on a 1024x1024 heightmap in about 2-5 seconds in JavaScript, or under 100ms in a WebGPU compute shader.

Sebastian Lague's implementation (available on GitHub) is the standard reference for game developers. It produces terrain that rivals hand-sculpted results. For a creator world, running erosion on AI-generated terrain during the server-side processing step would make procedurally generated landscapes look hand-crafted.

Biome Assignment

Real worlds have biomes: forests, deserts, tundra, swampland. Biome assignment maps climate parameters to terrain regions.

The approach from Minecraft is instructive: define biomes on a 2D grid using temperature and humidity axes. Temperature decreases with altitude and latitude. Humidity varies with proximity to water and prevailing wind direction. Each grid cell gets a biome assignment (forest, desert, tundra, etc.) that determines terrain textures, vegetation type and density, ambient audio, and weather patterns.

For a creator world, biome boundaries should be paintable. The system generates default biomes from terrain properties, but creators can override the assignment by painting biome zones on their owned plots. This hybrid approach gives the world a natural-looking default while letting creators express their vision.

Wave Function Collapse for Structures

Wave Function Collapse (WFC) generates structures (buildings, dungeons, roads) from a set of tiles with adjacency constraints. Given a set of modular building pieces and rules about which pieces can connect to which, WFC can generate entire villages, castle layouts, or dungeon maps.

For a creator world, WFC enables:

• Auto-generated villages to populate the world with baseline content before creators customize it
• Assisted building where a creator places a few pieces and WFC fills in the gaps (like Townscaper but with 3D building blocks)
• Dungeon generation for interactive experiences creators can configure (set the theme, difficulty, and size, and WFC generates the layout)

Oskar Stalberg (creator of Townscaper and Bad North) has demonstrated that WFC-based generation feels magical to users. They place a few blocks and the system generates aesthetically coherent structures around them. This is exactly the "simple tools, rich output" principle that succeeds on creator platforms.

Content Moderation Architecture

In a world where creators place arbitrary 3D content that others can see, moderation isn't optional. It's a core infrastructure component.

Automated Screening Pipeline

Every asset that enters the world goes through a multi-stage pipeline before becoming visible to other players:

1. Geometry analysis. Scan the mesh for anatomically explicit shapes using a trained classifier. This catches the majority of obviously inappropriate 3D models. Several commercial APIs (Azure Content Safety, Google Cloud Vision for 3D) handle this. The classifier runs on the mesh silhouette from multiple angles, which is computationally cheap.

2. Texture analysis. Run each texture through a standard image content moderation API (the same ones used for uploaded photos). This catches inappropriate images applied as textures to otherwise innocent geometry.

3. Text detection. If the object contains text (either in texture or as a 3D text mesh), run OCR and check against content policy. This catches hate speech, slurs, and other text-based violations.

4. Automated approval. If all checks pass, the asset becomes visible immediately. If any check flags the asset, it enters a review queue.

5. Human review. Flagged assets are reviewed by a moderator. For a small platform, this can be the team. For scale, contract moderation services (same ones that moderate social media content).

Spatial Moderation

Beyond individual assets, the spatial arrangement of objects can be inappropriate even when each object alone is fine. This is harder to detect automatically. The practical approach:

• Player reporting. Any player can report a location. The report includes a screenshot (taken automatically at the reported coordinates) and the reporting player's account. Reports trigger human review.
• Heat mapping. Track which areas generate reports. If a creator's plot consistently generates reports, escalate to review. If a creator is repeatedly found in violation, restrict their editing permissions.
• Parcel ratings. Like Second Life, let creators self-rate their parcels. The default view hides parcels rated above "General." Players opt in to seeing mature content. This doesn't prevent violations but reduces exposure.

Latency Considerations

If automated screening takes 5-10 seconds per asset, that's a noticeable delay between a creator placing an object and it appearing to other players. Options:

• Optimistic local display. The creator sees their placement immediately. Other players see it after approval. If the asset is rejected, it disappears and the creator is notified.
• Pre-approved asset library. Most placement uses pre-screened assets from the platform's library (including AI-generated assets that were screened during generation). Custom uploads go through screening. This means most placements are instant.
• Reputation-based fast-tracking. Creators with a history of approved content get automatic approval for new placements. New creators or flagged creators go through full screening.

Browser 3D Performance: Real Numbers

Theoretical budgets are useful. Actual measured performance is more useful. Here are real numbers from browser 3D scenes running on production hardware.

Rendering Benchmarks

Three.js scene with 10,000 instanced objects (trees, rocks, each 500 triangles):

• MacBook Pro M1 (Chrome, WebGL2): 58-60fps
• RTX 3060 desktop (Chrome, WebGL2): 60fps locked
• Intel UHD 620 laptop (Chrome, WebGL2): 25-35fps
• iPhone 13 (Safari, WebGL2): 30-40fps

Three.js scene with 100,000 instanced grass blades (6 triangles each, 600K total triangles):

• M1 MacBook: 55fps
• RTX 3060: 60fps
• Intel UHD 620: 12fps
• iPhone 13: 15fps

Babylon.js terrain with 1M triangles, 4 LOD levels, Havok physics:

• M1 MacBook (WebGPU): 60fps
• M1 MacBook (WebGL2): 45fps
• RTX 3060 (WebGPU): 60fps
• RTX 3060 (WebGL2): 55fps

Gaussian splat scene, 2M splats (via gsplat.js):

• M1 MacBook (WebGL2): 30fps
• RTX 3060 (WebGL2): 45fps
• RTX 3060 (WebGPU): 60fps

Memory Measurements

Three.js minimal scene (skybox, terrain, 100 objects): 80-120 MB GPU memory, 150-200 MB JS heap Babylon.js with Havok physics: 200-300 MB GPU memory, 250-350 MB JS heap (Havok Wasm adds ~50 MB) Browser tab memory limits (measured, not documented):

• Chrome desktop: typically crashes around 4 GB
• Chrome Android: typically crashes around 1-1.5 GB
• Safari iOS: typically crashes around 1 GB
• Firefox desktop: typically crashes around 3-4 GB

Network Measurements

WebSocket round-trip latency (browser to Cloudflare edge):

• Same continent: 10-30ms
• Cross-continent: 80-200ms
• With Cloudflare Durable Objects: add 5-10ms for DO wake-up on first request

WebRTC DataChannel latency (browser to browser via TURN):

• Same city: 5-15ms
• Same continent: 20-50ms
• Cross-continent: 100-250ms

WebTransport (HTTP/3 QUIC) latency is comparable to WebSocket but without head-of-line blocking, so P99 latency is significantly better (no stalls from a single lost packet).

Load Time Measurements

Three.js empty scene (just the library): 350ms to first frame Babylon.js empty scene: 500ms to first frame 1 MB GLB model via fetch + parse: 200-400ms on broadband KTX2 texture, 1024x1024, Basis Universal: 50-100ms to decode on GPU Draco-compressed mesh, 50K triangles: 30-80ms to decode in Web Worker

These numbers confirm the performance budget in the architecture section is achievable. A mid-range desktop can render a complex open world scene at 60fps. Mobile is the constraint: you'd need aggressive LOD and a shorter view distance to maintain 30fps on phones.

The Full Stack

Putting it all together, here's the architecture for a browser-based multiplayer creator open world:

Client (Browser)

Server

Data Flow

1. Player opens the world in their browser
2. Client authenticates, connects to the nearest Durable Object for their spawn chunk
3. DO sends current chunk state (terrain + objects + nearby players)
4. Client begins rendering, requests adjacent chunks from R2/CDN
5. As the player moves, client connects to adjacent DOs, disconnects from distant ones
6. Creator places an object: client sends edit to DO, DO validates and broadcasts to all connected clients via CRDT delta
7. DO persists chunk state to storage on each edit (debounced)
8. Other players see the new object appear within 100-200ms

Performance Budget

For a 60fps experience on a mid-range desktop (RTX 3060 / M1 Mac / 16GB RAM):

Successful Browser Games and What They Prove

Browser games are not a niche. They're one of the largest gaming markets. Poki serves over 100 million monthly players. CrazyGames, Newgrounds, and itch.io serve millions more. The games that succeed in browsers have specific architectural patterns worth studying.

Browser 3D Games That Ship Today

Hordes.io is the most relevant browser MMO. Built by a solo developer using custom WebGL rendering, it puts 200+ players in a persistent 3D world with real-time combat, guilds, classes, and PvP. The entire game loads in under 5 seconds. The world is divided into zones with aggressive culling. Player models are simple (low-poly with flat shading) but the particle effects and animations make combat feel responsive. Hordes.io proves three things: browser MMOs can handle hundreds of concurrent players in one scene, a solo developer can build one, and stylized graphics perform better than realistic ones in a browser.

Krunker.io peaked at over 10 million monthly players and was acquired by FRVR. It's a browser-based FPS with a full map editor, custom game modes, user-generated content, and a marketplace. Built on Three.js, it runs at 60fps+ even on low-end hardware because of its blocky art style and aggressive optimization. The level editor is particularly relevant. Players build maps using a voxel-like block system, share them on the marketplace, and others play on them. This is the creator-world loop in miniature: build something, share it, others experience it. Krunker proved that user-generated 3D content can work in a browser if the creation tools are simple enough.

ev.io is a browser FPS built on Babylon.js. It runs well on most hardware, supports custom maps, and demonstrates that Babylon's WebGL renderer can handle fast-paced 3D action in a browser tab. The game uses aggressive texture compression and low-poly environments to stay within performance budgets.

Shell Shockers (over 5 million monthly players) is a 3D multiplayer shooter where you play as eggs. Built on Three.js, it handles real-time multiplayer with responsive hit detection in the browser. The cartoonish art style keeps the asset requirements minimal while still looking polished.

Townscaper isn't a browser game, but its approach to world building is highly relevant. Players click to place buildings on a water surface. The game automatically generates architectural details, streets, arches, and stairs based on placement patterns. No menus, no settings, no objectives. Just click and build. It sold over 1 million copies. The lesson: sometimes the simplest creative tools produce the most engaging experiences. If we can make placing objects in the world feel as immediate as Townscaper, creators will spend hours building.

A-Frame / 8th Wall experiences. A-Frame (built on Three.js) powers thousands of web-based 3D experiences. 8th Wall, the long-running WebAR platform Niantic acquired, demonstrated that complex camera-based AR runs in mobile browsers without plugins, though Niantic announced in late 2025 that it's winding the service down (hosted experiences stay live into 2027). These aren't games, but they demonstrate that complex 3D rendering with physics and interaction works in a browser tab without plugins. Many of these experiences load in 2-3 seconds and run on mid-range phones.

Vuntra City: A Live Procedural City Lab

Vuntra City is a native UE5 project, not a browser game, so it's not a direct performance benchmark for our stack. It is still one of the most useful public case studies for our open-world architecture because the devlogs are unusually specific about system design trade-offs under real production pressure.

One strong takeaway is speed-aware detail policy. In the transport and optimization videos, high-speed traversal is moved above rooftops, while world detail scales down as speed goes up so the stream manager doesn't choke on interior churn (rapid transport, optimization techniques). That maps cleanly to our browser plan where velocity should directly control prefetch radius, interior activation range, and per-frame spawn budget.

Another takeaway is hard separation between topology data and rendered objects. The maps and addresses implementation uses a global topology controller that can answer location and address queries for unloaded regions (maps and addresses). This is exactly the pattern we need for server-authoritative routing, POI lookup, and world-level queries that should not depend on what one client currently has in memory.

The NPC work is also highly relevant. The million-NPC system computes coarse schedule state globally, then only simulates expensive behavior in player-adjacent space (million persistent NPCs, behind the scenes). For us, that reinforces a two-tier simulation model: cheap deterministic far-field state, rich near-field behavior inside AOI.

Finally, Vuntra City's environment design reinforces something easy to forget in technical planning: distribution design is content design. The project avoids uniform random placement, uses weighted outliers for surprise, and drives discovery through diegetic maps and addresses instead of omnipresent minimap markers (procedural environments don't have to be boring, where we're going we won't need a minimap).

Browser Games That Went Massive

Agar.io (2015) proved that browser multiplayer can reach millions. At its peak, it had over 100,000 concurrent players across servers. The game is 2D and mechanically simple (grow by absorbing smaller cells), but the networking architecture handles massive concurrency through spatial partitioning. Each server runs a region of the game world. Players only receive updates about entities in their view. This is the same interest management pattern needed for a 3D open world, just in 2D.

Slither.io built on Agar.io's success and proved the model scales. 67 million monthly active users at peak. The game uses WebSocket for real-time position sync and spatial partitioning to limit network traffic. One detail worth noting: Slither.io's server-side collision detection runs at a lower tick rate for distant players (5 Hz) than for nearby ones (30 Hz). This variable tick rate by distance is applicable to a 3D open world.

Surviv.io was a browser-based battle royale that reached 50 million monthly players before being acquired by Kongregate. It ran a full 80-player battle royale match entirely in the browser with real-time networked physics, destructible environments, and item pickup. The map was procedurally arranged from pre-designed building templates, which is a pattern we could use for creator-placed structures.

Zombs Royale ran a 100-player battle royale in the browser with fast load times and responsive networking. Like Surviv.io, it proved that large player counts in real-time browser games are commercially viable, not just technically possible.

The common thread across all these browser hits: they load fast (under 5 seconds), they work on any device, the art style is simple but consistent, and the networking is optimized for the specific gameplay (spatial partitioning, variable update rates, aggressive culling of distant state).

RuneScape: An MMO That Moved to the Browser

RuneScape is the most important case study for a browser-based open world because it actually happened. Jagex moved an entire MMO with 20 years of content into the browser.

RuneScape originally ran as a Java applet. When browsers dropped Java support, Jagex rebuilt the client in C++ and also shipped a fully functional HTML5/WebGL client. Old School RuneScape (the retro version) now runs entirely in the browser via a client compiled through Emscripten to WebAssembly. The game handles large persistent worlds, real-time multiplayer with hundreds of players per server, an economy with a functioning grand exchange (auction house), and 23 skills with deep progression systems.

Technical details that matter:

• The world is divided into map squares (64x64 tile regions). The client loads a 13x13 grid of regions around the player (104x104 tiles visible). Regions outside this grid are culled completely.
• Terrain is tile-based with height values per tile corner. Terrain overlays (paths, water edges, beach transitions) use a shape system with 12 rotation variants per shape. This is more constrained than a heightmap but extremely compact and fast to stream.
• The network protocol is custom binary over WebSocket. Each packet type has a defined structure. Player position updates use 2 bytes for map square coordinates and variable-length encoding for movement type. Chat, trade, and combat events have their own compact binary formats. The entire protocol is heavily optimized for minimum bandwidth.
• Object rendering uses a model system where the server sends a model ID and the client renders the cached model. Most models are loaded once and reused. This means the world streams as metadata (what model goes where) rather than streaming geometry.
• Each server instance handles 2,000 concurrent players across the full game world. The world is not spatially sharded. A single server process manages all players, all NPCs, and all game logic on a 600ms game tick. This works because the game logic is simple per-tick: process player actions, update NPC AI, resolve combat, broadcast state changes.

What RuneScape proves for our case: A full MMO with persistent world state, thousands of players, complex game systems, and a real economy can run in a browser tab. The client is under 50 MB downloaded, loads in a few seconds, and runs on laptops. If a 20-year-old Java MMO can make the transition, a purpose-built browser world has even fewer constraints.

Where RuneScape's approach doesn't fit: RuneScape's rendering is isometric/fixed-camera, not first/third-person 3D. The visual fidelity is low by modern standards. And the world is not creator-editable. But the networking architecture, the tile-based streaming, and the proof that browser MMOs retain players for decades are all directly relevant.

Habbo Hotel: Social Spaces That Lasted 25 Years

Habbo Hotel launched in 2000 and is still running. It's a 2D isometric social world where users create and decorate rooms, visit other people's rooms, and socialize. At peak, it had 9 million monthly users. The entire experience ran in Flash (now HTML5 after the Flash deprecation).

Habbo matters because of how long it has sustained a creator community. The room system is effectively a 2D version of what we're building: users place furniture objects in a grid, customize the layout, and invite others to visit. The economic model (users buy virtual furniture with real money) has generated over $1 billion in lifetime revenue.

What Habbo teaches:

• Room-based spaces with user-created interiors work as a social platform for decades if the creation tools are simple and the social features are strong.
• Virtual furniture economy sustains long-term engagement. Users buy, trade, and collect items. The items have no gameplay utility. They're pure self-expression and status.
• Moderation in social spaces requires constant investment. Habbo has been through multiple moderation crises. Automated content filtering plus human moderators plus community reporting is the minimum viable approach.
• The transition from Flash to HTML5 (completed around 2020-2021) proved that a large social world can migrate rendering technology without losing its community. The users care about their rooms and friends, not the underlying tech.

Among Us and Spatial Social Games

Among Us is not an open world, but its success revealed something important about multiplayer spaces: players want to be in a place together, not just in a game together. The spatial proximity chat mods that went viral showed that being in the same virtual room with directional audio transforms multiplayer from a game mechanic into a social experience.

Spatial social features that enhance a creator world:

• Proximity voice chat where volume fades with distance. Walk up to someone to talk. Walk away and they fade out. This creates natural social clusters without requiring voice channel management.
• Emote and gesture systems let players express themselves without voice. A wave, a dance, a pointing gesture. These are cheap to implement (animations on the player avatar) and disproportionately increase social engagement.
• Shared activities that happen in-world (not through menus) turn a space into a venue. If two creators can sit at a virtual table and look at 3D models together, the world has a reason to exist beyond displaying static content.

Browser Games That Scaled on Poki and CrazyGames

Poki and CrazyGames together serve over 150 million monthly players. The games that perform best on these platforms offer insights into what works in the browser specifically.

Top-performing patterns on browser game portals:

• Instant play (under 3 seconds to interactive). No login required. No tutorial required. The game should make sense within 5 seconds of landing.
• Session flexibility. Players drop in for 2 minutes or 2 hours. The game accommodates both. For a creator world, this means the world should be explorable without committing to a session. Walk around, see cool things, leave. Or stay and build for hours.
• Mobile compatibility. Over 60% of Poki traffic is mobile. A browser world that only works on desktop loses the majority of potential visitors.
• Social features that don't require friends. Leaderboards, reactions to other players' content, asynchronous features (see what other players built without being online at the same time).

The most successful 3D games on these portals (like Shell Shockers, 1v1.LOL, Smash Karts) keep the poly count low, the textures simple, and the framerate high. They prove that players accept simple graphics if the experience is smooth and responsive.

Browser-Based Creator Platforms

Hubs by Mozilla (now community-maintained). Multi-user 3D spaces in the browser built on Three.js and A-Frame. Supports voice chat, avatars, and shared objects. Not an open world (it's room-based), but the networking and rendering architecture are relevant. Mozilla open-sourced it before shutting down the hosted service, so the full codebase is available to study. GitHub.

Hyperfy. A web-based metaverse platform running entirely in the browser. Three.js rendering, multiplayer, avatar customization, and world building. Closer to our target than Hubs because it emphasizes creator tools. Worlds load in a browser tab with no download required. hyperfy.io.

Ethereal Engine (formerly XREngine). Open-source engine for multi-user worlds, built on Three.js and bitECS. Supports WebXR, spatial audio, and world editing. Has built-in ECS architecture, networking layer, and editor tools. The closest existing open-source project to what we're describing. Worth studying for how they integrate bitECS with Three.js for entity management and how their networking handles spatial state. GitHub.

Dusk (formerly Rune). Multiplayer game SDK for web games. Handles the networking layer so developers focus on gameplay. Their state sync approach uses predicted state with server reconciliation, which is the standard model for responsive multiplayer. The SDK abstracts away the complexity of rollback netcode. Worth studying for their developer experience.

Niantic Studio. A browser-based visual editor and web game engine (the successor to 8th Wall's tooling) for building 3D and XR experiences that others can visit in a browser. It demonstrates that non-technical creators can build 3D scenes in a browser if the tools are approachable. (Niantic spun its geospatial work into Niantic Spatial and sold its games business, including Pokemon GO, to Scopely in 2025.)

PlayCanvas Editor. Not a game itself, but PlayCanvas's cloud-based 3D editor shows that collaborative world-building tools can run in the browser. Multiple team members edit the same scene simultaneously. The editor communicates changes via a real-time sync layer. This is the collaborative creation model we'd need, except with our world as the canvas instead of a game editor.

Native Creator Platforms (Lessons for Browser)

These platforms run as native apps, but their design decisions about creator tools, world persistence, and social dynamics are directly applicable.

Roblox is the single most important reference for a creator world. 80+ million daily active users. Creators build full 3D experiences (games, social spaces, stores) that other players visit. The platform handles hosting, networking, discovery, and monetization.

What Roblox gets right:

• Creation is in-engine. Roblox Studio is the same environment players experience. Creators test their work instantly. There's no export/upload/wait cycle. For a browser world, the editor should be the world itself.
• Scripting is accessible. Lua (Roblox's scripting language) is simple enough that children learn it. Complex behavior is possible but optional. The floor is low and the ceiling is high.
• Discovery is social. You find experiences because friends are playing them. The homepage shows trending experiences. For a browser world, the world itself is the discovery surface. You explore and find things by walking around.
• Monetization works. Creators earn real money (Roblox paid out $740 million to creators in 2023). This attracts serious creative effort. Without economic incentive, creator platforms become hobby projects that fade.
• Roblox's rendering engine is custom and runs in a native app, not a browser. But the per-experience asset budgets are modest by modern standards (100 MB recommended max). Most successful Roblox experiences use low-poly stylized art, which aligns with browser rendering constraints.

Fortnite Creative / UEFN (Unreal Editor for Fortnite). Epic brought Unreal Engine's full editor to Fortnite creators. The result is a platform where people build islands (self-contained worlds) using professional-grade tools. Fortnite handles hosting, multiplayer, and distribution.

Relevant insights:

• Professional tools attract professional content. UEFN produces visually stunning experiences because creators have access to UE5's full capabilities. The trade-off is complexity. UEFN has a steep learning curve.
• Island-based instancing (each creation is a separate world) avoids the moderation and conflict problems of a single shared world. But it also means creators don't naturally discover each other's work by exploring. You visit islands through menus, not by walking.
• The business model works. Fortnite's creator program pays based on engagement. Top creators earn millions per year. Again, economic incentive drives quality.

Dreams (Media Molecule / PlayStation). Dreams gave console players a full 3D creation suite (modeling, animation, music, logic, level design) and a platform to share creations. It's the most ambitious creator-platform ever built in terms of tool depth.

Relevant insights:

• Sculpture-based modeling instead of polygon editing. Creators shape soft volumes with move/grab/smooth tools, similar to ZBrush but more intuitive. The learning curve is gentle. This approach maps well to in-browser creation because it doesn't require understanding vertices and UV maps.
• Everything is a shared asset. If someone creates a tree model, anyone can use it in their own creation (with attribution). This creates a compound creative ecosystem where each creation makes the platform more valuable.
• Dreams struggled commercially despite critical praise. The problem was distribution: it was locked to PlayStation, and the creation tools were so deep that most players never moved beyond consuming content. The lesson: the creation tools need to be simple enough that the majority of users try them, even if only a minority become serious builders.

Core (Manticore Games). A free platform for creating and playing multiplayer games, built on Unreal Engine with a simplified editor. Core's editor runs as a native app, but the philosophy is relevant. Templates and community-shared scripts let beginners assemble games from pre-made components. Advanced creators can write Lua scripts for custom behavior. Core struggled to find a large audience, partly because the games had to be played through the Core launcher. A browser-based version wouldn't have this friction.

VRChat and Rec Room are social platforms where creators build spaces others visit. VRChat uses Unity and runs on PC/VR. Rec Room runs on everything including mobile. Both prove that user-generated 3D worlds can sustain large communities. VRChat is more technically impressive (custom shaders, complex avatars). Rec Room is more accessible (in-app creation tools, simpler graphics, wider platform support). For a browser world, Rec Room's approach of simple in-app creation tools is more applicable than VRChat's external-tool workflow.

Second Life is the grandfather of all creator worlds. Launched in 2003, it's still running with 200,000+ daily active users. The entire world is user-created. Land is owned and traded. Creators sell objects, clothing, and buildings. The in-world scripting language (LSL) enables interactive content.

What Second Life teaches after 20+ years:

• Persistence matters more than graphics. Second Life's visuals are dated, but the world persists. Creations stay where you put them. Relationships and history accumulate. This persistence is what keeps people coming back.
• Economy drives creation. Second Life's GDP is estimated at $500 million per year. Creators build because they can sell. Without economic incentive, the volume and quality of creator content drops off.
• User-generated content requires moderation infrastructure. Second Life has had 20 years of moderation challenges. Any platform where users can place arbitrary content in a shared space needs automated scanning, reporting tools, and human review.
• Land-based spatial organization works. Second Life divides its world into parcels that users own. Each parcel has a prim (object) limit. This naturally prevents any one creator from consuming all the resources. For a browser world, chunk-based ownership with per-chunk object budgets is the equivalent pattern.

Comparative Analysis: What Each Game Teaches Us

The games that matter most for our specific case (browser-based, creator-focused, multiplayer) are Minecraft (editable world, compact data), Roblox (in-engine creation, economy), Krunker (browser UGC at scale), and Hordes.io (browser MMO architecture). The AAA titles (Skyrim, BotW, Witcher 3) teach rendering and streaming. The browser hits (Agar.io, Slither.io, Surviv.io) teach networking at scale. The creator platforms (Roblox, Dreams, Second Life) teach community dynamics. Vuntra City adds a current, implementation-level reference for speed-aware streaming, diegetic navigation, and million-agent simulation patterns in a modern procedural city.

What Skyrim and The Witcher Would Look Like in a Browser

Let's get concrete. If you took Skyrim's Whiterun and rebuilt it for browser delivery:

Terrain: The area around Whiterun is roughly 2km x 2km. At our chunk size (64m), that's about 32x32 = 1024 chunks. At 2-4 KB per chunk's heightmap, that's 2-4 MB of terrain data. The terrain textures (grass, dirt, rock, snow) as KTX2 atlas tiles might add another 5 MB. Total terrain: under 10 MB for the entire region.

Structures: Whiterun itself has maybe 40-50 buildings. Each building as an optimized GLB (3 LOD levels) might be 200-500 KB at highest detail. But you only need full detail for the 5-10 closest buildings. The rest are medium or low LOD (50-100 KB each). Total visible structures at any time: 2-5 MB.

Foliage: Skyrim's trees and grass around Whiterun are all instanced. You need maybe 10 unique tree models (200 KB each at full LOD, 20 KB as billboard impostors) and a grass system that generates blades from a density map on the GPU. Total foliage assets: 2-3 MB. The instancing data (positions, rotations, scales) for a 5x5 chunk area: under 500 KB.

NPCs: Whiterun has roughly 70 named NPCs plus guards. Each avatar at medium quality: 100-200 KB. But only 10-20 are visible at any time. Total NPC rendering data: 2-4 MB.

Grand total for a Whiterun-scale area visible at any time: 15-25 MB. That's absolutely viable in a browser. The initial load would show terrain and major structures within 3-5 seconds on broadband, with detail filling in over the next few seconds.

The Witcher 3's Novigrad is larger and denser but the same principles apply. You'd need more aggressive LOD and streaming, but the total visible data at any moment stays within browser memory limits.

What We'd Build First

A full open world is a multi-year project. Here's the path to get something real in creators' hands quickly:

Phase 1: Shared Island (3 months). A single terrain chunk (512x512m island) with heightmap terrain, water, basic foliage, day/night cycle. Multiplayer via Durable Objects (up to 50 concurrent users). Creators can place AI-generated 3D assets from their existing Cinevva library. Think of it as a shared diorama.

Phase 2: Expandable World (3 months). Chunk streaming for a 4x4 km world. Creator-owned plots where they have edit permissions. LOD system for terrain and objects. Persistent world state. Up to 200 concurrent users across the world.

Phase 3: Living World (6 months). AI-assisted terrain sculpting. Procedural foliage and atmosphere. Quest/event system so creators can build interactive experiences, not just static scenes. Voice chat. Avatar customization. The world becomes a destination, not a demo.

Open Questions

Art style. Stylized (low-poly, cel-shaded) is cheaper to render and more forgiving of AI-generated assets. Realistic requires higher-quality assets and more rendering budget. Skyrim worked despite dated graphics because the art direction was consistent. BotW is gorgeous on tablet-class hardware because the cel-shaded style hides low polygon counts. Minecraft uses 16x16 textures and is one of the most recognizable games ever. Krunker.io and Hordes.io both succeed with simple stylized graphics in the browser. The evidence overwhelmingly favors a stylized direction for a browser world. We need to pick a specific visual identity early and enforce consistency across AI-generated assets.

World persistence vs. instancing. Does everyone share one world (like an MMO or Second Life) or do creators each get their own instance that others can visit (like Minecraft servers or Fortnite Islands)? The tech supports both, but the social dynamics are completely different. Second Life's persistent shared world creates serendipitous discovery (you stumble onto other people's work by walking around). Fortnite's instanced islands require a menu or portal system for discovery. Roblox uses a hub-based approach: games are separate but you browse and discover through a shared interface. A hybrid could work: a persistent shared overworld where creators own plots (like Second Life parcels), with the option to enter standalone experiences through portals.

Creation tool depth. Dreams proved that deep creation tools impress critics but intimidate users. Townscaper proved that minimal tools can sell a million copies. Roblox Studio sits in the middle: simple enough for kids, deep enough for professionals. Krunker's voxel editor is simpler still. For a browser world, we should start with Townscaper-level simplicity (place objects, they snap and connect) and add depth over time. The initial experience of placing your first object in the world should take under 30 seconds.

Economy. Roblox, Second Life, and Fortnite Creative all prove that economic incentive is what turns a toy into a platform. Without a way for creators to earn from their work, the most talented creators will build elsewhere. This doesn't need to launch on day one, but the architecture should support it (object ownership, visit tracking, creator attribution).

Mobile. WebGPU on mobile is years away from reliable. A mobile experience would need to be a reduced version: simpler terrain, fewer objects, shorter view distance. Rec Room's approach of running on everything with adaptive quality is worth studying. Or we ship a native app for mobile and keep the full experience in the browser.

Moderation. An open world where anyone can place anything is a moderation nightmare. Second Life has been dealing with this for 20 years. Every placed asset needs automated content review before it becomes visible to others. This adds latency to the creative process but is non-negotiable. We should also consider parcel-based content ratings (like Second Life's General/Moderate/Adult system) so creators can choose their content boundaries.

Systemic interactions. BotW and Minecraft both show that material-based interaction systems create exponentially more interesting worlds than static object placement. If a creator places a wooden bridge and another creator starts a fire nearby, should the bridge burn? If someone places a dam, should water pool behind it? These interactions make the world feel alive but require consistent physics and material rules across all creator content. Deciding how far to go with systemic design is an early architectural decision.

Key Takeaways

Browser 3D open worlds are viable today. Hordes.io runs 200+ players in 3D in a browser. Krunker.io had 10 million monthly players with a full 3D map editor. The .io games proved browser multiplayer scales to millions. The tech isn't speculative.

The rendering is ready. Three.js and Babylon.js handle 3D scenes comparable to early-2010s AAA games. WebGPU unlocks compute shaders for terrain and foliage. Wasm physics engines run within 2-3x of native. A Whiterun-scale area fits in 15-25 MB of visible data.

The networking is ready. Cloudflare Durable Objects give you per-chunk authoritative servers at the edge. CRDTs handle collaborative editing without conflicts. Spatial partitioning (proven by every game from Agar.io to Skyrim) keeps network traffic manageable at hundreds of concurrent players.

The AAA open worlds (Skyrim, Witcher 3, BotW, Elden Ring, Minecraft, No Man's Sky) aren't just graphical benchmarks. They're textbooks on chunk streaming, LOD management, procedural generation, systemic design, and world composition. Every technique they use has a browser-compatible equivalent.

The creator platforms (Roblox, Second Life, Fortnite Creative, Dreams) teach the social and economic lessons. Creation must happen in-world, not in an external tool. Economic incentive drives quality. Persistence creates attachment. Simple tools reach more creators than powerful ones.

The creative pipeline is where Cinevva has an edge. We already generate 3D assets, textures, and audio. Connecting that pipeline to a world placement system is the missing piece. AI-generated content fills the world. Creator curation and arrangement gives it soul.

What we'd end up with isn't Skyrim in a browser. It's closer to the intersection of Minecraft (editable world), Roblox (creator economy), and BotW (systemic interactions), running in a browser tab with AI-powered creation tools. The technology to build it exists. The question is execution.

Research Papers and Academic References

The techniques in this guide aren't invented from scratch. They're grounded in decades of research. These are the papers that matter most for each subsystem, with notes on how they apply to a browser open world.

Terrain Generation and Rendering

"An Image Synthesizer" -- Ken Perlin (SIGGRAPH 1985). DOI. The paper that introduced Perlin noise. Every procedural terrain generator in every game since 1985 traces back to this. The noise function produces the smooth randomness that, when layered in octaves (fractional Brownian motion), generates natural-looking heightmaps. Simplex noise (Perlin, 2001) is the faster successor. For our terrain pipeline, this is the foundation: noise-based heightmap generation runs in a WebGPU compute shader at interactive speeds.

"Texturing and Modeling: A Procedural Approach" -- Ebert, Musgrave, Peachey, Perlin, Worley (1994, 3rd edition 2003). The textbook on procedural generation. Musgrave's chapters on terrain modeling, including multifractal terrain with erosion-like features, are the direct basis for modern game terrain generators. The fBm parameters (lacunarity, persistence, octave count) described here are the same ones we'd expose to creators for terrain customization.

"Geometry Clipmaps: Terrain Rendering Using Nested Regular Grids" -- Losasso and Hoppe (SIGGRAPH 2004). DOI. This paper solved how to render massive terrain at interactive rates using concentric LOD rings (clipmaps). The terrain mesh is a fixed set of nested grids centered on the camera. As the camera moves, the grids shift and update. This is the technique recommended in our terrain section for WebGL 2, and it works because the GPU workload is constant regardless of world size. The original implementation predates WebGL but maps directly to it.

"Fast Hydraulic Erosion Simulation and Visualization on GPU" -- Mei, Decaudin, Hu (2007). PDF. Moved hydraulic erosion from a CPU-bound offline process to a real-time GPU computation. The paper's shallow-water simulation model (treating water as a height field, computing flow between grid cells) runs in a compute shader. For our pipeline, server-side GPU erosion can transform noise-generated terrain into geologically plausible landscapes in under a second, making AI-generated terrain look hand-sculpted.

"Real-Time Rendering of Procedurally Generated Planets" -- Hybrid approaches from No Man's Sky's GDC talks and Sean Murray's descriptions. While not a single paper, the GDC 2017 talk "Building Worlds Using Maths" by Innes McKendrick (Hello Games) details how No Man's Sky generates planet-scale terrain using stacked noise functions, voxel representation with marching cubes, and GPU-side generation. Directly relevant to our procedural terrain approach, particularly for generating terrain features that heightmaps can't represent (caves, arches).

"C-DBLOD: Hybrid LOD for Terrain Rendering" -- Filip Strugar (2014). Paper. An improvement on geometry clipmaps that adds quadtree-based selection for better adaptivity around camera position. The key insight: instead of fixed concentric rings, use a quadtree to select terrain patches at varying resolutions. This handles irregular terrain (where some areas need more detail than others) better than pure clipmaps. Implementable in WebGL 2 with a small CPU-side quadtree traversal.

3D Gaussian Splatting and Neural Rendering

"3D Gaussian Splatting for Real-Time Radiance Field Rendering" -- Kerbl, Kopanas, Leimkühler, Drettakis (SIGGRAPH 2023). Project page. The paper that launched the Gaussian splatting revolution. A scene is represented as millions of 3D Gaussians, each with position, covariance (shape), opacity, and spherical harmonic color coefficients. Rendering sorts splats by depth and rasterizes them as 2D Gaussians. The approach is 100-1000x faster to train than NeRFs and renders at real-time rates. Multiple WebGL/WebGPU implementations exist. For our platform, this enables creators to capture real-world objects with phone photos and place them in the browser world.

"NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis" -- Mildenhall et al. (ECCV 2020). Project page. The foundational neural scene representation paper. A neural network maps 3D coordinates to color and density, enabling photorealistic novel view synthesis from a set of input photographs. While NeRFs are too expensive to render directly in a browser (they require per-pixel network evaluation), the NeRF-to-mesh extraction pipeline (training a NeRF, then running marching cubes on the density field) produces high-quality textured meshes from photos.

"Instant Neural Graphics Primitives with a Multiresolution Hash Encoding" -- Müller, Evans, Schied, Keller (SIGGRAPH 2022). Project page. Reduced NeRF training from hours to seconds using a multi-resolution hash table for spatial encoding. This made NeRF practical for production use. The hash encoding technique is also applicable to other spatial data in a browser world (fast lookups in large 3D datasets).

"Neuralangelo: High-Fidelity Neural Surface Reconstruction" -- Li et al. (CVPR 2023). Project page. Extracts high-quality triangle meshes from neural representations using multi-resolution hash encoding and numerical gradients for SDF (signed distance function) estimation. The output meshes are directly usable in browser 3D engines. For our asset pipeline, Neuralangelo (or similar tools like NeuS2) can convert NeRF captures into web-ready GLB files with clean geometry and baked textures.

Multiplayer Networking and State Synchronization

"Interest Management in Massively Multiplayer Online Games" -- Boulanger, Kienzle, Verbrugge (2006). DOI. A comprehensive survey of area-of-interest (AOI) management techniques for MMOs. Covers grid-based, aura-based, and hybrid approaches for filtering network updates based on spatial relevance. The grid-based approach (which maps to our chunk system) is the most efficient for uniform-density worlds. The aura-based approach (per-entity influence radius) works better for variable density. Our recommendation of chunk-based AOI with priority-based update rates within the AOI draws from this research.

"Dead Reckoning: Latency Hiding for Networked Games" -- Pantel and Wolf (2002). DOI. Formalizes dead reckoning (predicting entity positions based on last known velocity) for networked games. The paper quantifies the trade-off: higher prediction thresholds reduce bandwidth but increase position error visible during corrections. For browser games with 50-200ms latency, a prediction threshold of 0.5-1.0 meters keeps corrections invisible while cutting position update bandwidth by 60-80%.

"Conflict-free Replicated Data Types" -- Shapiro, Preguiça, Baquero, Zawirski (2011). DOI. The foundational CRDT paper. Defines state-based and operation-based CRDTs that converge without coordination. For our world editing system, the relevant CRDTs are: LWW-Register (Last-Writer-Wins Register) for object properties that have a single value (position, rotation, color), and OR-Set (Observed-Remove Set) for the collection of objects in a chunk (handles concurrent add/remove without conflicts). Yjs implements these efficiently in JavaScript.

"Time Warp: A Mechanism for Distributed Simulation" -- Jefferson (1985). DOI. The original optimistic distributed simulation paper. While Time Warp itself is too complex for a browser game, the core insight (process events optimistically and roll back if a conflict arrives from another node) is the basis for modern client-side prediction with server reconciliation. This is how every responsive multiplayer game works: the client predicts locally, sends actions to the server, and corrects if the server disagrees.

"The TRIBES Engine Networking Model" -- Frohnmayer and Gift (GDC 1999). One of the first practical descriptions of client-server game networking with interest management, prioritized state updates, and bandwidth budgeting. The "ghost manager" concept (the server maintains a per-client view of what the client knows, and only sends deltas from that view) is exactly what our chunk-based Durable Object architecture implements. This GDC talk is the intellectual ancestor of most modern game networking.

"Source Multiplayer Networking" -- Valve (2009). Developer documentation. Valve's documentation of the Source engine networking model (used in Half-Life 2, CS:GO, Team Fortress 2). Covers client-side prediction, entity interpolation, lag compensation, and the "snapshot" system where the server sends full world state at regular intervals while the client interpolates between snapshots. This is the gold standard for authoritative server networking and directly applicable to our architecture.

Procedural Generation

"Model Synthesis: A General Procedural Modeling Algorithm" -- Merrell (2007). DOI. One of the precursors to Wave Function Collapse. Generates 3D structures from example models by propagating local constraints. The algorithm ensures global consistency by iteratively collapsing cells with the fewest possibilities (minimum entropy heuristic). This is how Townscaper and similar generators produce coherent structures from simple user input.

"WaveFunctionCollapse" -- Maxim Gumin (2016). GitHub. Not a traditional paper but a seminal open-source project with extensive documentation. The algorithm takes a small example image or tileset and generates larger outputs that are locally similar to the input. For a creator world, WFC can generate building layouts, road networks, dungeon maps, and terrain details from a small set of creator-defined rules. Multiple JavaScript implementations exist.

"Wave Function Collapse is Constraint Solving in the Wild" -- Karth and Smith (FDG 2017). DOI. An academic analysis of WFC that clarifies its relationship to constraint satisfaction and shows how to analyze and extend it. Relevant for understanding WFC's limitations (it can get stuck and need backtracking) and how to design tilesets that avoid these issues.

"Superposition Theorem and Its Implications for the Procedural Generation of Game Content" -- Sandhu et al. (2022). Explores using quantum-inspired superposition concepts for procedural content generation. While speculative, the mathematical framework for maintaining multiple possible states before "collapsing" to a final configuration is exactly how WFC works and could inform more sophisticated generation systems.

Real-Time Rendering Techniques

"Real-Time Rendering" -- Akenine-Möller, Haines, Hoffman (4th edition, 2018). The standard textbook. Chapter 19 (Acceleration Structures), Chapter 20 (Efficient Shading), and Chapter 21 (Virtual and Augmented Reality) are particularly relevant. The frustum culling, occlusion culling, and LOD algorithms described here are what Three.js, Babylon.js, and every game engine implement. Not a paper, but the definitive reference.

"A Survey on Baking Neural Radiance Fields for Real-Time View Synthesis" -- Reiser et al. (2023). DOI. Surveys methods to convert NeRFs into formats that render in real time (meshes, textures, sparse voxel grids). Directly relevant to our asset pipeline where server-side neural capture needs to produce browser-renderable output.

"Scalable and Accurate Online Feature Matching Using 3D Gaussian Splatting" -- Various groups (2024-2025). Multiple recent papers explore editing, compositing, and dynamic scenes with Gaussian splats. Relevant because creator worlds need to composite multiple splat scenes (each creator's captured objects) into a single coherent scene. Methods for splat editing (recoloring, deformation, compositing) are active research areas.

"Efficient GPU Screen-Space Ray Tracing" -- McGuire and Mara (JCGT 2014). DOI. The paper behind screen-space reflections used in our water rendering section. Traces rays through the depth buffer for approximate reflections without the cost of full ray tracing. The "hierarchical tracing" variant (using a min-max depth mipmap) runs efficiently on WebGL 2.

"Simulating Ocean Water" -- Jerry Tessendorf (2001). PDF. The foundational paper on FFT-based ocean simulation. Describes the Phillips spectrum (statistical model of ocean waves) and how to transform it into a spatial displacement map via inverse FFT. Used by every major game with realistic ocean (Sea of Thieves, Assassin's Creed, Uncharted). The FFT computation maps cleanly to WebGPU compute shaders.

"Precomputed Atmospheric Scattering" -- Bruneton and Neyret (EGSR 2008). DOI. The paper behind physically accurate sky rendering. Precomputes atmospheric scattering into lookup tables that a fragment shader samples in real time. Produces correct sky colors, aerial perspective (distant objects look bluish/hazy), and sunset/sunrise colors from first principles. The precomputed tables are small (a few hundred KB) and the runtime shader is cheap. Both Three.js's Sky shader and Babylon.js's procedural sky are simplified versions of this approach.

"Ambient Occlusion Volumes" -- McGuire (HPG 2010) and "Scalable Ambient Obscurance" -- McGuire, Mara, Luebke (HPG 2012). PDF. The papers behind modern SSAO implementations. SAO is the most commonly implemented variant in browser 3D engines because it's efficient (one depth buffer sample per pixel) and produces plausible contact shadows. The algorithm samples the depth buffer around each pixel to estimate how "occluded" it is by nearby geometry. Both Three.js and Babylon.js implement SAO-derived SSAO.

Crowd Rendering and Animation

"GPU Crowd Rendering" -- Dudash (2007) and subsequent GDC/SIGGRAPH presentations on instanced crowd rendering. The core technique: bake skeletal animation frames into textures (Vertex Animation Textures), then render crowds as instanced meshes where each instance reads its bone transforms from the animation texture based on its current frame. This decouples animation evaluation from draw calls, allowing a single instanced draw call to render hundreds of uniquely animated characters.

"Position Based Dynamics" -- Müller et al. (2007). DOI. The foundational paper on PBD, which is how modern game engines simulate cloth, hair, and soft bodies. Rapier (our recommended Wasm physics engine) uses PBD-derived solvers. For avatar customization (capes, flowing hair, loose clothing), PBD provides responsive simulation at game frame rates. The Wasm implementation keeps it off the main JavaScript thread.

"FABRIK: A Fast, Iterative Solver for the Inverse Kinematics Problem" -- Aristidou and Lasenby (2011). DOI. The paper behind the IK solver recommended in our avatar section. FABRIK works by alternately reaching from end effector to root and root to end effector, converging in 3-5 iterations. It's faster than Jacobian-based IK, handles joint constraints naturally, and is simple to implement (about 50 lines of code for the basic solver). Both Three.js and Babylon.js IK implementations are derived from FABRIK.

Virtual Worlds and Collaborative Environments

"Massive Multiplayer Online Games: A Survey of the State-of-the-Art" -- Yahyavi and Kemme (2013). DOI. Comprehensive survey of MMO architecture covering client-server models, peer-to-peer approaches, interest management, consistency models, scalability techniques, and cheating prevention. The taxonomy of consistency models (strict, eventual, causal) maps to our CRDT-based approach (eventual consistency with causal ordering via vector clocks).

"A Distributed Architecture for Multiplayer Interactive Applications on the Internet" -- Diot and Gautier (1999). DOI. Early research on distributed virtual environments that identified the core tension: strict consistency requires coordination (adding latency), while weak consistency allows responsiveness but risks visible inconsistency. The paper argues for "local lag" (delaying local display by a small amount to give time for remote updates to arrive) as a middle ground. For world edits (placing objects), a 100-200ms local lag is imperceptible and gives the server time to validate.

"The Second Life Grid: The Architecture of a Near-Contemporary Open Source Virtual World" -- Linden Lab technical documentation and community reverse-engineering. While not a single paper, the technical analysis of Second Life's architecture is extensively documented. The key insights: each 256x256m region runs on a dedicated server instance. Objects are stored as a tree of "primitives" (basic shapes with transforms, textures, and scripts). The viewer streams object descriptions and textures on demand. This parcel-based model with per-object persistence is the closest existing architecture to what we're building, and Second Life's 20+ years of operation prove it scales.

Web Graphics and Browser Performance

"WebGPU: A High-Performance Graphics API for the Web" -- W3C GPU for the Web Working Group (2023-ongoing). Specification. The formal specification for WebGPU. Not a research paper, but the definitive technical document for browser GPU programming. The compute shader specification (Section 23) is particularly relevant for terrain generation, foliage scattering, and particle systems described throughout this guide.

"WebAssembly: A Framework for Running Compiled Code in the Browser" -- Haas et al. (PLDI 2017). DOI. The original WebAssembly paper from the browser vendors. Demonstrates that Wasm achieves within 2x of native performance for compute-intensive workloads. This validates our recommendation of Wasm-compiled physics engines (Rapier, Havok) for browser open worlds. The paper's performance analysis shows that the overhead comes primarily from bounds checking and indirect function calls, not from the compilation model itself.

"Not So Fast: Analyzing the Performance of WebAssembly vs. Native Code" -- Jangda et al. (USENIX ATC 2019). PDF. A rigorous benchmark of Wasm vs native performance. Finds that Wasm runs 1.45x-1.55x slower than native C on average across the SPEC CPU benchmark suite. For game physics specifically (floating-point heavy, few system calls), the overhead is at the lower end (~1.3x). This confirms that Wasm physics in a browser is viable for real-time game workloads.

"Bringing the Web up to Speed with WebAssembly" -- Rossberg et al. (2018). DOI. Describes the design rationale and formal semantics of WebAssembly. Particularly relevant: the discussion of memory safety guarantees (Section 3) explains why Wasm modules can safely share a browser tab with JavaScript without the security risks of native plugins. This is what makes running physics engines (which were traditionally C++ libraries) safe in a browser.

AI-Powered Content Generation

"Text-to-3D Generation with Bidirectional Diffusion Using Both 2D and 3D Priors" -- Various groups (2023-2025). Multiple recent papers (DreamFusion, Magic3D, ProlificDreamer, MVDream, Zero-1-to-3++) explore generating 3D assets from text prompts by using 2D diffusion models as priors for 3D optimization. The quality has improved dramatically from early 2023 to 2025, going from blobby shapes to detailed, textured meshes. For our platform, these models (running server-side on GPUs) are the "AI generation" step in the creator asset pipeline.

"DreamFusion: Text-to-3D using 2D Diffusion" -- Poole et al. (ICLR 2023). Project page. The foundational paper on Score Distillation Sampling (SDS), which uses a pre-trained 2D diffusion model to guide 3D optimization. The key insight: you don't need 3D training data if you can evaluate whether rendered views of a 3D object match a text prompt using an existing 2D model. This opened the door to text-to-3D generation and is the basis for subsequent work (Magic3D, ProlificDreamer) that improved quality and speed.

"LRM: Large Reconstruction Model for Single Image to 3D" -- Hong et al. (ICLR 2024). Project page. Reconstructs a 3D model from a single image in 5 seconds on a single GPU. The model outputs a NeRF-like representation that can be converted to a mesh. For a creator world, this means a creator could take a photo of any real-world object and get a 3D model within seconds. The speed makes it viable as an interactive tool rather than a batch process.

"Procedural Content Generation via Machine Learning (PCGML)" -- Summerville et al. (2018). DOI. A survey of using machine learning for procedural content generation in games. Covers level generation, item generation, narrative generation, and world generation. Particularly relevant: the discussion of "controllable generation" where designers set high-level parameters and the ML model fills in details. This is the paradigm for AI-assisted world building: creators set intent ("make this area a spooky forest"), AI fills in the geometry, textures, and population.

How These Papers Connect to Our Architecture

The research maps to our architecture in layers:

Terrain pipeline: Perlin/simplex noise (Perlin 1985, 2001) generates the base heightmap. Hydraulic erosion (Mei et al. 2007) adds geological realism. Geometry clipmaps (Losasso and Hoppe 2004) or CDLOD (Strugar 2014) render the terrain efficiently in the browser. Atmospheric scattering (Bruneton and Neyret 2008) makes distant terrain look correct.

Asset pipeline: Text-to-3D (DreamFusion et al.) and image-to-3D (LRM) generate assets server-side. Gaussian splatting (Kerbl et al. 2023) enables photogrammetry capture. Neuralangelo (Li et al. 2023) extracts clean meshes from neural captures. All outputs are processed into browser-ready GLB/KTX2.

Rendering: SSAO (McGuire 2012) adds depth. Screen-space reflections (McGuire and Mara 2014) power water. FFT ocean (Tessendorf 2001) simulates water. Vertex animation textures (Dudash 2007) render crowds. FABRIK (Aristidou and Lasenby 2011) drives character IK.

Networking: Interest management (Boulanger et al. 2006) filters updates by spatial relevance. Dead reckoning (Pantel and Wolf 2002) reduces bandwidth. CRDTs (Shapiro et al. 2011) handle collaborative editing. Client prediction with server reconciliation (Jefferson 1985, Valve Source Networking) provides responsiveness.

World generation: WFC (Gumin 2016, Karth and Smith 2017) generates structures and layouts. PCGML (Summerville et al. 2018) provides the framework for AI-assisted generation where creators set intent and models fill details.

The research is mature. Most of these techniques have been used in shipped games for years. The innovation in bringing them to a browser isn't the algorithms. It's the engineering of making them work within browser memory, GPU, and network constraints, which is what the rest of this guide addresses.

Further Reading

• Web Games Tech Stack in 2026 covers WebGL, WebGPU, and WebAssembly fundamentals
• Web Game Engines Comparison compares engines for browser delivery
• Three.js + USDC Tech Report on loading 3D assets in the browser
• Frontier Open-Source Gen AI Models for the AI generation pipeline
• Co-op Game Design on multiplayer design patterns that keep players engaged
• Landscape Generation for Browser Open Worlds — terrain, erosion, and vegetation rendering
• WebSocket multiplayer basics — getting started with browser multiplayer networking
• WebGPU getting started — the rendering API for next-gen browser 3D
• Streaming asset loading — progressive loading patterns for large 3D worlds

There's a term floating around that developers either love or hate: "friend slop."

It started as an insult. A way to dismiss low-budget co-op games as shallow content farms designed for viral clips. But then Peak sold 11 million copies in two months. R.E.P.O. dominated Steam. And suddenly the developers making these games started embracing the label.

"We actually think friend slop is a fire term," said Paige Wilson from Aggro Crab, Peak's publisher. "It's used to describe a low-cost game that you and your friends can pick up whenever, have some fun, hang out, and expect a bit of jank. I think some people might not view it as this, but we do!"

Here's what I've learned about why these games work, and how you can design one.

The Games That Defined 2025

Lethal Company (2023) kicked off the wave. Solo developer Zeekerss created a co-op horror game where you collect scrap from abandoned facilities while avoiding monsters. It sold millions and spawned an entire genre.

View Lethal Company on Steam →

R.E.P.O. (2025) took the formula and made it goofier. Robot avatars. Exaggerated physics. Players swore it was "so funny it was hard to be scared." It became one of the biggest co-op games of 2025. When Steam published its year-end best-sellers in December 2025, R.E.P.O. came out as the top-selling new game of the year by units, with Peak right behind it. Co-op friend slop games took the top slots outright, beating out big-budget releases that cost five times as much.

View R.E.P.O. on Steam →

Peak (2025) proved the formula works outside horror. Climb a mountain with friends. Fall off. Laugh. Try again. 11 million copies sold. The game was made in 4 weeks during a game jam in South Korea.

Peak kept adding free content after launch, rotating in a new Mesa biome in August 2025 and running a limited-time in-game concert with rapper bbno$ in December 2025. The low-budget jam game became a live service in everything but name.

View Peak on Steam →

The Core Design Principles

After studying these games and reading developer interviews, here's what they all share:

1. Low Barrier to Entry

These games cost under $20. Most are under $10. Peak launched at $5, then $8. The developers had a whole theory about price psychology: "Six bucks is still five bucks. Three bucks is two bucks. Two bucks is basically free."

But it's not just price. The games are easy to learn. You don't need prior gaming experience. You don't need to understand complex systems. Your non-gamer friends can join.

2. Failure Is Funny

In Peak, I've never survived to the end of a run. I keep playing anyway.

The most entertaining moments aren't victories. They're watching your friend miss a jump and fall off a mountain. Eat poisonous food. Get chased by the scoutmaster.

Design for comedic failure states. Make death spectatable. Let players laugh at each other.

3. Proximity Chat

Almost every successful friend slop game has proximity voice chat. Your voice gets quieter as you move away from teammates. Character mouths lip-sync to what you're saying.

This creates natural comedy. Someone screaming for help in the distance. Whispered conversations. The panic when you realize you've wandered too far alone.

4. Physics Jank (Intentional)

The wobbly, unpredictable physics aren't bugs. They're features.

When your character ragdolls unexpectedly, it's funny. When objects bounce in weird ways, it creates stories. The jank is the content.

5. Shareable Moments

These games are designed for clips. Twitch. TikTok. YouTube Shorts.

Every session produces moments worth sharing. The algorithm loves them. Players become marketers without realizing it.

What Makes Them Different From Traditional Co-op

The Verge's analysis nailed it: "In friend slop, the main objective is humor. Winning is secondary."

The Loneliness Angle

Here's something the critics miss when they dismiss these games as "slop."

We're in a loneliness epidemic. People are isolated. Remote work scattered friend groups. These games give people a reason to hang out virtually. They're low-commitment social spaces.

As one writer put it: "a real path to salvation from divisive and isolating times."

That's not nothing.

The Economics

These games are cheap to make. Small teams. Short development cycles. Low-fidelity graphics that players actually prefer because they're charming.

Content Warning took a small team and minimal time. Peak was a game jam project. R.E.P.O. came from a studio that makes experimental games.

The studios behind these hits aren't slowing down. Aggro Crab, Peak's co-developer, shipped its next co-op game Crashout Crew on May 28, 2026, a chaotic forklift warehouse simulator for one to four players that launched on Steam and Xbox Game Pass. Same recipe: small team, physics-driven mayhem, designed to be played with friends.

The math works: low investment, high potential upside, and a genre that rewards authenticity over polish.

Notable Examples

The Essentials:

• Among Us (2018) - the prototype
• Phasmophobia (2020) - ghost hunting with proximity chat
• Lethal Company (2023) - the genre-definer
• Content Warning (2024) - filming monsters for views
• Peak (2025) - climbing chaos
• R.E.P.O. (2025) - goofy horror extraction
• Crashout Crew (2026) - forklift co-op mayhem from Peak's co-developer Aggro Crab
• Gamble With Your Friends (2026) - party game that sold a million copies in its first week, proof the genre kept rolling into 2026

The Experiments:

• Guilty as Sock! - courtroom drama with sock puppets
• Mage Arena - $2.99, cast spells by shouting
• Lockdown Protocol - Among Us meets chore simulator
• YAPYAP - wizards vandalizing a tower

Why Web Games Fit Perfectly

Here's where it gets interesting for browser-based developers.

Friend slop games have two requirements: low barrier to entry and instant multiplayer. Web games nail both.

No downloads. No installs. Share a link, everyone's playing in 30 seconds. That's the dream for co-op games.

The technical challenges (WebSocket multiplayer, physics sync, voice chat) are solvable. We have WebSocket tutorials for a reason.

The bigger question is design. Can you create moments worth sharing? Can you make failure funny? Can you give friends a reason to hang out in your game?

If You're Building One

Start with the social loop. What will players talk about after? What stories will they tell? Design for those moments.

Embrace the jank. You don't need AAA polish. Players prefer charm. Ragdolls are your friend.

Price it low. $5-10 is the sweet spot. You want entire friend groups buying in. Four copies at $8 beats one copy at $30.

Add proximity chat. It's practically required. Players expect it now.

Make it clippable. If nobody's sharing clips of your game, you're missing free marketing.

Test with actual friend groups. Solo playtesting won't tell you if the social dynamics work.

The games that win aren't the most polished. They're the ones that give friends a reason to laugh together.

Common Questions

What makes a good co-op game?

The best co-op games create situations where players need each other but also get in each other's way. You want moments of chaos that turn into stories people retell later. Low skill floors help because nobody wants to be the friend who can't figure out the controls. Funny failure states matter more than you'd think. When losing is entertaining, people keep playing. Proximity voice chat adds a layer that text chat can't replicate. And keep sessions short, 15 to 30 minutes, so people can fit a few rounds into an evening without it feeling like a commitment.

Why are co-op games so popular right now?

A few things came together. Lethal Company proved in 2023 that a solo dev could build a co-op game and sell millions of copies. Then Peak hit 11 million copies in 2025 as a game that started as a jam project. The economics got hard to ignore. These games are cheap to make, they spread through friend groups like wildfire, and they generate clips that go viral on social media. The "friend slop" genre, as people call it, rewards authenticity over polish. You don't need a big studio or a big budget. You need funny moments and friends who want to share them.

Can you make a co-op game for the browser?

Absolutely. Browser games are actually a great fit for co-op because the biggest barrier to playing with friends is getting everyone set up. With a web game, you share a link and everyone's in. No downloads, no version mismatches, no "hold on I need to update." The technical pieces, WebSocket multiplayer, physics sync, even basic voice chat through WebRTC, are all doable in the browser. We've got a WebSocket multiplayer tutorial that covers the networking side.

Related:

• WebSocket Multiplayer Basics
• Steam Next Fest Strategy
• Kickstarter for Indie Games
• Gamepad API — local co-op with multiple controllers
• Game Jams & Hackathons — co-op jam games get more ratings and shares
• How to Launch Your Game on itch.io — browser co-op games thrive on itch.io

Let's talk about what's actually happening with game crowdfunding right now, because the vibes don't match the data.

The golden age is over. In 2013, video games raised $51 million on Kickstarter. By 2024, that dropped to $25.2 million across roughly the same number of campaigns. The pie shrunk by half, but the number of people fighting for slices barely changed. That's the math you're walking into.

And yet, developers keep turning to Kickstarter. Publishers are saying no. Investors got spooked by the layoff wave. Around 15,000 game industry jobs vanished in 2024 alone, and trackers note the real number is likely higher since not every cut gets reported. For a lot of teams, crowdfunding isn't Plan A anymore. It's the backup when everything else falls through.

Two Stories From 2025

The One That Worked

Elestrals Awakened raised over $1.3 million from more than 9,000 backers, making it the most funded video game on Kickstarter in 2025. The team behind it? A former Pokemon Championship caster with nearly a million YouTube subscribers, and two successful Kickstarters already under their belt (a trading card game and its digital version). They had an audience. They had a track record. They had years of community-building before they ever asked for money.

View the Elestrals Awakened Kickstarter →

The One That Didn't

Alzara: Radiant Echoes raised €300,000 from over 5,000 backers in 2024. One of the most-backed video game campaigns that year. The studio, Camelia, shut down in April 2025. No game. No refunds. The team had a playable demo, a senior team, half their funding secured, and the Kickstarter success as proof of market interest. They pitched at Tokyo Game Show, DICE, and every industry event they could find. Nobody bit.

"This is part of the harsh reality of the industry. Sometimes, even having a promising game concept with proven market interest and a strong team is not enough."

— Studio Camelia, final update to backers

View the Alzara Kickstarter (archived) →

Two campaigns. Both "successful" by Kickstarter's definition. One delivered nothing. The difference wasn't the money raised. It was everything that came before.

The Numbers That Matter

Data from 17,000 Kickstarter campaigns since 2009 tells a clear story:

Goal size predicts success. None of the 15 campaigns aiming for $2.5 million or more succeeded. But games with goals under $500K often overperformed. Shenmue 3 asked for $2M and got $6.3M. Bloodstained asked for $500K and got $5.5M. Set a goal people can rally behind, then blow past it.

Timing matters. December is the worst month to launch (28% success rate). February is the best (35%). The overall average is 32%. Yes, two-thirds of campaigns fail.

Most money comes from modest campaigns. 81.8% of all funds raised came from campaigns with goals of $500K or less. 68.1% from campaigns under $250K. The mega-campaigns get the headlines, but the mid-size ones actually fund.

The 2025 numbers are now in, and they tell a more nuanced story than the long decline. Video games drew 443 successful campaigns in 2025, raising just shy of 2024's roughly $26 million. The interesting shift was at the top: 11 campaigns cleared $500,000, the most in that band since 2015 and more than double the count in 2024. The pool isn't growing, but the big, well-prepared campaigns are coming back.

What $50,000 Actually Buys You

Here's a postmortem from the Space Chef team, who raised $50,000 from 1,119 backers:

"To get $50,000, we had to spend $20,000+ on marketing, ads, and creators. The trailer took 3 months to make. Kickstarter isn't free money. It's a multi-year commitment to hundreds of people."

Their math:

At $20/hour, that's 1,300 hours. About 32 weeks. Nowhere near enough to finish their game.

They eventually got publisher investment to complete it. The Kickstarter didn't fund the game. It proved the game was worth funding.

View the Space Chef Kickstarter →

When It Makes Sense

Kickstarter works when you're using it for validation, not salvation.

Good reasons to run a campaign:

• You have an audience that's already asking for your game
• You need proof of market demand to attract a publisher
• You want to build a community that's literally invested in your success
• You have a track record (previous games, successful projects, visible expertise)

Bad reasons:

• You need the money to start development
• You have no existing audience
• You're hoping the campaign itself will build awareness
• You can't afford to fail

The Brutal Truths

You probably can't finish your game on Kickstarter money alone. Most successful campaigns cover 25-50% of actual development costs. The rest comes from publishers, grants, savings, or day jobs.

Your backers can't get refunds if you fail. They know this. They're increasingly skeptical of first-time creators. Studios with shipped games or proven track records get more trust.

Running a campaign is a full-time job. Updates every few days. Backer messages. Press outreach. Stretch goal planning. Social media. For 30+ days, you're not making your game. You're marketing it.

Success rate is 32%. That means 68% of campaigns fail. If you launch without preparation, you're almost certainly in that 68%.

If You're Still In

Do this before you launch:

1. Build your audience first. 1,000-2,000 people who actually want your game. Discord, email list, social following. Whatever works.

2. Have something playable. A demo, a vertical slice, something. Players have been burned too many times. Showing beats promising.

3. Set a goal you can actually hit. Ask for what you need to finish, not what you'd like to have. Fund fast, then stretch.

4. Plan for the work after. Updates. Fulfillment. Years of development with people watching. This isn't a transaction. It's a relationship.

The teams that succeed treat Kickstarter as the start of a conversation, not the end of a fundraising problem. Everyone else treats it like a lottery ticket.

Don't be a lottery ticket.

Common Questions

Is Kickstarter still worth it for indie games?

It depends on what you're bringing to the table. If you've got an audience, a playable demo, and realistic expectations, it can work. The success rate for video game campaigns sits at about 32%, which isn't great odds. But teams with existing communities and prior track records do much better than that average. The real question is whether you've done the groundwork. If you're launching cold with no following, your chances are rough.

How much money do indie games raise on Kickstarter?

The total pool for video games on Kickstarter has been shrinking. It peaked around $51 million back in 2013 and dropped to about $25.2 million in 2024. Most successful campaigns raise between $10,000 and $50,000. The thing people forget is how much of that disappears. Kickstarter takes about 5%, payment processing takes another 3-5%, then you've got reward fulfillment, shipping, and taxes. A campaign that raises $50,000 might leave you with $26,000 in actual working money.

What percentage of game Kickstarters fail?

About 68% of video game campaigns on Kickstarter don't reach their funding goal. That number has stayed pretty consistent over the years. The campaigns that fail usually share a few traits: no existing audience, no playable demo, unrealistic funding goals, or launching during a crowded period without any way to stand out.

Related:

• How to Launch Your Game on itch.io
• Game Jams & Hackathons Guide
• Steam Next Fest Strategy — building wishlists before and alongside crowdfunding
• Co-op Game Design — co-op games generate organic social sharing that helps campaigns

Last updated: June 2026.

Here's the thing about Next Fest: everyone treats it like a lottery. Throw your demo in, cross your fingers, hope the algorithm smiles on you. That's not how it works.

I've been reading postmortems obsessively. Hundreds of them from 2025 alone. The pattern is so consistent it's almost boring: the games that "win" Next Fest aren't discovering success during the festival. They're amplifying momentum they already built.

Let me show you what I mean.

The Formula Nobody Wants to Hear

Eilean Mor went into October 2025 with 2,117 wishlists. Gained 2,082 during the festival. Almost exactly 2×.

Dice of Kalma went in with 335 wishlists. Gained 659. Just under 2×. Their developer wrote: "Our wishlists nearly tripled, which is awesome, but we have still lot of work to do."

That's the uncomfortable truth. Next Fest doubles what you already have. If you have nothing, you get nothing doubled.

View Eilean Mor on Steam →

The 2,000 Wishlist Cliff

Data from 208 games in February 2025 shows a sharp inflection point at 2,000 wishlists.

Here's what a developer from that survey said: "You really start to see how the wishlists a game earns takes off for games that have 2000+ wishlists before Steam Next Fest."

It's not an algorithm thing exactly. It's a validation thing. 2,000 wishlists means you've figured out something about marketing. You know how to reach people. That skill compounds during the festival.

What February 2026 Confirmed

The February 2026 Next Fest (February 23 to March 2) was the most crowded yet, with over 3,500 demos. That's a 51% jump over February 2025. And the bigger the field gets, the louder the data screams the same thing.

How To Market A Game surveyed 182 developers afterward. The single strongest correlation in the whole dataset was between a game's pre-fest wishlist count and the wishlists it earned during the event: a Spearman r of 0.825. For comparison, most marketing correlations in indie data sit between 0.2 and 0.4. Nothing else came close.

The spread was brutal. The median demo gained 806 wishlists. The 70th percentile hit 1,839. You needed 13,461 to crack the 95th percentile, and the top earner pulled 57,074. A cold-launched demo with no follower base had roughly a 1 in 20 shot at reaching that top 5%. The festival rewards momentum you bring with you. It does not manufacture it.

The timing data backed up the early-demo rule too. Of the games that gained 15,000+ wishlists, five had demos out months ahead, four launched about a month before, and only one shadow-dropped days before the fest.

The First 48 Hours Are Everything

Valve gives everyone a shot on days 1 and 2. After that, the algorithm decides who deserves more visibility based on performance.

Bronze tier games? Their impressions fall off a cliff on day 3. Diamond tier games? Their impressions triple.

What Actually Won in 2025

Do No Harm (February 2025)

The numbers: 45,655 wishlists gained during Next Fest. Started with 52,102.

What they did:

• Trailer dropped on GameTrailers January 29 (weeks before Next Fest)
• PR outreach timed with trailer
• Demo went live February 4
• Paid ads to amplify organic momentum

This is the playbook. They didn't rely on Next Fest to discover them. They used Next Fest to multiply attention they'd already earned.

View Do No Harm on Steam →

Cairn (October 2025)

Hit #5 demo on day 1. Trending Upcoming and Top Demos by day 2.

What they did:

• Demo launched December 2024 (10 months before their Next Fest)
• 32,000 followers before the festival
• Accepted into 18 festivals before Next Fest
• The Game Bakers (Furi, Haven) - established studio with track record

They didn't need Next Fest. But they used it anyway because it's free amplification when you've already done the work.

View Cairn on Steam →

YAPYAP (October 2025)

The "friend slop" phenomenon. Co-op horror where you play as wizards vandalizing a tower. Went viral on social media, then dominated Next Fest.

Their trailer dropped August 25, 2025 - one month before Next Fest. It earned 434K views. That was the inflection point. Everything after was just riding the wave.

View YAPYAP on Steam →

The Genres That Keep Winning

From the February 2025 diamond tier:

• Horror (×2) - Steam loves horror. Always has.
• Strategy/Management (×4) - "Crafty-buildy-simulationy" is the vibe
• Co-op "friend slop" - Lethal Company opened the floodgates
• Point-and-click (×2) - but both were sequels with existing audiences

The point-and-click thing is interesting. Beholder: Conductor was the 4th game in its series. Duck Detective was a sequel. They had built-in audiences. First-time point-and-click devs don't see those results.

What The Small Games Say

Not everyone is chasing diamond tier. Here's what developers at the smaller end reported from October 2025:

"62% conversion rate from visits to wishlists. Store page itself seems to work. But impressions were high and people didn't click. Capsule art needs an update."

"57% of store visitors played our demo. That sounds high? But only 15% who played also wishlisted. Something didn't meet expectations."

"Got permanently banned from r/cozygames for asking if they found our game cozy. Always try to follow the rules but this was pretty surprising."

The grind is real. These developers are testing capsule art, tweaking tags, getting banned from subreddits. It's not glamorous.

The Honest Playbook

Months before Next Fest:

1. Launch your demo early. Like, way early. The winners had demos out for 6-12 months.
2. Enter smaller genre festivals first. Use them as polish runs.
3. Build to 2,000+ wishlists. That's your real threshold.

During Next Fest:

1. Days 1-2 are make or break. Push everything.
2. Stream if you can. It shows as "live" on your page.
3. Coordinate with streamers. Pay them if you have to.
4. Put your demo on a separate Steam page so it can collect reviews.

After Next Fest:

1. Don't launch immediately. A few weeks later actually performs better.
2. Use the momentum for your actual launch marketing.

When To Skip It

Real talk: Next Fest isn't for everyone.

Skip it if:

• You have under 1,000 wishlists (you'll be invisible by day 3)
• Your demo isn't polished (you get one shot at first impressions)
• It's your first festival ever (do smaller ones first)
• You're launching immediately after (wait a few weeks)

There are dozens of smaller festivals every month. More focused audiences. Less competition. Check howtomarketagame.com/festivals for the list.

Next Fest is the default because Valve prompts everyone to join. But the best marketing isn't the stuff everyone gets prompted to do.

Common Questions

How many wishlists do you need for Steam Next Fest?

The data points to about 2,000 wishlists as the threshold where Next Fest starts working for you. Below that, you're basically invisible by day three. The algorithm favors games that are already getting traction, so you need momentum going in. The rough pattern from the February 2025 festival: whatever your wishlist count was before Next Fest, you can roughly double it during the event. So if you go in with 500, you might come out with 1,000. Go in with 5,000, you could hit 10,000. The starting number matters way more than most people realize.

When should you release a demo for Steam Next Fest?

Earlier than you'd think. The games that performed best in recent festivals had demos out for 6 to 12 months before Next Fest even started. That gives you time to collect feedback, fix the obvious stuff, and build wishlists organically before the festival multiplies whatever you've already got. Dropping your demo the week of Next Fest is a common mistake. You're competing with hundreds of other new demos and you've got no reviews, no feedback loop, nothing to work with.

Is Steam Next Fest worth it for small indie games?

It can be, but only if you're prepared. Small games that go in with under 1,000 wishlists and an unpolished demo usually get buried fast. The festival runs for a week and there are hundreds of games competing for attention. If you're early in development, smaller genre-specific festivals are a better first step. They have more focused audiences and less competition. Save Next Fest for when your demo is tight and you've got some momentum behind you.

Related:

• Kickstarter for Indie Games
• How to Launch Your Game on itch.io
• Co-op Game Design — co-op games consistently overperform at Next Fest
• Analytics for web games — tracking demo performance and conversion metrics
• Game Jams & Hackathons — building audience before your first festival

Last updated: June 2026.

Here's the thing about generative AI — for years, the best models lived behind API keys and unpredictable pricing. You'd build your workflow around a tool, get comfortable, then wake up to an email saying the pricing changed. Or worse, the company pivoted entirely.

That changed in late 2024. Tencent, Alibaba, DeepSeek — they started releasing models you can actually download. Models that rival the closed alternatives. And suddenly, creators have options that don't depend on someone else's business model.

What if you could generate video, 3D assets, music, and voices — all from models you control? That's where we are now. This guide walks through what's real, what works, and what you can start using today.

Video Generation

For years, video generation meant Runway or Pika — closed platforms, subscription fees, limits on what you could do with the output. Now? You can run comparable models on your own hardware.

HunyuanVideo text-to-video generation — 720p output from the leading open-source video model

Weights: HunyuanVideo ↗ · Mochi 1 ↗ · Wan 2.1 ↗ · Open-Sora ↗

See samples: HunyuanVideo gallery ↗ · Mochi examples ↗ · CogVideoX samples ↗

What this means for creators

HunyuanVideo outperforms Runway Gen-3 in professional evaluations, and it's fully open. The catch? You need serious hardware. An A100 or H100 with 80GB VRAM. For most of us, that means renting cloud GPUs when you need them.

HunyuanVideo-1.5 changed the calculus on hardware. Tencent released it in November 2025 as an 8.3B-parameter open model that runs on a 14GB consumer GPU, with text-to-video and image-to-video at 480p/720p and optional 1080p upscaling. A new Selective and Sliding Tile Attention (SSTA) mechanism gives it roughly double the inference speed of the original HunyuanVideo. If you wanted HunyuanVideo quality but couldn't justify an 80GB card, this is the version you can actually run at home.

Mochi 1 is the one you can actually run. A 12GB GPU — that's RTX 3060 territory — handles it fine. The output is genuinely creative, with a distinct artistic quality. Not quite HunyuanVideo's fidelity, but you own the process.

LTX-2 is where things get interesting for games. It's the first open model that generates synchronized audio with video. Imagine cutscenes where the sound just... matches. No post-production sync. Lightricks open-sourced the full weights, inference, and training code in January 2026, with native 4K output at up to 50fps and synced audio up to 20 seconds.

Wan 2.1 runs on a gaming laptop. An 8GB GPU works for the smaller variants. If you've ever wanted to prototype with video generation but couldn't justify the hardware, this is your path in.

Wan 2.2 (Alibaba, July 2025) is the first open-source video model built on a Mixture-of-Experts design: 27B total parameters with only 14B active per step. It ships as text-to-video (T2V-A14B), image-to-video (I2V-A14B), and a hybrid 5B variant that runs on consumer GPUs, all under Apache 2.0 for commercial use.

The workflow that makes sense: Mochi 1 or Wan 2.1 for prototyping locally. HunyuanVideo on cloud GPUs when you need final quality.

Image Generation

This is where open-source already won. The models you can download today genuinely compete with Midjourney. Not "almost as good" — actually competitive.

FLUX.1 samples — photorealistic quality from an Apache 2.0 licensed model

Try them directly: FLUX.1 schnell demo ↗ · SD 3.5 Large demo ↗ · GitHub (FLUX) ↗

See samples: FLUX gallery ↗ · FLUX LoRA gallery ↗ · Replicate examples ↗

For building game assets

FLUX.1 [schnell] is the one to know. Apache 2.0 license — meaning you can ship commercial games without worrying about licensing drama. It generates in just 4 steps, so you can iterate fast. Describe what you want, see the result, adjust, repeat.

SD 3.5 Large finally handles text rendering properly. Previous versions mangled any text you tried to include. This matters for UI mockups, in-game signage, title screens — anywhere you need readable words in your images.

FLUX.2 (Black Forest Labs, November 2025) is the bigger successor: a 32B model that handles text-to-image and image editing in one checkpoint, with strong multi-reference character/style consistency and reliable text rendering at up to 4 megapixels. The open-weight FLUX.2 [dev] uses a non-commercial license, but the size-distilled FLUX.2 [klein] ships under Apache 2.0, so there's still a commercial-safe option for shipping game art. Quantized pipelines bring [dev] down to 18-24GB GPUs.

The ecosystem around Stable Diffusion is still unmatched. ControlNet for precise composition. Inpainting for fixes. LoRA fine-tuning for custom styles. FLUX is catching up, but if you need deep customization today, SD's tooling maturity gives you more to work with.

Here's how I'd think about it: textures and sprites, either works. Concept art with specific style requirements, SD 3.5 with LoRAs. Pure quality for commercial shipping, FLUX schnell.

3D Generation

If you've ever spent eight hours modeling a prop that appears in your game for three seconds, this section is for you. 3D generation went from "interesting research" to "actually usable" in 2024. You can now go from a sketch to a textured mesh in under a minute.

TRELLIS generates textured 3D meshes with PBR materials from single images

Try them directly: TRELLIS 2 demo ↗ · Hunyuan3D demo ↗ · InstantMesh demo ↗

See samples: TRELLIS 2 project page ↗ · 3D AI Studio gallery ↗

A workflow that actually works

The approach that's clicking for creators right now chains models together. Start with an image — generated or photographed, doesn't matter. Run it through Stable Zero123 or Wonder3D to get multiple views. Feed those views to InstantMesh or TripoSR for the mesh. Then TRELLIS 2 or Hunyuan3D for proper materials.

TRELLIS 2 from Microsoft is the new leader for production-ready assets. It handles the geometry that breaks other models — thin surfaces, holes, complex topology. The 4B parameter version outputs meshes with real PBR textures, not just vertex colors pretending to be materials.

TripoSR is about speed. Half a second from image to mesh. The mesh needs cleanup and texturing, but for prototyping? For figuring out if an idea works before you invest hours? Unbeatable.

Hunyuan3D 2.5 focuses on simulation-ready assets. Game props that actually work in physics engines without manual fixes. No more invisible collision issues because the mesh topology is weird. Its LATTICE shape model scales up to 10B parameters for sharper detail, with PBR texturing built in.

Here's the realistic expectation for indie creators: generate concept art with FLUX, run it through InstantMesh for geometry, then texture in Blender or use TRELLIS for automated PBR. You're looking at 30-60 minutes per asset instead of 4-8 hours. Not zero time — but a real difference.

Audio and Music

Audio generation hasn't caught up to images and video yet. But there's enough here to change how you work — especially for prototyping and sound effects.

AI-generated music sample — describe the mood you want, get music that fits

Try them directly: MusicGen demo ↗ · AudioCraft playground ↗

See samples: MusicGen examples ↗ · AudioGen samples ↗

What you can actually ship with

MusicGen from Meta is the practical choice for game audio. Describe the mood you want, get music that fits. MIT license means you can ship it. The 3.3B model runs fine on a 12GB GPU — describe, generate, iterate.

AudioGen handles sound effects: footsteps, door creaks, ambient wind, mechanical sounds. Same deal — MIT licensed, runs locally, actually useful for filling out your game's soundscape.

YuE is genuinely exciting. It's the first open model that generates full songs with vocals. Theme songs. Background music with actual singing. The quality varies, but it's miles ahead of anything else you can download and run yourself.

ACE-Step (StepFun and ACE Studio, May 2025) is the open music model worth knowing now. It's a 3.5B Apache 2.0 foundation model that generates roughly four minutes of music in about 20 seconds on an A100, supports 19 languages, and handles voice cloning, remixing, and lyric editing. For game prototyping it closes a lot of the gap that YuE and MusicGen left open.

Stable Audio Open is limited. 47-second clips, research-only license. Good for prototyping ideas, not for shipping.

Here's the honest take: the gap between open models and closed ones (Suno, Udio) is still real for music. For sound effects, open models are genuinely competitive. For full songs you want to ship, expect to iterate heavily — or bring in a musician for final production and use these tools for everything else.

Speech and Voice

Voice generation crossed into "good enough for games" territory in 2024. And that changes what's possible for small teams.

AI-generated game narration — natural speech with proper pacing and emotion

Hear samples: Fish Audio voices ↗ · OpenVoice demo ↗

Making NPCs sound like people

CSM (Conversational Speech Model) from Sesame was built specifically for dialogue. It produces natural pauses. Intonation shifts. The rhythm of actual conversation. Most TTS sounds like someone reading a script — you can hear it instantly. CSM sounds like someone talking. That difference matters more than you'd think.

Fish Speech and OpenVoice handle voice cloning. Record 10-30 seconds of a voice actor, then generate unlimited dialogue in that voice. Think about what this means: you can hire voice talent for key lines, then extend their performance to cover hundreds of variations and ambient dialogue.

NVIDIA ACE (not fully open, but worth knowing) now supports Qwen3-8B for on-device NPC deployment. Local LLM + local TTS + lip sync — all running on consumer GPUs. This is the stack for real-time NPC conversations that don't need cloud calls.

The approach that makes sense for indie creators: hire voice actors for main characters and the lines that matter most. Use Fish Speech or OpenVoice to extend coverage for ambient dialogue, variations, and all the incidental lines that would otherwise be silent or prohibitively expensive.

World Models and Game Simulation

This is where things get genuinely weird — and genuinely exciting. These models don't generate static assets. They generate experiences that feel like games.

See the research: DIAMOND project page ↗ · Cosmos blog ↗

Try it: Oasis live demo ↗ · Genie 2 examples ↗

Why you should care about this

DIAMOND proved something that changes how you think about game AI. You can train an agent entirely inside a generated world. No real game engine needed for training. The AI plays in a diffusion model's imagination — and then transfers to the real game. The implications here are significant.

Oasis runs a Minecraft-like world in real-time. Frame by frame. No game engine, no textures, no pre-built assets. Just a transformer predicting what comes next. It's a proof of concept, but imagine where this goes. The 500M parameter version is already open.

GameGen-X released the largest dataset for open-world game video. If you want to train your own models or fine-tune existing ones to generate game-like content, this is your starting point.

NVIDIA Cosmos was built for robotics and autonomous vehicles, but the world foundation models work for games too. They understand physics. Object permanence. Spatial relationships. Open weights, permissive licensing.

Genie 3 (DeepMind, announced August 2025) is the leap worth noting: the first world model with real-time interaction, generating navigable 720p worlds at 24fps that stay consistent for a few minutes, plus 'promptable world events' that change weather or add objects on command. It opened to the public as Project Genie in January 2026 for Google AI Ultra subscribers in the US. Still closed weights, but it shows where playable, generated worlds are heading.

For practical game development today, these are still research tools. But if you're working on AI-driven content, procedural generation, or just thinking about where this is all going — this is the frontier.

Large Language Models

LLMs power dialogue, quest generation, and game logic. And the open options now genuinely compete with GPT-4. This wasn't true two years ago.

Get started: Qwen3-8B on HuggingFace ↗ · DeepSeek-V3 on HuggingFace ↗

For building games

Qwen3 is the practical choice for most game uses. Apache 2.0 license — meaning you own your integration. Strong multilingual support, which matters if you're thinking about localization. Good at following structured instructions. The 7B and 14B variants run locally on consumer GPUs.

DeepSeek-V3 matches or beats GPT-4 on most benchmarks. The architecture is clever — only 37B parameters activate per token despite the 671B total. You need serious hardware (multi-GPU), but the quality is frontier-level without the API dependency.

DeepSeek-V3.2 (December 2025) is the current open DeepSeek frontier. It folds reasoning and tool-use into one model and introduces DeepSeek Sparse Attention (DSA) for cheaper long-context inference, with a high-compute Speciale variant aimed at the top reasoning benchmarks. For game logic and dialogue it's a stronger, more agent-capable drop-in than V3.

Qwen2.5-VL adds vision understanding. Useful for games that need to analyze screenshots, understand player-drawn content, or process camera input. The 7B variant runs on a single GPU.

For on-device NPCs — characters that respond in real-time without cloud calls — Qwen3-8B through NVIDIA ACE is the most practical path right now. It runs alongside your game on the player's hardware.

Utility Models

These don't generate content directly — but they make your pipelines work.

SAM 2 segments any object in images and video — click once, get a perfect mask

SAM 2 segments objects in video in real-time. Click on something, get a perfect mask. Useful for rotoscoping, compositing, or extracting objects from footage to use as game assets. Try SAM 2 ↗

Depth Pro from Apple produces metric depth maps from single images in under a second. This unlocks a lot: converting 2D art to 2.5D with parallax effects, generating depth data for 3D reconstruction, creating normal maps from flat images. Depth Pro on HuggingFace ↗

gsplat is the fast implementation of Gaussian splatting. If you're capturing real environments for games — photogrammetry, environment scans — this is the library that makes it practical.

What I'd Actually Use

If you're starting a game project today, here's the stack that makes sense:

Textures and sprites: FLUX.1 [schnell] — Apache 2.0, fast iteration, quality that ships

Concept art: SD 3.5 Large with LoRAs for style control

3D assets: InstantMesh for geometry, then Blender for texturing or TRELLIS 2 for automated PBR

Sound effects: AudioGen — MIT licensed, runs locally, fills out your soundscape

Music: MusicGen for prototypes, then bring in a composer for final production

Voice: Fish Speech for prototyping, voice actors + cloning for production

NPC dialogue: Qwen3-8B locally, or cloud LLM for complex reasoning

Video (cutscenes): Mochi 1 locally, HunyuanVideo on cloud when you need final quality

Here's the thing about all of this: the common mistake is trying to use AI for everything. These are tools, not replacements. They compress the tedious parts — iteration, variations, placeholder assets — so you can spend your time on the creative decisions that actually matter. The parts that make your game yours.

Hardware Reality Check

Let's be honest about what you actually need to run this stuff:

8GB VRAM (RTX 3060, 4060): SD 1.5/SDXL, Wan 2.1 small, AudioGen, Fish Speech, small LLMs (7B quantized). This is gaming laptop territory — and it's enough to get started.

12GB VRAM (RTX 3080, 4070): SD 3.5, FLUX schnell, Mochi 1, MusicGen, TripoSR, Qwen 14B quantized. This is where things get comfortable. Most of the useful models run here.

24GB VRAM (RTX 3090, 4090): Most models at full precision, InstantMesh, larger LLMs. If you're serious about this workflow, this is the sweet spot.

48-80GB VRAM (A100, H100): HunyuanVideo, LTX-2, DeepSeek-V3, production-scale generation. Enterprise hardware. You're not buying this — you're renting it.

Cloud instances on RunPod, Lambda Labs, or Modal cost $2-4/hour for A100s. For occasional use, that's cheaper than hardware. Spin up when you need final quality, shut down when you're done.

About the cost estimates in this guide: Per-generation costs assume self-hosted inference on cloud GPUs at ~$2-3/hour (A100) or ~$0.40/hour (RTX 4090). Actual costs vary based on hardware, optimization, and batch sizes. These are ballpark figures for planning — your mileage will vary.

What's New in 2026

Just released: LTX-2 weights dropped — the first open model with synchronized audio and video. Hunyuan3D 2.5 is now available for simulation-ready 3D assets that work in physics engines.

Coming this year: Real-time video generation with sub-second latency. Better world models for game simulation. And smaller models that run on integrated graphics — meaning laptops without dedicated GPUs.

The trajectory is clear: every capability that exists in closed models shows up in open models 6-12 months later. The question isn't whether open models will be good enough — they already are for most uses. The question is how fast they become the default.

And here's what that means for creators: the tools that used to require enterprise budgets or monthly subscriptions are becoming something you can just... run. On your own hardware. With no one else's permission.

That's the shift. That's what we're building toward.

Common Questions

What is the best open-source AI model for generating game assets?

It depends on the asset. For 3D models, Hunyuan3D is the strongest open option in 2026. For 2D art and textures, FLUX leads on quality. For sound effects and music, the open audio models have caught up fast. There's no single "best" model because games need many asset types, so most creators chain a few together rather than relying on one.

Are open-source AI models good enough to replace closed APIs?

For most game-asset work in 2026, yes. Open models now match closed APIs on image, 3D, and audio generation, and you can run them on your own hardware with no per-call fees or surprise pricing changes. Closed models still lead on a few frontier tasks like long-form video, but the gap usually closes within 6 to 12 months.

Can I run these generative AI models on my own GPU?

Many of them, yes. Image and audio models run comfortably on a single consumer GPU like an RTX 4090. Larger video and 3D models want more VRAM and often an A100-class cloud GPU. The hardware section above lists what each model needs so you can plan before you commit.

Is it legal to use AI-generated assets in a commercial game?

Generally yes for open-source models you run yourself, but it depends on the model's license and your training-data assumptions. Always check the specific model license, and disclose AI-generated content where your platform requires it. See our AI-generated content policy for how we handle this.

More Reading

• AI controversy, trust, and the post-AI economy — our position on AI in games
• AI-generated content policy — how we handle AI disclosure
• Web Games Tech Stack in 2026 — WebGL, WebGPU, and Wasm for games
• Browser 3D Open World Tech — using AI-generated 3D assets in browser worlds
• Landscape Generation for Browser Open Worlds — diffusion-based terrain synthesis
• Where to Find Free Game Assets — traditional asset sources alongside AI generation
• Agentic AI code tools — AI for writing game code, not just generating assets

Last updated: June 2026.

Everything you need to know about asset licenses so you can confidently use free and paid assets in your games without legal trouble.

Why Licenses Matter

Every creative work - 3D models, textures, sounds, music - is protected by copyright. The creator owns it by default. A license is the creator's permission for you to use their work under specific terms.

Using assets without understanding their license can lead to:

• DMCA takedowns of your game
• Legal action from copyright holders
• Forced removal from app stores
• Financial damages

The good news: Most game assets have clear, permissive licenses. You just need to understand what they allow.

License Quick Reference