BatchedMesh addon

[takahirox]

Related issue: #22376

Description

This PR adds BatchedMesh to addons (not to the core).

BatchedMesh allows to render many dynamic and different shape objects referring to the same material with a single draw call. Please read #22376 (comment) for basic concept in more details.

The idea is originally from @donmccurdy

Demo: https://raw.githack.com/takahirox/three.js/BatchedMeshAddon/examples/index.html#webgl_mesh_batch

API

// Initialize
const mesh = new BatchedMesh(material, maxGeometryCount, maxVertexCount, maxIndexCount);
const ids = [];

for (let i = 0; i < geometries.length; i++) {
  const id = mesh.applyGeometry(geometries[i]);
  mesh.setMatrixAt(id, matrices[i]);
  ids.push(id);
}

// update
for (let i = 0; i < ids.length; i++) {
  const id = ids[i];
  mesh.getMatrixAt(id, matrix);
  matrix.decompose(position, quaternion, scale);

  // update translate/quaternion/scale here

  matrix.compose(position, quaternion, scale);
  mesh.setMatrixAt(id, matrix);
}

Benefits

Good rendering performance by reducing the number of draw calls.

On my Windows 10 + Chrome, the demo above runs at 60fps with 8192 dynamic and different shape objects. The fps counter drops to like 40fps if I switch to regular meshes.

Implementation

• Packing multiple geometries into a single geometry. This is the key for reducing the draw calls.
• Using data texture to handle local matrix per geometry
• Using material.onBeforeCompile() hook to customize the vertex shader for handling local matrices

TODOs

There are lot of TODOs left. You will find them in the code. And I'm not sure if tested well enough.

If the basic concept and API looks fine, I hope this PR can be merged and other contributors will help for testing and implementation.

Whether to get it in the core

BatchedMesh API may greatly fit to WEBGL_multi_draw. With BatchedMesh can be further optimized with WEBGL_multi_draw. But it requires some changes in the core.

As discussed in #22376, it would be good to start with addons without WEBGL_multi_draw because we are not sure if it is a good moment to add change in the core now.

After adding BatchedMesh into addons, we may think of getting it into the core if all the followings are satisfied.

• Many users are interested in BatchedMesh API and demand for more stable support
• Write a test with WEBGL_multi_draw and confirm it provides us noticable performance improvement
• Good moment to add some changes in the core

Also see #22376 (comment)

[takahirox]

FYI: If we want to support color per geometry, we may add another data texture for color.

Demo: https://raw.githack.com/takahirox/three.js/BatchedMeshColorAddon/examples/index.html#webgl_mesh_batch

Branch: dev...takahirox:three.js:BatchedMeshColorAddon

It may be good if it allows arbitary material (uniform) parameter per geometry, but I couldn't come up with good API. Maybe node based material system fits to it? It may be future work.

[donmccurdy]

Can be optimized more with WEBGL_multi_draw.

To me this is a big question ... doesn't WEBGL_multi_draw constrain the number of objects per batch quite a bit? Which could still be the better choice, but probably means a larger number of smaller batches, ideally co-located for culling, and more complex batch management...

I suspect it'll take us some time to figure that out, so I'm inclined to do our experimentation in addons and not rush this into core right away.

[takahirox]

Yeah, I don't think we should be in rush to move it to the core and use WEBGL_multi_draw, or even I'm not sure we really need to do them now. To clarify it, I added "future work" to the comment.

[donmccurdy]

I think maxGeometryCount and maxVertexCount are probably two parameters that users really need to provide an intentional value for. We don't give a default count for InstancedMesh either. I'd be fine with material or maxIndexCount maybe having default values though. How about:

constructor( maxGeometryCount, maxVertexCount, maxIndexCount = maxVertexCount * 2, material = new MeshBasicMaterial() );

[takahirox]

Sounds good to me. Updated the constructor parameters. Default material is defined in Mesh (super) constructor, so I don't define it here.

[donmccurdy]

just an idea — it might be nice to have some getters...

• .geometryCount
• .vertexCount
• .indexCount

... so users can see how much space is left in the batch. The second two values would presumably change after removing a geometry and then calling .optimize().

[takahirox]

Sounds good to me. Added the getters.

And I renamed ._currentGeometry/Vertex/IndexCount to ._geometry/vertex/indexCount. I don't really remember why I named them _current.

[donmccurdy]

nit: I think removeGeometry or deleteGeometry would be more consistent with other three.js APIs.

[takahirox]

OK, updated. Thanks.

[donmccurdy]

I like .optimize(). Or .rebuild() would also work.

[takahirox]

OK, removed the TODO comment.

[donmccurdy]

I'm undecided on this ... would .setVisibleAt be better? Just to match .visible.

[takahirox]

OK, updated. Thanks.

[donmccurdy]

In general — I think it's OK (and possibly better) for these methods to hit runtime errors naturally on invalid input; we don't typically sanitize it, and I silently doing nothing may cause developers some headaches.

[takahirox]

OK, removed the TODOs. Thanks.

[takahirox]

Similar to #25078 there might be a chance that BatchedMesh would be better not to extend Mesh for flexibility. But I hope we can go with this PR for now and revisit later if needed.

[ghost]

Just wanted to point out I tested this with multiple TextGeometry and manged to get some performance improvement as well 👍.

I imagine texts require a lot more computing since I can't get good enough fps when rendering many instances.

Without BatchedMesh, pure ThreeJS + troika-three-text (I have an example in https://github.com/rebeckerspecialties/threejs-tests). I opted to use troika since I got slight better performance than just TextGeometry:

• Oculus Go: ~60fps with 22 texts - ~20fps with 100 texts
• Hololens2: ~60fps with 27 texts - ~15fps with 100 texts

With BatchedMesh, TextGeometry from addon (no troika):

• Oculus Go: ~60fps with 110 texts
• Hololens2: ~60fps with 40 texts

Not related to this PR, but I can't seem to mix BatchedMesh with texts from troika-three-text.
When calling batchedMesh.applyGeometry(troikaText.geometry), it just rendered a rectangle in the scene with no texts.
I wonder if it's even possible to batch it, that would be really nice.

[ghost]

Not related to this PR, but I can't seem to mix BatchedMesh with texts from troika-three-text.
When calling batchedMesh.applyGeometry(troikaText.geometry), it just rendered a rectangle in the scene with no texts.
I wonder if it's even possible to batch it, that would be really nice.

Just looping back to this PR - I've been trying (somewhat intensively 😅) to integrate BatchedMesh with troika-three-text, but no luck so far. I also tried to use the material created in troika to instance the BatchedMesh, but that seems to break the API.

@takahirox, would you know if it's possible to integrate BachedMesh with troika-three-text? I can create a sandbox of what I tried if necessary.

[donmccurdy]

@william-mimura-thisdot i think the underlying question here would be, does troika support rendering many text instances from the same geometry and shader material? If you cannot merge troika text with BufferGeometryUtils.mergeBufferGeometries( ... ), because each instance requires different shader parameters, then BatchedMesh will not work out of the box either. The Troika developers could probably extend their project to either (a) support merged geometries, or (b) use batching by default. I'm not sure how their shader materials work, or what changes that would require.

[takahirox]

Any blocking conerns? @mrdoob @Mugen87 Do you think it's ok to add this PR to r150 milestone?

[vlucendo]

This is very useful, thank you.

I believe performance could be optimized a bit more by setting the updateRange of buffers as new geometries are added. Also, as @donmccurdy mentioned, updating the index buffer and the draw range could enable frustum culling.

I too would love to see support for WEBGL_multi_draw in the core at some point.

[takahirox]

Let me put this PR to r152 milestone as a reminder because many users are interested in, and no change is required in the core so it doesn't add complexity.

[chaogao]

Thanks for your addon, can I use raycaster or other solutions to select geomertyid?

[takahirox]

Unfortunately, I think raycasting raycasts the entire geometry, not per geometry instance, in the current PR. But I think it would be possible to implement. Please feel free to make a follow up PR if this PR lands.

[gkjohnson]

@Mugen87 @mrdoob I think this would be good to get in sooner rather than later. Geometry batching can be a big benefit to performance for complex applications and it would be great to take a step towards hashing out the API and using the WEBGL_multi_draw extension.

I've had multiple clients with draw call performance issues with dynamic geometry that I would have liked to point at something like this but so far I can only say that this feature may be available at some point in the future.

[Mugen87]

Sorry for the delay on this one! Reviewing now...

[Mugen87]

I have applied some minor refactoring in the example to make it more similar to the ones using InstancedMesh. I guess the screenshot needs an update now but we can take care of this after the merge, too.