NodeMaterial: Add support for compute() integrated into the material

[sunag]

Related issue: Fixes #30612 (comment)

Description

Compute can now be used directly on materials. They will only be computed before the object is rendered, if the object is not in the camera view or is invisible for another reason the computation will also be ignored.

The Node returned within compute() is computed directly in the Material, while all other calculations within Fn will be done just before the object is rendered by compute().

The example below guides the position of Sprites based on PointsNodeMaterial and SkinnedMesh.

const pointPositionArray = instancedArray( countOfPoints, 'vec3' );

const skinningPosition = computeSkinning( skinnedMesh );

const materialPoints = new THREE.PointsNodeMaterial();
materialPoints.sizeNode = float( 5 );
materialPoints.sizeAttenuation = false;
materialPoints.positionNode = Fn( () => {

	const pointPosition = pointPositionArray.element( instanceIndex );
	const skinningWorldPosition = objectWorldMatrix( skinnedMesh ).mul( skinningPosition );

	pointPosition.assign( skinningWorldPosition );

	// return the value for positionNode

	return pointPositionArray.toAttribute();

} )().compute( countOfPoints );

const pointCloud = new THREE.Sprite( materialPoints );
pointCloud.count = countOfPoints;
scene.add( pointCloud );

[github-actions]

📦 Bundle size

Full ESM build, minified and gzipped.

	Before	After	Diff
WebGL	336.08 78.29	336.08 78.29	+0 B +0 B
WebGPU	525.21 146.28	526.03 146.59	+826 B +310 B
WebGPU Nodes	524.67 146.18	525.5 146.49	+826 B +311 B

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

	Before	After	Diff
WebGL	465.15 112.18	465.15 112.18	+0 B +0 B
WebGPU	598.01 162.57	598.23 162.62	+220 B +47 B
WebGPU Nodes	553.1 152.04	553.32 152.09	+220 B +51 B

[cmhhelgeson]

Does this mean that the type of an assigned positionNode will be inferred based on whether compute() is called? For instance, if material.positionNode is assigned the value of Fn(() => { return positionLocal })(), will that be evaluated as code that should run in the vertex shader, whereas Fn(() => { return positionLocal })().compute() will try be evaluated as a compute shader?

[sunag]

@cmhhelgeson Compute functions do not return values ​​by default, but now the returned value will be bound to the chosen material input. In this sense, the computation will be done whenever the material is rendered, the value returned from the compute function will be the value that will be generated in the vertex-stage or fragment-stage as the user created the code.

You can return positionLocal in this case, but to compute an attribute you still need to use storage().

[cmhhelgeson]

@cmhhelgeson Compute functions do not return values ​​by default, but now the returned value will be bound to the chosen material input. In this sense, the computation will be done whenever the material is rendered, the value returned from the compute function will be the value that will be generated in the vertex-stage or fragment-stage as the user created the code.

You can return positionLocal in this case, but to compute an attribute you still need to use storage().

I'm not sure I understand, so the values that gets assigned to position in positionNode will equal the values that are returned per each vertex/instance in the compute shader. Additionally, the compute shader generated by assigning a ComputeNode to positionNode is automatically executed once per material render, meaning the code flow per frame would be something along the lines of A. Run compute node/compute shader to assign position values -> B. Run position/vertex shader which will use the values computed by the compute shader as its position values.

[sunag]

A. Run compute node/compute shader to assign position values -> B. Run position/vertex shader which will use the values computed by the compute shader as its position values.

Yes, that's basically it. The main difference here is that you don't need to use renderer.compute() and then assign the buffer to the material in isolation, this is done automatically in the moment the TSL Fn return a value. This should simplify creation and optimize once the compute process now consider the frustrum culling.

[mrdoob]

Definitely much better than the renderer.compute() API 💯
I think we should update the examples to use the new API wherever possible.