Machine Words

Creating portals in three.js

Talin's photo
Talin
·

9 min read

https://www.youtube.com/watch?v=HF2QVbK40VQ

In this article, I’ll show you everything you need to know in order to create portals in 3D scenes using the three.js framework.

Portals are probably best known because of the game Portal published by Valve software, but they are useful in all kinds of games and graphics. An example of this is connecting various game levels together, such as a portal that provides a transition between and above-ground and underground environment. Portals can also be used to simulate mirrors and reflective surfaces.

Conceptually, a portal is a kind of in-scene “window” which displays a view into a different part of the virtual world. The portal has two “ends”, one which is the “local” or “source” end that is close to the player’s current location, and a “remote” or “destination” end which represents the location seen through the window. Each end has a separate camera. For purposes of this article, the “primary” camera is the local camera — the one that renders the scene surrounding the portal. The other camera, which we’ll call the “virtual” camera, represents the view within the portal.

Code Organization

Because we may want to have more than one portal in a scene, we’ll need a way to organize the portals which are currently in view. I manage this by creating an ActivePortal class that contains the details for one portal, and then maintain an array of these objects. The ActivePortal class has a number of key methods:

  • Setter methods to adjust the parameters of the portal, including source position, destination position and aperture size.
  • An update() method which is called every frame before rendering to compute the on-screen location of the window.
  • A render() method which is called for each portal after the main scene is rendered.

For a small scene, you can get by with pre-creating ActivePortal instances for every portal in the world. However, for a large game world, you’ll probably want to create them dynamically — adding new ActivePortal instances that are near to the current view position, and removing ones that are too far away.

Synchronized Cameras

One key to making the portal illusion is that the two cameras need to be synchronized in their movements. I like to think of this as working like an old-fashioned pantograph, where the movement of one end causes a corresponding movement in the other end.

We start with the coordinates of each end (note all the code in this article is in TypeScript):

class ActivePortal {
  public readonly sourcePosition = new Vector3();
  public readonly destinationPosition = new Vector3();
}

To calculate the position of the virtual camera, we compute the difference vector between the near and far ends of the portal, and apply that difference to the virtual camera, relative to the primary camera. This calculation is done on every update. Note that both the position and lookAt parameters need to be set.

this.differential
  .copy(this.destinationPosition)
  .sub(this.sourcePosition);

Aperture Size

Another thing we need to know is how big the portal is going to be. Theoretically, a portal doesn’t need to be “flat”, it can be any 3D shape. However, for most purposes you’re going to want a plane surface. For my code, I just use a Box3 to represent the portal aperture, with one of the dimensions set to zero.

Rendering the Aperture

In the primary scene, we’ll render the aperture using a simple box. This box will use a material that renders to the OpenGL stencil buffer. To do this, we’ll need to make a mesh:

this.geometry = new BoxBufferGeometry(
  this.apertureSize.x,
  this.apertureSize.y,
  this.apertureSize.z
);
this.material = new MeshBasicMaterial({
  colorWrite: false,
});
this.mesh = new Mesh(this.geometry, this.material);
this.mesh.geometry.computeBoundingSphere();
this.mesh.frustumCulled = true;
this.mesh.matrixAutoUpdate = false;
this.mesh.renderOrder = 2;
this.mesh.visible = true;
this.mesh.name = 'Portal';

What this mesh is going to do is render a quad shape to the stencil buffer in the primary scene. A later rendering pass will fill in the content of the window.

Note: the reason for setting the renderOrder to 2 is because we want to render the portals after all of the other objects in the primary scene. The reason for this is somewhat non-intuitive: we render the aperture last because we don’t want it to overwrite objects that are in front of the portal. This is because of the stencil buffer: when the aperture is rendered, the depth test is enabled, so the stencil buffer won’t be written in places where there are objects closer to the primary camera than the portal. If we rendered the portals first, then the stencil buffer would be written unconditionally, and any pixels in front of the portal would get overwritten on the subsequent rendering pass.

(Note that if you have translucent objects in the primary scene, you’ll probably want to render them after all of the portals have been rendered, but that is an advanced topic which I won’t cover in this article.)

Because we’re going to be rendering multiple portals, we’ll need a way to distinguish them at the pixel level. This can be done by giving each active portal a unique integer index, and writing that index to the stencil buffer. So the first portal will write the value 1 to the stencil, the second portal uses value 2, and so on. The value of 0 represents the areas of the screen where there is no portal. This theoretically allows up to 255 portals, although it’s unlikely you would want this many because rendering would be very slow.

(There is another way to do portals without using stencils, which is by manipulating the depth buffer — in that technique the portal camera is rendered before the main scene. This technique is simpler in some ways, but it has trouble when you have more than one portal on the scene.)

Although three.js has a way to tell a material to write to the stencil buffer, I needed finer control than what three.js allows. Instead, I use callbacks to set the OpenGL state directly, which also turns out to be simpler and easier to understand than using the three.js API:

this.mesh.onBeforeRender = renderer => {
  if (this.mesh.visible) {
    const gl = renderer.getContext();
    gl.enable(gl.STENCIL_TEST);
    gl.stencilMask(0xff);
    gl.stencilFunc(gl.ALWAYS, this.stencilIndex, 0xff);
    gl.stencilOp(gl.KEEP, gl.KEEP, gl.REPLACE);
  }
};
this.mesh.onAfterRender = renderer => {
  if (this.mesh.visible) {
    // Set everything back to the way it was before
    const gl = renderer.getContext();
    gl.disable(gl.STENCIL_TEST);
    gl.stencilMask(0);
    gl.stencilFunc(gl.ALWAYS, 1, 0xff);
    gl.stencilOp(gl.KEEP, gl.KEEP, gl.KEEP);
  }
};

Computing the screen bounds of the portal

Once we have rendered the primary scene, we need to render each portal as a separate pass. During this pass, we’ll be rendering to a small subset of the screen. This requires computing the 2D bounding box of the aperture. This computation happens in the update() method, before rendering.

const widthHalf = screenSize.width / 2;
const heightHalf = screenSize.height / 2;
this.nearDepth = Infinity;
// Start by making the bounding box empty
this.portalScreenRect.makeEmpty();
const addPortalPoint = (x: number, y: number, z: number) => {
  // Project point to screen space from aperture space
  this.worldPt.set(x, y, z);
  this.worldPt.applyMatrix4(this.mesh.matrixWorld);
  this.worldPt.project(camera);
  // Convert to pixels. Note rounding to prevent jitter.
  this.screenPt.x = Math.round(
    this.worldPt.x * widthHalf + widthHalf);
  this.screenPt.y = Math.round(
    -(this.worldPt.y * heightHalf) + heightHalf);
  // Expand the bounding box to include the point.
  this.portalScreenRect.expandByPoint(this.screenPt);

  // Also track the depth of the nearest aperture corner.  
  // This is used to sort the portals by depth.
  this.worldPt.applyMatrix4(camera.projectionMatrixInverse);
  this.nearDepth = Math.min(this.nearDepth, -this.worldPt.z);
};
// Compute all 8 points of the aperture box and expand the rect.
addPortalPoint(-apertureSize.x, -apertureSize.y, -apertureSize.z);
addPortalPoint(apertureSize.x, -apertureSize.y, -apertureSize.z);
addPortalPoint(-apertureSize.x, apertureSize.y, -apertureSize.z);
addPortalPoint(apertureSize.x, apertureSize.y, -apertureSize.z);
addPortalPoint(-apertureSize.x, -apertureSize.y, apertureSize.z);
addPortalPoint(apertureSize.x, -apertureSize.y, apertureSize.z);
addPortalPoint(-apertureSize.x, apertureSize.y, apertureSize.z);
addPortalPoint(apertureSize.x, apertureSize.y, apertureSize.z);
// Add an extra 2 pixels around the edge of the screen rect to
// account for rounding errors.
this.portalScreenRect.expandByScalar(2);

At this point, we can now determine whether the portal is actually on screen or not:

this.mainScreenRect.min.set(0, 0);
this.mainScreenRect.max.copy(screenSize);
this.isOnscreen = this.mainScreenRect.intersectsBox(
  this.portalScreenRect);
this.mesh.visible = this.isOnscreen;

If the portal is not on screen, we can skip rendering it.

Next, we need to convert the portalScreenRect to a form which can be used to adjust the camera view offset and the scissoring rect. This calculation can also be done in the portal’s update() method:

// Compute viewport coordinates for scissor (Vector4).
this.portalViewport.x = this.portalScreenRect.min.x;
this.portalViewport.y = 
  screenSize.height - this.portalScreenRect.max.y;
this.portalViewport.z = 
  this.portalScreenRect.max.x - this.portalScreenRect.min.x;
this.portalViewport.w = 
  this.portalScreenRect.max.y - this.portalScreenRect.min.y;
this.portalCamera.setViewOffset(
  screenSize.width,
  screenSize.height,
  this.portalScreenRect.min.x,
  this.portalScreenRect.min.y,
  this.portalScreenRect.max.x - this.portalScreenRect.min.x,
  this.portalScreenRect.max.y - this.portalScreenRect.min.y
);

Managing Clip Planes

One common problem with portals is dealing with objects that come between the camera and the portal. For the primary camera, we want this objects to appear on top of the portal. For the virtual camera, however, we don’t. Any objects that are between the virtual camera and the remote end of the portal should be omitted from the scene.

There are a couple of ways to do this — an easy but somewhat inaccurate way to do this is to adjust the near clipping plane of the virtual camera, based on the distance to the portal (which we have already computed).

However, the most robust way to do this is to use three.js’s clipping planes feature.

However, there’s a catch: the number of clipping planes is baked into every three.js shader at compile time. If you change the number of clipping planes, it has to recompile every shader, which is super slow.

The way to get around this is to always have the same number of clipping planes, even in the primary scene. To do this, the primary scene will have a “dummy” clip plane that is far enough away that it doesn’t clip anything.

Rendering the portal content

Because we’re rendering in multiple passes, we don’t want to clear the screen at the beginning of each pass. So we’ll need to clear the color buffer, the depth buffer, and the stencil buffer before rendering the primary scene.

Once the primary scene is rendered (including the apertures, which have now rendered the stencil buffer), we can render the individual portals.

The main rendering loop looks like this:

this.renderer.render(this.scene, this.camera);
this.renderer.getScissor(this.saveScissor);
this.renderer.getViewport(this.saveViewport);
this.renderer.setScissorTest(true);
this.saveClipPlane.copy(this.renderer.clippingPlanes[0]);
portals.forEach(portal => {
  portal.render(this.renderer);
});
this.renderer.setScissorTest(false);
this.renderer.setScissor(this.saveScissor);
this.renderer.setViewport(this.saveViewport);
this.renderer.clippingPlanes[0].copy(this.saveClipPlane);

And the render method of the portal looks like this:

renderer.clippingPlanes[0].copy(this.clippingPlane);
renderer.autoClearStencil = false;
renderer.setScissor(this.portalViewport);
renderer.setViewport(this.portalViewport);
this.mesh.visible = false;
const gl = renderer.getContext();
renderer.autoClearColor = false;
gl.enable(gl.STENCIL_TEST);
gl.stencilFunc(gl.EQUAL, this.stencilIndex, 0xff);
gl.stencilMask(0);
gl.stencilOp(gl.KEEP, gl.KEEP, gl.KEEP);
renderer.render(this.destinationScene, this.portalCamera);
gl.disable(gl.STENCIL_TEST);
this.mesh.visible = this.isOnscreen;

Note once again we’re doing calls directly to GL to set the stencil buffer params. The reason is because three.js doesn’t provide a way to globally set the stencil params for all materials; fortunately three.js doesn’t touch the stencil state unless there is a material that has stencilWrite set to true, which none of my materials do.

Odds and ends

There are a few extra things you can do. The code shown above will render double-sided portals: that is, both the front and the back of the aperture quad will be rendered as a portal. You may not want this, in which case you can test the normal of the quad to see if it’s facing the camera, and skip rendering if it is not.

That is pretty much it. Good luck with your portals!

(Note: this article was originally published on medium.com .)

See also

ThreeJS3dGame Development