Ray-Tracing, Beyond Realism

Beckett
12 min readOct 27, 2020

It seemed odd to me, at first, that the most interesting use of ray-tracing I’d seen appeared in a videogame from twenty-three years ago.

Put simply — ray-tracing simulates the physical behavior of light by shooting rays out from the camera and bouncing them around the environment. Traditional videogame graphics have developed all sorts of tricks and hacks to imitate many of the same behaviors, but subtle differences — that someone like Digital Foundry would do a better job of explaining than I — still register consciously or subconsciously in our brains. Primitive reflections in certain surfaces, ambient global light that seems to come from nowhere — these failings of rasterization will become more apparent in time, as the triple-A crunch death march towards the ever-retreating target of realism continues. Ray-tracing threatens to solve part of the problem with not just better lighting, but real lighting. In theory. Even into the second generation of dedicated hardware, ray-tracing still doesn’t quite completely work as a stand-alone solution for most games. Unless the game is from 1997.

Many dismissed Quake 2 RTX as a novelty, or tech demo; a less than ideal way to play the game that drastically alters the original artstyle for the sake of shiny surfaces. My launch-window 2070 card didn’t earn me a playable experience in 2019, but after some patches, resolution tweaking rendered it playable. What I found surprised me: re-contextualizing the geometry, animations, and art direction from a quarter-century ago into a real physical space feels surreal. The hard edged, clean geometry, born of infinitesimal polygon budgets, looks too perfect when it casts and receives real shadows. The materials added to the levels create the appearance not of a hard sci-fi military industrial complex, but of a Disneyworld roller-coaster anteroom, where some television or employee gives in-character backstory to the experience, playing to a room full of parents and children boxed in by clearly fake plaster-cast rocks. It evokes the set of a low-budget science fiction serial from years past, abandoned after hours. Floating powerups, rotating slowly, fixed in space, wouldn’t cause a single player of the original Quake 2 to bat an eye, but here, they suddenly become eerie, their perfect motion suddenly unnatural.

Textures that, in the original, presented as exposed circuitry now read more like abstract glitch art. All of a sudden, the alien nature of the world around you becomes clear, its constructed-ness more apparent. You ask questions you never thought to before: why does this cart float so perfectly above this rail? Why space these ammo boxes out like this? Moving platforms rise and fall on supports that appear to emerge seamlessly out of the ground around them — when the ground itself appears as a sheet of metal, instead of a blurry videogame texture, it feels wrong, too exact, like a glitch in the Matrix.

These inconsistencies suddenly draw attention to themselves. When light obeys the laws of physics, we expect the same laws to apply to the rest of the world, but here, they clearly don’t. Lights become convincingly fluorescent, imitating the incandescence and tone of light sources from our lived experience. Unlit corners that would otherwise be cast in a general global illumination stay as dark as they actually would be, in a way that implies whoever made this space didn’t make it for humans.

The creatures who do occupy these spaces don’t seem human, or even humanoid. Instead, their warped, horrific polygonal forms, once justifiable by graphical limitations, jerk, shudder, and fall unnaturally. Sure, the artists intended these creatures to appear warped and horrific — the Strogg perform experiments on themselves and captured soldiers, creating body-horror monstrosities that you put down in droves — but you don’t pick up on that specific horror as much when playing through Quake 2 RTX. The cyborg combatants, especially with their rudimentary AI, present less as themselves and more like weird, primitive constructs. When their forms lay still on the ground, their bodies still seem strange — casting soft shadows onto the ground around them, taking up physical space with their impossible being, an unbroken blob of exotic shining material. The practically primordial gibbing is especially surreal — their forms instantaneously disappear without effect, replaced with misshapen chunks of matter, lit realistically as if such a thing were perfectly normal. Flying drones silhouette themselves with their own laser blasts — what initially seem like stiff video-gamey flying boxes become real by the light they create.

The answer to most of the questions I’ve asked in the previous paragraphs might, for some, seem screamingly obvious: it’s a videogame. These things in the game happen the way they do because iD needed to program them that way with the resources they had at the time.

Nobody asked these questions in 1997, when Quake 2 dominated computer stores as the marquee demo of the day. “Thing In Videogame Doesn’t Make Sense In Real World” is the most tired webcomic trope of all time. But that misses the point: internal consistency is extremely important for how we perceive fiction, and Quake 2 RTX’s visuals so blatantly and consistently flout its own internal logic that it loops back around to feeling intentional, creating a unique mood of its own. Think about it this way: in a cartoon, style reigns, shapes and colors stay simple enough to reliably draw frame-by-frame. You don’t bat an eye when characters squash and stretch, or make distorted facial expressions. But if you saw the same cartoon character, say, going to the store, you’d probably lose your god darned mind.

The story I began to construct in my head as I played didn’t feel like a lone soldier taking the fight to an oppressive alien force, destroying the pieces of the machine one by one. Instead, I imagined that I, a human being in the real world, had suddenly stumbled onto this weird, hostile, unnatural space, constructed for unknowable reasons. I believed the lighting, and that the material composing these walls appeared solid, but their perfect composition unnerved me, the inhuman behavior of the creatures within repulsed me. Hefting increasingly surreal weapons, I fought deeper and deeper, in search of answers.

I had a playwriting professor in college who told us early on that, in good writing, the “strange becomes familiar, and the familiar becomes strange”. This paraphrase of the German romantic poet Novalis forms the mission statement of many artists and even more cultural anthropologists. To gain real insight about the world, we have to triangulate it — we can’t simply look from one perspective, or within one cultural context. When we examine the familiar with a different lens, we inevitably find valuable new information. When Quake 2 turns ray-tracing on, it turns the familiar — the classic design of a late 90’s sci-fi shooter — into something strange, inspiring new possibilities for storytelling, design, and art direction. Taking artifacts of technological limitation and placing them into real physical space defies suspension of disbelief, confronting players with their artificial nature. Strong, sharp geometry feels almost like a sculpture garden, and the longer I stared, I gained more of an appreciation for the intentional, deliberate, artistic rationing of polygons.

Natural ray-traced light can also make strange settings feel more familiar. I loaded up Minecraft’s ray-tracing beta looking for a similar kind of alienation, expecting the voxel geometry to seem stilted and odd under a more natural sun. Instead, the opposite happened.

The forests became more forest-like. Light plays through leaves, and gentle diffuse bounces absorb the green of the world around you. Underwater environments blew me away: coral structures shadowed each other in a gorgeous dance of color, and mesmerizing beams of sunlight pierced below the surface. Firelight from torches and stoves gave newfound warmth, and shaded the space around them with convincing stark shadows. These same shadows made spelunking more deliberate, the way the light creates perceived safe space in a cave totally reworked to make darkness more of a palpable constant: convincing shadows fell across even the geometry of the safe areas to highlight the harsh, claustrophobic terrain around. Lava shines brilliantly with an aura of both awe and menace, even from a dramatic distance or out of direct line of sight. Many of the mechanics of Minecraft’s survival mode already revolve around light and the safety it represents, so the implementation even makes sense thematically.

Watching the sun rise or set over the wild mountains of a randomly generated world, as rays of believable light stream over and through cavernous, impossible spaces, turns these odd artifacts of procedural generation into fascinating, real geography.

Out of curiosity, I loaded up a custom map: the cerebral, precise survival challenge of SkyGrid places a random block every five or so spaces and leaves the rest open air. At night, the entire world becomes a psychedelic carnival of light which shifts the character of the constant existential threat the map represents. Once you’ve established some kind of base, carving out your own safe area in this incredible, inhospitable, beautiful space looks unlike anything I’ve seen in a videogame.

It’s worth noting that the nature of the experience could change significantly based on the resource packs a player chooses: I think a large part of why these strange voxel-worlds become so familiar with ray-tracing is the consistent art style. The current version of Minecraft’s official ray-tracing doesn’t have any official materials, so everything looked sort of like plastic — but with a simple vanilla-friendly resource pack, the art held up. Chunky pixels fit the blocky terrain, but if I applied some high-resolution, realistic textures, it might feel different.

As far as I know, Quake 2 and Minecraft remain the only older games with major official ray-tracing support. I bring them both up because, with both next-generation consoles coming standard with their own ray-tracing hardware, we’ve only seen the tip of the iceberg, and I want to talk about how the technology can keep some creative integrity. It’s easy to look at what’s already out there and dismiss it as a gimmick, or a hasty addition for the sake of marketing. I don’t mean to say that none of the AAA ray-tracing games justify theirs: CONTROL’s aesthetic is meaningfully, if subtly, enhanced. Objects behaving strangely reflect correctly in the environment, their shadows falling where they ought to — the familiar becoming strange is thematically in line with CONTROL already, and when the surreal geometry sells itself in a real space, it doesn’t just give Digital Foundry something to zoom in on, but it helps in part to ground the world in thematically consistent ways. My Quake 2 headcanon is almost literally the plot of CONTROL. It’s become cliche to bring these two games up in the same paragraph, but something like Naissance would benefit in similar ways from this technology.

So what else would prove an interesting use of ray-tracing? Any 3D artists watching would already know that ray-tracing has actually been around for decades. Modern advances have only made it possible to perform reliably in real time. Anyone can create a ray-traced image on their computer — just download Blender. Consumer-level 3D modeling software with ray-traced lighting existed since the early 90s. But of course, it takes minutes, or even hours, to render these images. The amount of compute power put towards rendering CGI movies in a matter of days or even weeks is astronomical. So in 1993, when The 7th Guest and MYST both released on computers with CD-ROMs full of pre-rendered 3D art, people’s brains exploded. In the decades since, the adventure genre has moved elsewhere — but the specific look of these old adventure game stills has never been exactly replicated. Sure, realMyst exists, and you can walk around that island in real time, but it doesn’t look the same. Even in ’93, the lighting in these “primitive” 3D renders doesn’t get reproduced in modern real-time graphics. But now that we can, I think revisiting this aesthetic could create new, dreamlike experiences.

There’s a fanbase for this specific era of early 3D art, which I count myself a part of. In Brian Eno’s published 1996 diary, A Year With Swollen Appendices, he explains why:

“Whatever you now find weird, ugly, uncomfortable and nasty about a new medium will surely become its signature. CD distortion, the jitteriness of digital video, the crap sound of 8-bit — all of these will be cherished and emulated as soon as they can be avoided. It’s the sound of failure: so much modern art is the sound of things going out of control, of a medium pushing to its limits and breaking apart … The excitement of grainy film, of bleached-out black and white, is the excitement of witnessing events too momentous for the medium assigned to record them.”

Mid-nineties PS1 nostalgia has already come back around, encouraging developers to make experiences that channel the feeling of playing those old games without adhering to the more boring restrictions like memory space and processor speed. In the spirit of the Playstation aesthetic, I hope to see this kind of early CGI replicated with ray-tracing — smooth, detailed, realistic lighting on low-res objects. Sure, it would seem more realistic to prioritize natural geometry, but the atmosphere of these adventure games remains significantly more interesting to me. The simplicity and clarity of the art generate a kind of magical realism — or even romanticism.

Our friend from earlier, Novalis, was a capital-R Romantic, and a strong influence on the artistic movement of the early 19th century. Here’s the full quote of his that I paraphrased earlier:

“To romanticize the world is to make us aware of the magic, mystery, and wonder of the world; it is to educate the senses to see the ordinary as extraordinary, the familiar as strange, the mundane as sacred, the finite as infinite.”

Now, that’s a bit much — but that’s kind of what Romanticism was about. I’ll keep the art history lesson short: Romanticism was a reaction to the restrained reality of Neoclassical art and architecture, which valued structure and harmony, and idealized ancient Greco-Roman aesthetics and philosophy. Romanticism instead values color, movement, and sentiment over form, clarity, and realism. Romanticism, as Novalis might tell it, is to find the beauty in the mundane world around you, to look at a glass of water on a table and feel awe at the patterns the light plays on the surface. It is to watch the sunset, mesmerized, until the dusk dies. It’s when you’re walking to or from somewhere, and you stop suddenly, realizing the beauty of the world you see before you, something about the exact composition of your specific viewpoint inexplicably beautiful, but when you want to communicate exactly how or why to someone, you come up short. It’s like wearing rose-tinted glasses all the time.

Ragging on videogame realism, over the years, became a bit of a tired point. These decisions are made based on factors well beyond the subjective taste of a few loud critics. I’m not here to pretend to be an authority on anything — instead, I hope to inspire people to think about other possibilities. We’ll find ourselves quite soon with the ability to generate unprecedented real-time lighting in videogames. It’s possible some developers will default to using this power without thinking about what it adds beyond nice reflections. And it doesn’t have to — but if you’re going to put in all the technical effort to make something light realistically — I think there’s something to be learned from the landscapes of these Romantic painters, stylized, exaggerated landscapes awash in convincing sunlight. Look at CG from years past, look at Quake 2, look at Minecraft, and imagine what games could look like with a different mindset.

Imagine what we can do with light.

Thank you so much for reading.
Available in video essay form here:
https://youtu.be/ENEne4XFc44

--

--

Beckett

Writer and designer of virtual worlds, games, and interfaces. (Also a professional voice actor!)