On the Perceived Inevitability of Unreal
Well, shit!
Overview
The topic of Unreal's impact on AAA game development is a big one with a lot of avenues to explore. I won't claim to have the knowledge or expertise to explore many of them, these are just the thoughts and ramblings of one graphics programmer who has spent his career almost entirely on in-house engines. It's interesting to talk about though, and I suspect I'll just be regurgitating the thoughts of people in similar situations, but who knows, maybe someone will find this interesting. This post is somewhat of a reaction to a number of posts I've seen in recent months that say something to the tune of "You'd better start working with Unreal, because one day you'll all be working in it." I have a hard time disagreeing with that in spirit to be honest, having witnessed the decline of in-house engine development over the last decade, but I also have reasons to believe in-house tech is here to stay (there's more than you might think!), and may even see a resurgence in the future. It will be interesting to look back on this post in 5-10 years and compare it with what's happened in the meantime, at any rate.Disclaimers
In the instances where I discuss Unreal below, I want it to be clear that this isn't a post shitting on that engine, or an attempt to discourage people from using it. Unreal is an incredible and well-proven suite of technology built by many of the industry's most brilliant developers. Unreal also delivers this technology to a massive audience across tons of platforms, for multiple industries, all while keeping up with the game industry's latest and greatest and often blazing new trails for the rest of us (and they readily share that knowledge!). To boot, they do this with (generally speaking) great art and design workflows, tools, and UX, to the point where you can prototype a new game in a matter of days or weeks. This is all to say that the long shadows cast by Unreal are well justified, make no mistake.The Shrinking Pool of Game Engine & Graphics Talent
For a myriad of reasons, the game industry has become somewhat starved for engine and graphics programming talent. Some of this is due to the nature of our industry (at least for many companies), for example our notorious rate of crunch stacked with lower-on-average pay tends to cause many engineers to exit games and join adjacent industries as they reach the point in their lives where more guaranteed financial stability and life balance become increasingly important. Some of this is due to Unreal, Unity, and a couple others soaking up disproportionately high numbers of these engineers, borderline starving out many companies of the staff necessary to service their in-house engines. Hell, there are feature-focused graphics sub-teams at Unreal that have been bigger than any graphics team I've ever worked with. All this combined means it's extremely difficult to attract and hire experienced graphics and engine programmers. What we are commonly left with (and I very much advocate for this) is hiring brand new engineers right out of school or another industry, and training them, though this proves more difficult with a lack of experienced programmers to teach them while also developing in-house engines. It's also made more difficult with schools increasingly generating programmers that are only experienced with Unreal and Unity, and not with lower level foundations that are necessary for custom engine development. All of this makes it difficult to justify keeping in-house engines when you struggle to even staff it enough to service your studio.There are some other options though, depending on what's feasible for a given studio. There are quite a few decent sized contract and support studios out there with the talent (or at least the international infrastructure and reach for the talent) required to support in-house engines in cooperation with their studios. To name a few: The Multiplayer Guys, Nemesys, Triple Boris, Iron Galaxy, Lost Boys Interactive, and many more exist that are quite capable to support in-house engine development, though it's likely you'd need to hire teams from a few of them to get everyone you would need. These companies also tend to soak up a lot of additional engine/graphics folks from the rest of the industry because they offer attractive contract/remote work support that game studios typically don't. Nowadays I'm not convinced a company could staff up significant engine development without support studios unless they had a metric ton of money and time, and at that point Unreal is likely seen as an easier route to navigate.
The Death of Independent Game Middleware
Especially in recent years, the amount of game and game-adjacent middleware being bought up by Unreal, Unity, and others is terrifying for any in-house engine development team. High quality middleware is a big part of how the rest of us have been able to keep up, or at least stay within the ballpark of off the shelf engines. Some of these technologies have remained available to license after these purchases, but it is still seen as a liability, and you know going in that their development will still be naturally skewed towards the engine/company that purchased them. It's hard to blame middleware companies for selling - publishers have a long history of devaluing middleware during licensing negotiation, and that's after them having to endlessly pitch their products to development teams for adoption. It's a smart move for them to sell, and this wound to studios with in house engines is (in my opinion) mostly self inflicted.Framework & Tooling Infrastructure
This is pretty much what it comes down to in a lot of cases, as far as I can tell. The sheer amount of resources and effort required to build up an engine framework (not the engine itself - the core, infra and tooling to support one!) is monumental. When you weigh that against something like Unreal that comes with most of the necessary ecosystem built-in right from the start, it's hard to turn away. It takes years to build this stuff up from nothing, before an artist or designer can even start working, and that's a lot of pressure for a studio to withstand on the promise that a specialized set of technology will be better for the studio in the long-term. It certainly can be too - the fact that Unreal has to cater to such a wide audience means it's often not good at doing studio-specialized work without significant effort and long-term maintenance load on a game team trying to modify it, but this is difficult to communicate to a team that just wants to make a game. Unreal (and others) have a huge step up here - a decade+ of hundreds of developers pouring effort into tooling, pipelines, UX, etc. Even after you've built your own, it will likely never match their quality level. The best case is it is really good at exactly what you need it to do, but even then a team will spend the rest of their careers justifying that against Unreal. That goes for pretty much every other aspect of the engine as well, honestly.And this, combined with the above (and more) is where many in house engines die, and in most cases it's probably the right call. The amount of effort required to build infrastructure for a new engine keeps growing as time goes on, and so more studios switch to Unreal, and very few make new ones. The pace would suggest that in AAA, we'll all eventually use Unreal, but there are a number of engines that have held steady, and with potentially new paradigms around the corner...
I'm speaking, of course, of technology like The Machinery. I think tech like this is how the pendulum swings back towards in-house engine development. You start with a strong foundation, most of the basic tooling and infrastructure that you need, and then plug in the parts that you want to custom-build yourself. I can't claim to know that The Machinery itself would work out like this in practice, but it's more along the lines of what I would expect to be adopted going forward. I have suspicions there are other companies working on similar technology, so maybe we have a bright future ahead for new custom engines, who knows.