Hello there! I have been involved in developing a Civilization 5 modification for a while now, and today I would like to discuss the game state, promote some popular projects, and highlight my own contributions in this field at times.

Brief introduction

Civilization 5 is a turn-based strategy game that was released in 2010. In the game, players take on the role of a leader of a civilization and guide it through various eras, from ancient times to the modern age. I think everyone who reads this article already knows what this game is.

The very last official patch for V was dated October 27, 2014. Since then, Civilization VI has become the main focus of franchise development, and there are even rumors about VII. However, there are still many people playing Civilization V. Of course, the online is much lower than VI, but it is still quite high as for a game that has not received official patches for almost ten years.

Modders community

One of the main reasons why it is still popular is the great and wide modding community. There are about 10k+ mods available in the Steam Workshop and around 500k+ messages in the modding subforum at civfanatics.com. Mods make this game feel fresh and exciting to play.

As of now, almost everything can be changed. The core library, which contains all the logic, is open-sourced (only the Windows DLL version, but at least that). Everything else can be changed by editing .lua, .xml, .sql, and other files.

Vox Populi (also known as VP)

As I mentioned, there are many mods available, but currently the “central” mod is Vox Populi (civfanatics subforum, discord channel). It brings together numerous developers who are focused on the single mod with the single course. The repository for this mod’s code currently has around 7k commits and 250 stars, which is quite impressive as for a whatever game mod.

If you are looking for a new Civilization 5 experience, this mod is a good place to start. It brings a lot of new features, gameplay mechanics, balancing, and more, while still maintaining the “style” of the game: it is still the “old” good Civilization 5. Some other notable mods to mention (there are actually many more good mods available, but I’m just mentioning a few): 4UC (adds more unique components to each civilization), More Wonders (adds more wonders, surprisingly), Even More Resources (adds more resources).

If you are going to try, be sure to follow instructions on how to play Civilization 5 + VP and guide about mod installation in general. One thing to note is that it is not recommended to install mods via Steam Workshop as they often contain outdated versions. Instead, use .zip archives from forum and GitHub.

What and how am I doing there

I have been playing Civilization 5 occasionally since its release, but only in multiplayer sessions. Some years ago, we decided to try modded multiplayer (with plain VP, around 2018) and it ran quite well. However, when we attempted to play again in 2023, multiplayer VP was literally unplayable. Since then, I have been contributing to VP to make it compatible with multiplayer.

Multiplayer compatibility

Until recently, there was no understanding of how to make mods completely multiplayer-compatible. After some research, I discovered that there are actually no major issues with it; just some basic rules must be followed. Understanding how multiplayer works exactly has helped in defining these rules.

Imagine we started session with several humans and several AIs. Multiplayer session works like so:

  • Each human player has their own game state (unit positions, founded cities, assigned citizens, and so on). There are a lot of in-game variables that are part of the game state.
  • Each human player calculates its own state by themselves after and during each turn. This includes AI processing logic, fight results, path routing, and so on. Every calculation is done at client side, and there is no server that runs any logic for clients.
    • For example, the game’s core logic must work identically on each end when processing how AI behaves at the beginning of a turn. If there is a flaw in the logic and the AI moves a unit to position (1;0) at player A’s end, but at player B’s end the AI has moved the same unit to (2;0), then a desync occurs. Such flaws in logic are the main reason why mods usually aren’t multiplayer-compatible.
  • Human actions (such as “move unit”, “delete unit”, “make road”, “build something in city”) are broadcasted to other humans via network messages.
  • Each human handles such network messages and modifies its own state.

  • After processing each network message, each human must have an identical game state. Otherwise, the game will continue to run in a “desynced” state, which can lead to crashes and undefined behaviors.
  • Under certain circumstances, the host can decide to “re-sync” everyone, so that every player will have the same state as the host. However, if there is some multiplayer-incompatible logic in a mod installed, it still doesn’t guarantee further stability.

Make modification multiplayer-compatible means code it so that:

  • Each human will calculate its’ own game state identically.
  • Any human actions are network-broadcasted to other players.

Example of code which causes multiplayer incompatibility

For a long time, VP was coded without multiplayer in mind. As a result, it worked perfectly in singleplayer mode, but multiplayer didn’t work at all. The reason for such incompatibility was the logic that resulted in a different game state on each end. Notable examples of such logic:

  • #9768: sort usage, which doesn’t guarantee consistent order for equal objects in C++.
  • #10112: using set with undefined sorting order.
  • #9867: using pointers as map keys.
  • #10250: using the same cache for both UI and core logic processing.
  • #9767, #9970: using player ID during calculations.

More examples along with all my contributions done are here. From time to time some exotic cases encounter, but the root cause for incompatibility is always the same: some code results in different game state on each end.

Reverse-engineering

Firaxis has open-sourced the Windows DLL that contains the game’s core logic, including AI processing, path algorithms, diplomacy and so on. While many things can be changed in this DLL, there is still a significant amount of code that cannot be modified. The code responsible for rendering, UI, camera interaction, network bytes transfer, and Steam integration are located directly inside the executable file and hasn’t been open-sourced. Although this limits some modding possibilities, it is still commendable that at least a portion has been made available to the community.

So, while we are able to build only some part of the game, the following problems appear:

  • The code is locked on Visual Studio 2008 and we are unable to use many modern C++ features.
  • All the compiled libraries and executable are 32-bit, which results in limited memory consumption.
  • Inability to fix closed-source bugs and memory leaks.
  • Inability to mod any behavior inside executable (rendering, network and so on).

Moreover, executables are CEG-protected (deprecated, but still working Steam DRM protection), which makes it impossible to patch them and distribute patched versions along with modification content.

However, there are some good news too:

  • Although executable is CEG-protected, we still can patch it in runtime. It is easy to implement because we already managing DLL code which injects there.
  • Windows executable is not obfuscated with some advanced techniques and it is not so hard to make patches there.
  • Finally, I’m unsure whether it was intended or not, but Linux executable ships with preserved namings and virtual tables. By investigating it, it is easy to match its’ named functions with the same inside stripped Windows executable.

So, it means we actually can change behaviors inside executable and, furthermore, it is not so hard to do.

As a result, a simple PoC that shows it is possible has been written. Here, we are modifying an underlying variable inside the executable that indicates whether or not a force re-sync was scheduled. By tying everything together with some buttons, we have a brand-new feature that works by interacting with closed-source executable code. Such method was not previously used in Civilization 5 modding (however, interaction with the executable was also previously implemented in a solid project MPPatch), and now it opens up a wide range of new possibilities.

Conclusion

So, nothing else to say here, just promoting the project and my participation. If you were a Civilization 5 player somewhere in the past, give it a try with mods, it definitely worths it. Thank you for reading this little article, I hope it will be useful to someone. Feel free to reach me if you have something to say. And also take a look on the posts on similar topics: