DEVLOG 2023-06-03

This week led to big progress in the prototypes (still very low-level techinque stuffs, sorry!).

After build2 devs added a missing feature that was preventing me from correctly running the python scripts from godot-cpp that generates the C++ bindings (the build2 devs were actually hitting similar issues and I was lucky to reach it while they were working on a solution), I managed to complete a working version of the build2 package of godot-cpp. As a reminder, godot-cpp is the library you need to use that provides the C++ api for GDExtensions which are non-GDScript plugins for Godot. Current prototype (number2) is about checking that working with the model of the game in C++ using build2 as buildsystem and package manager but having the “view” of the game implemented using Godot is viable, work-process-wise. (see this for more details)

Now using build2 and my godot-cpp package, it’s quite easy to start any new GDExtension. There is still some details (like documentation and some helpful tooling) to fix or complete before I push it into the official open-source build2 package repository but it’s usable today using it directly from build2-godot-cpp my git repository too (build2 allows to get packages both from github repositories and “archive” repositories like https://cppget.org, which is helpful when you have packages that are experimental and don’t want to push in the “default” repository).

So I proceeded to test my godot-cpp package by recreating the gdextension example from the Godot documentation. I also already had the GDExtension test used in Godot’s CI ready (which is a GDExtension + a Godot project using it) so I tried to open that test in Godot and my hello world GDExtension too in a custom test Godot project. Both worked as expected which was exciting! But more than that, I realized various new informations, which is the purpose of the prototyping process.

First, I realized that I did not need to have the GDExtensions built with any specific flags. Godot seems to only use the C api which have a stable ABI on all the platforms including Windows, as far as I understand. I still expected some issues with Debug vs Release builds because on Windows they dont use the same C runtimes, but actually as long as I dont use the C runtime I guess I’m good? Or maybe this is already isolated enough that ABI and runtime issues are taken cared of? Anyway so far, building a GDExtension with no flags or with some flags that dont match what Godot itself is built with doesnt seem to cause any issue: Godot loads the GDExtension and makes it work as expected, no problem. I still need to setup the Godot configuration for users that really want to build with the same flags though, but now this seems secondary and more like a bonus, I can go forward without having to do it, so I’ll do it sometime after I’m done with prototype 2.

Second, I started modifying my custom hello world GDExtension and using a handy build2 install command, I setup everything so that I can quickly modify the C++ and install the binaries in the right place in the directory of my Godot project. While doing so, I found that Godot was not hot-reloading the extension automatically. See, when you modify a script of an asset outside of Godot and then get the focus back on your Godot window, Godot will detect the modification and reload the modified assets/script/whatever automatically, which is crucial for developing fast. However, it does not do this for GDExtensions. Yet.

I found an issue thread about this which have some recent comments, but I suspect most of the people in there never implemented a hot-reloading system (this kind of thing exists since like 20 years but are rarely generalized as libraries). I have some experience in implementing hot-reloading systems, but no experience with tweaking Godot and I dont think I have enough free time to help them with that, unfortunately. The good thing is that a lot of people are interested in the subject because they say GDNative (the previous system) and GDExtension is simply unusable for projects that use C++ for most scripts. So there will probably be some solution implemented soon. You see the issue is that you need to reload the whole engine/project to get the modified GDEXtension library loaded. Also on Windows there might be file locking issues even preventing you from doing the install of it properly. Reloading a whole project in Godot is far faster than Unity, but the bigger the project the longer it takes. So it can be a massive pain if you tweak something in C++ and then have to wait several tens of seconds to see if it does what you want. In particular because you have to build using a C++ buildsystem, then move the binary in teh right place then reload the project in Godot then run the game then reach what you need to see.

Anyway this is a notable issue that I will have to track. It is not clear yet to me if that kills the working process for me yet, because I dont intend to have “all” the view code in C++… or actually I dont know yet because I didnt decide how to organize my game-specific GDExtension. This is the subject for next week’s work!

I also setup a Godot project (proto2/) in the prototypes repository in addition to our GDExtension for prototype 2 (proto2-view/) and I cloned the model library from proto1-model/ into proto2-model/ so that we can modify the model without having to maintain proto1-view/. I also setup a way to easilly install proto2-view.dll/so, our GDExtension, into proto2, the Godot project, with the right files so that Godot loads it. For now proto2-view only contains the hello-world example so that I could make sure everything works when loading it in Godot.

Next steps are:

• decide what exactly to expose to Godot through the GDExtension? I could have just the model’s world exposed and then script the rest with GDScripts, or I could have a maximum of code in C++ and just the visual interpretation in GDScripts, or even no GDSCripts, do it all in C++. This last option might help us test the limitations of GDExtension.
• implement a basic view of the state of the model, in 3D: basically we’ll have cylinders for characters, cubes for walls and all this placed on a plan, very simple. Then when we press a key we need to update the model and have the 3D things move accordingly.