Dissecting a Cogmind Changelog

Cogmind has had 15 major releases in the past 21 months, and the changelogs for especially the latest four have each been massive walls of text. While it’s fresh in my mind, and before we move on to a different system of smaller, quicker patch-style releases, I thought it would be interesting to share an inside look at how even an impressive changelog isn’t all that representative of the work involved, or even the true content of a given release.

I’ve talked about the bigger picture with regard to release cycles before, but here I’ll be going into a lot more detail focusing on a single changelog itself, specifically that of Alpha 14, the most recent update.

Alpha 14’s changelog as a scaled down image just to give a general idea of its length and composition.

I always split the changelog into three sections, NEW, MOD, and FIX, something I’ve done for about six years now, since the early X@COM releases. I got the idea from one of the libtcod devs back then, and liked its simplicity and the fact that it conveniently relies on only three major categories to cover every possibility, distinguishing explicitly what’s new vs. what’s changed, which I think is more important than categorizing by other characteristics such as “UI-related,” “gameplay-related” etc.

The largest single section tends to be new features (here 42%), followed by changes (37%), and a smaller number of fixes (21%), a distribution more or less common across all releases.

Cogmind gets put through a good amount of both manual and automated testing, so there aren’t usually many bugs to deal with, generally just a couple larger things with each release, and a handful of minor rare issues with no significant effect. Most are either UI-related or emergent issues that naturally sneak into such a large code base. By contrast, there is a fair amount of tweaking both to mechanics balance and QoL. Then of course new features should make up the bulk of a release, as that’s what everyone looks forward to the most!

This same ratio will likely continue for the entirety of Cogmind development, including post-1.0, because I’ve been operating from a “complete game” foundation since Alpha 1, so there won’t be much of an incomplete game vs. complete game shift going on (hypothetical public changelogs of Cogmind’s two years of pre-alpha versions, on the other hand, which correspond to what a lot of other games consider “alpha,” would easily be 95% “NEW” line items).

As for the changelog presentation itself, you’ll notice that I color some items. These are the important ones, basically what I think anyone who only wants to skim part of the list should prioritize--the features, changes, and fixes players would most likely want to know about.

Within each category, their order is honestly pretty random, though if there are a few related entries they’ll be grouped together, and I always stick the handful of biggest NEW features at the top. The fact that there’s a new map, the numbers of any new robots and/or items and major features (e.g. difficulty modes) need to be the most visible. (In this particular case melee multi-wielding also deserved to be at the top, but both it and difficulty settings have multiple associated entries, so one had to win out, and the former is a much bigger deal, after all.)

For the purposes of this post, I’ve rearranged the changelog in the order I actually implemented everything. They’re also colored by category, so we can examine how that aspect relates to the process (green = NEW, yellow = MOD, red = FIX).

Alpha 14 changelog, ordered and color-coded. (click for full size)

At the top there you can see a few fixes, generally made shortly after an update when a handful of new issues are discovered by the initial surge of players. These aren’t for a patch (in which case they wouldn’t even appear here!), but come at a time when I’m mostly just interacting with players--and playing myself! So I’m less busy and it’s convenient to take care of these right away.

Otherwise, most of the fixes don’t come until a phase after all the major features are complete. Before that point I’ll just collect them in a list, although if it’s not immediately clear what caused a reported bug, or I don’t think I have enough information to solve it (I usually do), I’ll tackle it right away in case whoever found it might be able to help by providing additional relevant information. (Another exception would be very simple bugs reported when I happen to not be too busy, in which case I’ll just take care of it immediately.)

Why prefer to do tasks in groups? Efficiency. It’s quicker to handle tasks of a similar nature together, both in terms of tools needed and mental state, and I don’t like to waste time :). On smaller projects wasted time doesn’t really have a chance to add up, but losing a little time here and there on a project spanning years can in the end add up to weeks and months!

Note the “issue resolution” section is named such as it includes MODs, not fixes per se, but usually balance tweaks or other changes often in response to player feedback, sometimes direct and explicit, and at others just based on my observations of player experiences.

At the end is a flurry of smaller features, feature requests, QoL and the like. AKA the “OMG I must release ASAP--WTF it’s been a month since last alpha!” period. It’s nice to have some semblance of milestone deadlines, even on an epic project with no real oversight :P

Now let’s take a look at how the changelog grew over time. Surprise!

Ordered Alpha 14 changelog, subdivided by development week. (click for full size)

Alpha 13 went out on January 17th, but I didn’t actually start on Alpha 14 until the 26th. It’s pretty normal to spend at least a week of time doing followup work from the previous alpha--compiling stats, watching how players interact with the new features, maybe writing a blog post or two, and certainly taking care of all the personal chores I no doubt neglected in the run up to a release :P. So technically this alpha was completed over a period of four weeks (half of which were spent out of country).

Look at that… two-thirds of the changelog was done in the final week?! One week!

Despite appearances, it was most certainly not two-thirds of the work. There are three major factors to consider here:

• Some of these individual entries have a lot of work behind them, not represented here. (This compared to some that took mere minutes.) Especially during the first couple weeks I’ll work on bigger features, which as described before is because the required time for these can be more difficult to gauge, and therefore they have priority over the many tiny features that can be rapidly packed in one after another at the end. In particular you can see “NEW: Branch map ‘Deep Caves'” sitting there by its lonesome in week 2, next to a handful of other related entries. That was a ton of work--a whole new map that includes new mechanics, new robots, new events, new parts… but none of the details are broken down there. I don’t want to ruin the chance for players to make new discoveries, so while the public changelog for Alpha 14 is 126 lines, the internal version contains 207, part of that attributed to spoilers. (Of course the other part is simply elements that don’t concern the player.)
• Some weeks I am actually busy with other non-changelog-related parts of development, like writing updates or articles, or for example during Week 3 when I had to take about a day and half out to work on and push the Greenlight submission. That particular week, as a result of all the other writing I was doing there was only a day and a half to work with the source code, much of which was allocated to the semi-major feature “High Security.”
• Some weeks I simply devote more time to development. This is also more likely closer and closer to release, where I really hope to fit in a certain chunk of features (the “final cut” list is always made at the beginning of the fourth week, and I usually get through about 75% of it). I always aim for about 45 hours per week, though in this case Week 4 was somewhat extreme, as I ended up putting in 70 hours. So that had something to do with the lopsided changelog, too :P

Interestingly, despite the underrepresentation of major features I do believe that overall the final length of the changelog does kind of balance out with the amount of work that went into it.

I hope you enjoyed that--I certainly found it enlightening. I mean, I more or less knew this stuff, but it’s nice to punch in the numbers and and turn it all into pretty diagrams to visualize it :D