How to find which Salt release will contain a fix you need
6 minute read Updated:
UPDATED ON 2019.10.23: SaltStack has adopted a new release strategy and this article is no longer relevant.
Salt is buggy. There are many threads (1, 2, 3, 4, …) about that. If you dig deep enough into any Salt feature, you are almost guaranteed to find bugs. This results in de-facto issue bankruptcy, plus it looks like Salt open source team is severely understaffed, and many talented folks recently left the company (Daniel Wallace, Nicole Thomas, Mike Place, Brett Benassi, Erik Johnson).
If you dig deep enough into any Salt feature, you are almost guaranteed to find bugs. How to deal with that?
Salt core tends to have fewer bugs than other modules contributed by the community. The main reason for that is the very liberal merging policy: right now there are ~2200 open issues, and just ~90 open pull requests. Salt team does code reviews and usually asks a submitter to add some test coverage, but obviously can’t test every feature against a real (and often highly specialized) system. So, because the original submitter may be no longer active, these bugs tend to stay open unless someone from the community steps in.
If you are lucky, your bug may be already fixed on GitHub. However, this doesn’t mean that the fix is already included in the latest stable release. Is there a way to find out if a particular PR got merged into a specific Salt release? Depending on the release ETA, you may decide either to invent a hacky workaround or just simply wait.
Alternatively, you may want to submit a fix yourself. However, it is not immediately clear on which branch you should base the fix. How the branches evolve? What is the difference between the version number (e.g., 2018.3.4) and codename (Oxygen)? Is there a mapping between them or a calendar that contains future release dates?
By closely watching the official GitHub repo, reading the documentation and hanging out in the Slack channel, I was able to collect and summarize various useful bits into this guide. I hope it helps.
When is an existing merged PR will be released?
There are several methods which you can use to find tags or branches that contain the PR you want.
1. ZRELEASED labels on GitHub
If a PR contains a yellow ZRELEASED - YYYY.MM.X
label, then it is already released, and you need to upgrade your Salt version.
2. Git tags on Github
You can look for Git tags that contain the necessary commits:
If there are no tags and the commit was merged into the develop
branch, then you probably have to wait for the next major release. Bugfixes for older branches are eventually merged into newer ones. As for bugfixes in the develop
branch, you can politely ask to backport them or do it yourself (see How to submit a PR).
3. Git branches or tags
Let’s say you want to find when the PR #46435 is going to be released. First, you need to clone the repo:
% git clone https://github.com/saltstack/salt
% cd salt
Then you need to find a commit that belongs to this PR. You can do this on GitHub (see the method 2 above), or using CLI:
% git log --grep '#46435'
commit d6263a0a4fcd2573afc5177831c20f003cbc74ce
Merge: f9c60ad6f7 b48255e101
Author: Nicole Thomas <nicole@saltstack.com>
Date: Fri Mar 9 09:52:22 2018 -0500
Merge pull request #46435 from onlyanegg/serializer_extmod
Adding serializer extmod
The last step is to find which branches (or tags) contain the specific commit:
% git branch -a --contains d6263a0a4fcd2573afc5177831c20f003cbc74ce
* develop
remotes/origin/2019.2
remotes/origin/2019.2.0.rc1
remotes/origin/HEAD -> origin/develop
remotes/origin/develop
remotes/origin/hotfix/pytest
remotes/origin/revert-50798-45839-develop
remotes/origin/revert-51444-develop.py2
% git tag --contains d6263a0a4fcd2573afc5177831c20f003cbc74ce
v2018.11
v2019.2
v2019.2.0rc1
v2019.2.0rc2
From the output above we can infer that the feature will be released in version 2019.2.0
, which is based on the 2019.2
feature branch (also known as Fluorine). The v2018.11
tag should be ignored because there are no branches with the same version (see the full git branch -a
output).
4. Version-check tool
The process above could be simplified using the version_check
tool and Docker. To install it clone the repo:
% git clone https://github.com/rallytime/version-check.git
% cd version-check/
And build the image:
% docker build -t version_check .
Finally, to use it run the following command (46435 is the PR number):
% docker run --rm -it version_check -p 46435
Branches:
2019.2
2019.2.0.rc1
develop
revert-50798-45839-develop
revert-51444-develop.py2
Tags:
v2018.11
v2019.2
v2019.2.0rc1
v2019.2.0rc2
For usage details see the project’s README.
5. Salt-version-as-a-service
The highest layer of abstraction is available as the online tool http://versions.gtmanfred.com (internally it utilizes the above-mentioned version_check
script). Unfortunately, it is no longer online. You could try to host it yourself or ping Daniel Wallace
on Twitter and politely ask if he can bring it back to life.
How to submit a PR
Many times you’ll have to submit a fix yourself. Below is a simplified Salt branching model:
- New features should be submitted against the
develop
branch. At some point, the branch is forked into a feature frozen codename branch. When a release date is decided upon, codename branch is renamed into the main release branch. - Bug fixes and docs should go into the oldest affected (and supported) main release branch (e.g.,
2017.7
). This is a base branch for dot release branches. - Hotfixes should go into the corresponding dot release branch (e.g.,
2018.3.4
) - Eventually, fixes will be merged into the newer branches by SaltStack’s release team
- Occasionally, some fixes that went into the
develop
branch will be back-ported to an older branch
The official documentation explains Salt branches in a more detailed fashion. To get more information on the release process, consult the Salt Git Policy and Salt Release Process pages.
Release schedule
Salt doesn’t currently have a scheduled release cadence. However, according to PyPI data, the expected time between releases is:
- 229 days on average for major (feature) releases (varies from 126 to 329 days)
- 48 days on average for bugfix releases (varies from 1 to 272 days)
These are rough estimates, to stay in the loop you can subscribe to the salt-announce mailing list, or join the #releases
channel on SaltStack Community Slack.
I haven’t found any reliable data that reflects the SaltStack feature planning process, but below are some clues you can use:
- Product support lifecycle table that outlines support phases and EOL dates. In short: there are two major active branches; oldest previous active is CVE-only; previous CVE-only is EOL.
- The most up-to-date list of codenames
- Github labels to track the progress on issues assigned to the specific tag (release candidates in particular)
- Github PR search to see how many open PRs are left against a specific branch
Just before a new release, you can see it being tagged on GitHub. After that it is typically a matter of days for packages to be built and uploaded. A release is usually happens in the first half of the week (70% of PyPI uploads happened in the first three days).
Next step
If you have some free time and want to improve Salt codebase, go to the issue tracker and help to fix a bug!
Want to get more tips like this one?
Salt Tips is a side project for me, so expect only a few emails per month. Unsubscribe at any time.
Powered by Mailgit
Also, you can follow me on Twitter where I periodically post things like this: