[Jack-Devel] Status of the jackaudio.org homepage (bad news)

Hello users and developers of JACK,

recently, there have been a lot more questions and remarks about the
jackaudio.org homepage and the releases, especially of jack2.

Since most of these questions and remarks are entirely justified, this
mail (bad news) will provide some insight I have about the current
situation of the page, the release process and how all these things did
happen up to now. This is a long mail, only read it if it is really
relevant to you.

1 the page
The main part of the current homepage for JACK ( jackaudio.org ) is a
set of static html pages, generated from markdown files with jekyll [1].
The whole build process for the static page is compatible to the GitHub
pages [2] infrastructure, which means it can be built completely with
and without GitHub and as well as deployed as a GitHub page. Further
details can be obtained from the README in the GitHub repository. The
page also contains a html version of the JACK API documentation and some
of the tutorials and Information were moved to a Wiki provided by
GitHub. The sources[3] for the page are hosted as a dedicated repository
on GitHub under the jackaudio organization, together with jack1 and
jack2. Attached to the repository, you can also find the mentioned
Wiki[4]. There is also a dedicated bug tracker for the page attached to
the repository of the homepage on GitHub.

2 webpage hosting
The page is currently hosted on a webspace owned by Paul Davis, who also
owns the jackaudio.org domain. To my best knowledge, only Paul and
Stèpahe Letz have currently access to the webspace. Hosting the page on
this webspace instead of using the GitHub pages infrastructure directly
was a decision made for various reasons.
The domain is also associated to the jack mailing lists which made it
difficult at that time (2014) to point the domain to GitHub pages instead.
Having a page that was not dependent on the GitHub pages infrastructure
was a requirement at the time we transitioned from a broken and hacked
Drupal installation to a static page.

3 webpage development and deployment
The GitHub repository is currently maintained by me with help in form of
bug reports, pull requests and edits of the wiki from the community. If
I determine (by the rule of thumb) that enough changes to the page have
accumulated or a release of jack1 or jack2 occurs, I prepare the page to
be pulled and basically send a pull request to the jack-devel mailing
list, signalling that the github repo should be pulled to the webspace.
Since the page is designed in a way to be compatible with GitHub pages
and already set up to work as a GitHub page, it became accessible under
the github domain [5], which was nice for development purposes and was
supposed to serve as a preview before pulling changes to the webspace.

4 The case of JACK release binaries
The page also links to release binaries and source tarballs of jack1 and
jack2 (at least it's supposed to). These binaries are not included in
the git repository since having multiple large binaries inside the
repository would make it unfriendly to clone, especially after a few
releases. Multiple approaches to hosting the binaries have been
discussed. Paul did upload the binaries for jack1 to the mentioned
webspace after a release. For jack2, Stéphane decided to use DropBox
public files to host the release binaries. The use of tags on the
respective repositories of jack1 and jack2 was discussed multiple times.
Attaching the release binaries to tags could have solved the hosting
problem. Still, there was the requirement to be independent from GitHub
so this idea was not followed through. Also GitHub LFS was considered
but wasn't ready and released by GitHub in 2014/2015 and also wouldn't
really have resolved the issue of depending on GitHub, since the files
would have still been hosted on GitHub infrastructure. On a new release,
the maintainers would announce new links to the binaries and I would
update the site accordingly. It required manual intervention but since
release cycles were already long I wasn't too worried about that.

5 how it all came down
Over time minor challenges arose. Sometimes it was just GitHub changing
some configuration for their jelkyll configuration, sometimes DropBox
would limit the number of Downloads, sometimes it would take a few days
longer or required a second reminder to get my changes puled to the
webspace. At some point Paul decided to step down as the lead developer
of jack1 but keeping the webspace and the domain online. Reaction times
for getting changes to the webspace became longer (at least it felt like
that to me) and then also DropBox terminated their free hosting of
public shares. At that point a lot of people were also asking for an
update of the jack2 installer on OSX. The whole thing resulted in
multiple questions on the mailinglists, reports in bugtrackers all over
place on lists and bugtrackers I didn't even knew I was subscribed to,
someone even managed to use the review feature on a commit of the jack
repo. I did some mitigation on the bugtrackers and after some time went
by I made a grave mistake.
I went ahead and grabbed the old binaries from archive.org, created tags
for the releases of jack2 for windows and OSX on the repository of the
homepage and attached the files to these tags. I updated the links on
the page, did some additional maintenance and requested the changes to
be pulled to the webspace on the jack-devel list. That was intended to
be an intermediate solution but you may have already guessed that one
part of my mistake: There is a saying, that intermediate solutions are
the ones that will never go away, so sad, so true. Second, attaching the
binaries to the repo of the webpage confused a lot of people since
normally the tag would have been attached to a commit of the jack2
repository. When a tag is created, GitHub automatically creates a source
tarball and zip for the respective commit, so that a developer only has
to attach the release binaries. I could no do this, since a have no
access to the jack2 repository. The problem is, that users do not expect
to find the releases attached to the repository of the homepage and
also, if you take a look at these tags [6], you will notice that they
have the source tarballs of the homepage attached where a user would
normally expect the source tarball of jack2.
It's a mess right now more so because shortly afterwards I received a
release of the new installer for OSX and Windows and now there is
another release that uses this suboptimal solution and the old page on
the webspace still has the DropBox links. I updatetd the links on the
page in the GitHub repo accordingly, hoping that people would search for
the binaries on the page after it was (supposed to be) pulled, requested
the changes to be pulled to the webspace... that was half a month ago
and there was apparently no pull.

So that's how we ended up with broken dropbox links on the webspace
(jackaudio.org), two* versions of the page (jackaudio. org and [5]) and
a whole lot of confusion and frustration about source tarballs
containing strange markdown files and not the jack2 source code.

The technical problems are actually not that hard to solve, but a think
wen have an organizational problem here.

There will be a follow up (good news) to this mail where I propose some
ways to clear this mess and how to avoid it in the future, it may be a
base for discussion and LAC should be a good opportunity to get this
sorted out since the most important decision makers will be there.

best regards

Markus


* To be more correct and as a very observant user mentioned, there are
three versions since the gh-pages branch of the repo also gets special
treatment (see GitHub pages docs for details) but that is not that much
relevant for this topic.

[1] http://jekyllrb.com/
[2] https://pages.github.com/
[3] https://github.com/jackaudio/jackaudio.github.com
[4] https://github.com/jackaudio/jackaudio.github.com/wiki
[5] https://jackaudio.github.io/
[6] https://github.com/jackaudio/jackaudio.github.com/releases

PS: It's already 1 AM and I am still *jacked* and *pun*ching

1494026677.24762_0.ltw:2, <d124907a-9530-f627-39d2-ab2a6a0b8f05 at spectralbird dot de>