2 Nightly_Builds_Setup
boklm edited this page 2023-07-05 16:30:20 +00:00

This page describes the setup we use to produce nightly builds and sign them. For downloading nightly builds see the Nightly Builds Download page.

Building of nightly builds

The nightly builds are done on tb-build-06.torproject.org (see tor-browser-build#40196 and tpo/applications/tor-browser-bundle-testsuite#40075).

To do the nightly builds, we use tor-browser-bundle-testsuite, which is responsible pulling the tor-browser-build repository (in /home/tb-builder/tor-browser-bundle-testsuite/clones/tor-browser-build), then starting the builds of the different platforms, generating build report pages and sending an email with the results. The main files related to this in tor-browser-bundle-testsuite are:

  • TBBTestSuite/TestSuite/TorBrowserBuild.pm
  • config/tb-build-06.torproject.org
  • rbm-config/tb-build-06.torproject.org.rbm.local.conf - rbm.local.conf configuration file copied to the tor-browser-build directory
  • tools/tb-build-06-start-nightly-build - the main script starting nightly builds. This is started every night by a cron job by the tb-builder user.
  • tools/rsync-to-tbb-nightlies-master - the script doing the rsync to tbb-nightlies-master, which is run after each build

The cleaning of old builds is done using the script /home/tb-builder/tor-browser-bundle-testsuite/clones/tor-browser-build/tools/prune-old-builds. This is called by the script tools/tb-build-06-start-nightly-build.

The main directories we use on tb-build-06 are:

  • /home/tb-builder/tor-browser-bundle-testsuite - git clone of tor-browser-bundle-testsuite
  • /home/tb-builder/nightly-builds - directory containing the build files and build reports. This is the directory that we rsync to tbb-nightlies-master.

rsync between tb-build-06 and tbb-nightlies-master

The builds done on tb-build-06 are transferred to tbb-nightlies-master using rsync. We use rrsync to restrict access. The script /home/tbb-nightlies/bin/rrsync is a copy of /usr/share/doc/rsync/scripts/rrsync.

When builds are finished, we also need to run static-update-component on tbb-nightlies-master. We do that using a second ssh key.

We have the following ssh keys on tb-build-06:

  • .ssh/id_rsa - Used for rsync
  • .ssh/id_rsa_static-update-component - Used to run static-update-component

In /etc/ssh/userkeys/tbb-nightlies we use the command option to restrict access to directory /srv/tbb-nightlies-master.torproject.org/htdocs/nightly-builds with rrsync, and to force running static-update-component with the second key:

command="/home/tbb-nightlies/bin/rrsync /srv/tbb-nightlies-master.torproject.org/htdocs/nightly-builds",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding ssh-rsa [...] tb-builder@tb-build-06
command="/usr/local/bin/static-update-component nightlies.tbb.torproject.org",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding ssh-rsa [...] tb-builder@tb-build-06

Signing of nightly builds

The signing of nightly builds is done with the script tor-browser-build/tools/signing/nightly/sign-nightly, and the file config.yml in the same directory contains its configuration.

The sign-nightly script is run on tbb-nightlies-master.torproject.org by the tbb-nightlies user as a cron job:

0,30 * * * * torsocks /home/tbb-nightlies/tor-browser-build/tools/signing/nightly/sign-nightly >> /home/tbb-nightlies/sign-nightly.log 2>&1
15 3 * * 0 mv -f /home/tbb-nightlies/sign-nightly.log /home/tbb-nightlies/sign-nightly.log.1

Logs can be found in the file /home/tbb-nightlies/sign-nightly.log.

If you push a commit changing sign-nightly or its configuration file, you need to checkout the new commit in /home/tbb-nightlies/tor-browser-build on tbb-nightlies-master.torproject.org.