Managing the Repositories¶

While many people contribute to our code-bases, this page is for the elect few who are responsible for maintaining them. You will need access to the service accounts to make use of the information contained in this page.

Lintrack¶

lintrack refers to the location of the codebase in the TN

/afs/cern.ch/eng/sl/lintrack/

as well as the service account handling the access to it.

The service account was created as in the past direct access to these repositories lead to accidental changes that affected all users. Limiting access to the service account ensures an active decision to make modifications and thus avoids unwanted changes and binary or data loss, which can be especially troublesome if only detected online during CCC measurements.

You can log into the lintrack account with the following command:

ssh lintrack@lxplus.cern.ch

You will then be greeted with a welcome message and some information about shortcuts we have set up for quick access and management of the repositories.

Access

Due to CERN regulations, you cannot log into the lintrack account from outside the CERN network.

Repositories¶

Local copies of the GitHub repositories are kept in /afs/cern.ch/eng/sl/lintrack/omc_repos/. There is a shortcut available on the lintrack welcome screen to update all repositories.

These repositories are used in the python_edge environment and might be modified online to test new features or fix issues.

Keeping them clean

Changes to the repositories should be short lived and patched into the packages properly within a day or two and the repository reset to its original state. In the LHC Run 2, changes made during shifts were often forgotten and after a while no one knew why the changes were made which version was the correct one to use.

Python Installations¶

The python installations are located in /afs/cern.ch/eng/sl/lintrack/omc_acc_py/. In this folder you can also find the acc-py installers as well as the omc_requirements_*.txt files to easily setup new environments.

/afs/cern.ch/eng/sl/lintrack/omc_acc_py/base/ contains the base acc-py installations from which the environments are derived.
/afs/cern.ch/eng/sl/lintrack/omc_acc_py/venv/ contains the environments. These are automatically picked up by the BetaBeat GUI.

The two maintained environments are:

omc_py3xx_releases: The main production environment which is used for the CCC. All packages are installed via their official PyPI releases.
omc_py3xx_repos: The development environment which is used for the latest un-releases features or quick patches during shifts. All packages are editable installs from the local git repositories.

These two environments are also symlinked to /afs/cern.ch/eng/sl/lintrack/omc_python3 and /afs/cern.ch/eng/sl/lintrack/omc_python_edge respectively. The /afs/cern.ch/eng/sl/lintrack/omc_python3 is activate by default for the lintrack account and is the default interpreter chosen by the BetaBeat GUI.

Some regular maintainer TODOs

On new releases

Whenever there is a new release for one our packages, it needs to be manually installed into the omc_py3xx_releases environment. Shortcuts are available (see the lintrack welcome screen), but for single updates one can use python -m pip install --upgrade <package>.

On new features to master

To keep the newest version of a repository in use in python_edge, you need to manually pull the changes into the local repository (see also the lintrack welcome screen for quick command aliases). Due to the editable installs this is enough.

On new Acc-Py releases

To upgrade to a new acc-py version, a new base environment needs to be created and the omc_py3xx_releases and omc_py3xx_repos environments derived. See the Acc-Py installation instructions on the previous page. There is also an AccPy_Installation_README.md file in /afs/cern.ch/eng/sl/lintrack/omc_acc_py/ to guide you through the process, specifically for our filesystem structure. You can use the omc_requirements_*.txt files to re-derive the environments. To avoid issues, the former and new bases/environments can be kept in parallel for a while. Don't forget to also update the omc_python3 and omc_python_edge symlinks!

Github¶

The settings for the pylhc organization should be automatically available to you, if you have been given the right permissions (either admin or code owner).

Our repositories use the .github configuration repository to manage:

Issue Templates
Workflows
Labels

Of these, the workflow to assign labels to all repositories is special: it is triggered on pushes to the master branch of the .github repository and assigns the defined labels to all repositories defined in the workflow.

While the other workflows are usually triggered directly via workflow inheritance from the other repositories, e.g.

jobs:
    tests:
        uses: pylhc/.github/.github/workflows/cron.yml@master

These automatically inherit the GITHUB_TOKEN of the repository and have access to the repository's data. The labels-workflow does not have this access and is using a limited personal access token (PTA) instead, which is saved as the ISSUE_WRITE_TOKEN. It is provided by the pylhctokens service account and needs to be renewed on a yearly basis.

Update Github Actions!

Many of the workflows are using pre-defined github actions. Check them sometimes for updates and try to keep their versions up-to-date!

pylhctokens¶

This service account was originally created to give workflows access to the repos, until GitHub introduced the automatically generated GITHUB_TOKEN. Nowadays we only use it to assign labels to all repositories.

To log into the pylhctokens account you need password and 2FA authentication.

CODEOWNERS¶

To avoid malicious or accidental changes to the repositories, the master branches are locked (Repo -> Settings -> Branch protection) and reviews are required before pull requests can be merged.

The approved reviewers are handled via the approved-reviewers group in the pylhc organization's teams which are assigned as code owners to each repository. For that purpose, each repository has a CODEOWNERS file, which can be found in the .github folder of the repository.

For more details, see the official documentation.

PyPI¶

The access to PyPI is provided by the pylhc service account, for which you will need password and 2FA authentication.

Publishing access is given to the github workflows via API-Token (Settings -> Account Settings -> API Tokens) to all repositories of the pylhc organization. This is done by using the publish workflow and passing the token as secret:

PYPI_USERNAME : __token__ (literal string)
PYPI_PASSWORD : the token value, including the pypi- prefix

These are usually inherited from each of the repos publishing workflows.

Zenodo¶

There is no special account for Zenodo access. Zenodo should be able to be reached with your normal account, preferably your GitHub account which, if linked correctly, will allow Zenodo to also reach the pylhc organization's repositories for accounts with admin permissions.

To add a new repository to Zenodo, make sure the .zenodo.json file is correct. Examples can be found in our repositories. Before making your first release, go into the Zenodo github settings (down-arrow on your user on the top right) and follow these instructions:

Flip the switch to On
Create a release on GitHub -> The page for your repo should now be visible on Zenodo!
(Optional) Add the badge to your README.md