AutoSix

AutoSix is a wrapper to automatically perform the necessary setup and steps needed for SixDesk use.

The functionality is similar to the pylhc_submitter.job_submitter in that the inner product of a replace_dict is used to automatically create a set of job-directories to gather the data. To avoid conflicts, each of these job-directories is a SixDesk workspace, meaning there can be only one study per directory. Beware that the max_materialize limit is set for each of these workspaces individually, not for all Jobs together (i.e. it should be <=MAX_USER_JOBS / NUMBER_OF_WORKSPACES).

The replace_dict contains variables for your mask as well as variables for the SixDesk environment. See the description of replace_dict below.

In any other way, these special variables behave like normal variables and can also be inserted in your mask. They are also looped over in the same manner as any other variable (if given as a list).

For additional information and guides, see the AutoSix page on the OMC documentation site.

Problems with search.f90

If you run into

ImportError: cannot import name 'search' from partially initialized module 'sixdeskdb'

that means that the SixDesk/utilities/externals/SixDeskDB/sixdeskdb/search.f90 has not been compiled for your current python and OS version (indicated by XXX below). There are two ways to do this:

a) Run

python -m numpy.f2py -c search.f90 -f search

with your desired python version (a shortcut f2py might be available if you run in an activated venv). Copy (or symlink) the resulting search.cpython-XXX.so file into the sixdeskdb folder (if it is not already there because you ran from that folder)

b) Run

python setup.py build_ext

on the setup.py in SixDesk/utilities/externals/SixDeskDB. Then copy (or symlink) the resulting build/lib.XXX/sixdeskdb/search.cpython-XXX.so into the sixdeskdb folder (the name needs to stay as it is).

Arguments:

--Required--

  • mask (Path):

    Path to the program’s mask to use.

  • replace_dict (DictAsString):

    Dict with keys of the strings to be replaced in the mask (required) as well as the mask_sixdeskenv and mask_sysenv files in the sixdesk_tools module. Required fields are TURNS, AMPMIN, AMPMAX, AMPSTEP, ANGLES. Optional fields are RESUBMISSION, PLATFORM, LOGLEVEL, FIRSTSEED, LASTSEED, RUNTYPE, NPAIRS, EMITTANCE, DIMENSIONS, WRITEBINS, ENERGY. These keys can also be used in the mask if needed. The values of this dict are lists of values to replace these or single entries.

  • working_directory (Path):

    Directory where data should be put into.

--Optional--

  • apply_mad6t_hacks:

    Apply two hacks: Removes ‘<’ in binary call and ignore the check for ‘Twiss fail’ in the submission file. This is hack needed in case this check greps the wrong lines, e.g. in madx-comments. USE WITH CARE!!

    action: store_true

  • da_turnstep (int):

    Step between turns used in DA-vs-Turns plot.

    default: 100

  • executable (PathOrStr):

    Path to executable or ‘madx’, ‘python2’, ‘python3’ to use the OMC default paths on AFS.Defaults to the latest MADX-Binary on AFS.

    default: /afs/cern.ch/user/m/mad/bin/madx

  • jobid_mask (str):

    Mask-String to name jobs, with placeholders from the replace_dict keys.

  • max_materialize (int):

    Maximum jobs to be materialized in scheduler (per SixDesk Workspace!).. Here: None leaves the settings as defined in the SixDesk htcondor_run_six.sub template and 0 removes it from the template. Warning: This setting modifies the template in the sixdesk_directory permanently. For more details see the htcondor API.

  • max_stage (str):

    Last stage to be run. All following stages are skipped.

  • python2 (PathOrStr):

    Path to python to use with run_six.sh (python2 with requirements installed). ONLY THE PATH TO THE DIRECTORY OF THE python BINARY IS NEEDED! And it can’t be an Anaconda Distribution. If None the system’s python is used (SixDesk internally).

    default: None

  • python3 (PathOrStr):

    Path to python to use with sixdb (python3 with requirements installed).Defaults to the system’s python3.

    default: python3

  • resubmit:

    Resubmits to HTCondor if needed (i.e. in case it finds errors with the previous run).

    action: store_true

  • sixdesk_directory (Path):

    Path to the directory of SixDesk. Defaults to the PRO-version on AFS. If you are using your own SixDesk Environment and it does not run, check the AutoSix doc.

    default: /afs/cern.ch/project/sixtrack/SixDesk_utilities/pro

  • ssh (str):

    Run htcondor from this machine via ssh (needs access to the working_directory)

  • stop_workspace_init:

    Stops the workspace creation before initialization, so one can make manual changes.

    action: store_true

  • unlock:

    Forces unlocking of folders (if they have been locked by Sixdesk).

    action: store_true

author:

jdilly

pylhc_submitter.autosix.main(opt)[source]

Loop to create jobs from replace dict product matrix.

pylhc_submitter.autosix.run_job(jobname: str, jobargs: dict, env: AutoSixEnvironment)[source]

Main submitting procedure for single job.

Parameters:

  • jobname (str) -- Name of the job/study

  • env (DotDict) -- The ensemble of autosix settings as an AutoSixEnvironment object.

  • jobargs (dict) -- All Key=Values needed to fill the mask!