Scripts

BetaBeat.src Output Converter

Script to convert most important output files produced by BetaBeat.src / GetLLM into the standard format used in omc3 to allow straight forward comparison of the two.

Good to know: in BetaBeat.src the _free files correspond to AC dipole (or ADT-AC dipole) compensation by analytic equation while _free2 corresponds to compensation via effective model. It is written in GetLLM (at least once) here: https://github.com/pylhc/Beta-Beat.src/blob/63c5e39f63b03c00d18289cc9813a912fa6b933f/GetLLM/GetLLM.py#L606

omc3.scripts.betabeatsrc_output_converter.convert_old_beta_from_amplitude(inputdir: Path | str, outputdir: Path | str, suffix: str, plane: str, old_file_name: str = 'ampbeta', new_file_name: str = 'beta_amplitude_') → None

Looks in the provided directory for expected beta from amplitude file from BetaBeat.src for a given plane, converts it to the output format used by omc3 and write them to the new location.

The file naming should be getampbeta(x,y).out, with the following expected columns: NAME, S, COUNT, BETX, BETXSTD, BETXMDL, MUXMDL, BETXRES, BETXSTDRES.

Parameters:

• inputdir (Union[Path, str]) -- Location of the directory with BetaBeat.src output files.

• outputdir (Union[Path, str]) -- Location of the output directory for converted files.

• suffix (str) -- Compensation suffix used in the provided BetaBeat.src output files.

• plane (str) -- the transverse plane for which to look for the output file.

• old_file_name (str) -- the standard naming for the old output file.

• new_file_name (str) -- the standard naming for the new converted file.

omc3.scripts.betabeatsrc_output_converter.convert_old_beta_from_phase(inputdir: Path | str, outputdir: Path | str, suffix: str, plane: str, old_file_name: str = 'beta', new_file_name: str = 'beta_phase_') → None

Looks in the provided directory for expected beta from phase file from BetaBeat.src for a given plane, converts it to the output format used by omc3 and write them to the new location.

The file naming should be getbeta(x,y).out, with the following expected columns: NAME, S, COUNT, BETX, SYSBETX, STATBETX, ERRBETX, CORR_ALFABETA, ALFX, SYSALFX, STATALFX, ERRALFX, BETXMDL, ALFXMDL, MUXMDL, NCOMBINATIONS.

Parameters:

• inputdir (Union[Path, str]) -- Location of the directory with BetaBeat.src output files.

• outputdir (Union[Path, str]) -- Location of the output directory for converted files.

• suffix (str) -- Compensation suffix used in the provided BetaBeat.src output files.

• plane (str) -- the transverse plane for which to look for the output file.

• old_file_name (str) -- the standard naming for the old output file.

• new_file_name (str) -- the standard naming for the new converted file.

omc3.scripts.betabeatsrc_output_converter.convert_old_closed_orbit(inputdir: Path | str, outputdir: Path | str, plane: str, old_file_name: str = 'CO', new_file_name: str = 'orbit_') → None

Looks in the provided directory for expected closed orbit file from BetaBeat.src for a given plane, converts it to the output format used by omc3 and write them to the new location.

The file naming should be getCO(x,y).out, with the following expected columns: NAME, S, COUNT, X, STDX, XMDL, MUXMDL.

Parameters:

• inputdir (Union[Path, str]) -- Location of the directory with BetaBeat.src output files.

• outputdir (Union[Path, str]) -- Location of the output directory for converted files.

• plane (str) -- the transverse plane for which to look for the output file.

• old_file_name (str) -- the standard naming for the old output file.

• new_file_name (str) -- the standard naming for the new converted file.

omc3.scripts.betabeatsrc_output_converter.convert_old_coupling(inputdir: Path | str, outputdir: Path | str, suffix: str, old_file_name: str = 'couple', new_file_name: str = 'f') → None

Looks in the provided directory for expected coupling file from BetaBeat.src, converts it to the output format used by omc3 and write them to the new location.

The file naming should be getcouple(x,y).out, with the following expected columns: NAME, S, COUNT, F1001W, FWSTD1, F1001R, F1001I, F1010W, FWSTD2, F1010R, F1010I, Q1001, Q1001STD, Q1010, Q1010STD, MDLF1001R, MDLF1001I, MDLF1010R, MDLF1010I.

Parameters:

• inputdir (Union[Path, str]) -- Location of the directory with BetaBeat.src output files.

• outputdir (Union[Path, str]) -- Location of the output directory for converted files.

• suffix (str) -- Compensation suffix used in the provided BetaBeat.src output files.

• old_file_name (str) -- the standard naming for the old output file.

• new_file_name (str) -- the standard naming for the new converted file.

omc3.scripts.betabeatsrc_output_converter.convert_old_directory_to_new(opt: DotDict) → None

Looks in the provided directory for expected BetaBeat.src output files, converts it to the output format used by omc3 and write them to the new location.

Parameters:

opt (DotDict) -- The entrypoint parameters parsed from the command line.

omc3.scripts.betabeatsrc_output_converter.convert_old_dispersion(inputdir: Path | str, outputdir: Path | str, plane: str, old_file_name: str = 'D', new_file_name: str = 'dispersion_') → None

Looks in the provided directory for expected dispersion file from BetaBeat.src for a given plane, converts it to the output format used by omc3 and write them to the new location.

The file naming should be getD(x,y).out, with the following expected columns: NAME, S, COUNT, DX, STDDX, DPX, DXMDL, DPXMDL, MUXMDL.

Parameters:

• inputdir (Union[Path, str]) -- Location of the directory with BetaBeat.src output files.

• outputdir (Union[Path, str]) -- Location of the output directory for converted files.

• plane (str) -- the transverse plane for which to look for the output file.

• old_file_name (str) -- the standard naming for the old output file.

• new_file_name (str) -- the standard naming for the new converted file.

omc3.scripts.betabeatsrc_output_converter.convert_old_normalised_dispersion(inputdir: Path | str, outputdir: Path | str, plane: str, old_file_name: str = 'ND', new_file_name: str = 'normalised_dispersion_') → None

Looks in the provided directory for expected normalized dispersion file from BetaBeat.src for a given plane, converts it to the output format used by omc3 and write them to the new location.

The file naming should be getND(x,y).out, with the following expected columns: NAME, S, COUNT, NDX, STDNDX, DX, DPX, NDXMDL, DXMDL, DPXMDL, MUXMDL.

Parameters:

• inputdir (Union[Path, str]) -- Location of the directory with BetaBeat.src output files.

• outputdir (Union[Path, str]) -- Location of the output directory for converted files.

• plane (str) -- the transverse plane for which to look for the output file.

• old_file_name (str) -- the standard naming for the old output file.

• new_file_name (str) -- the standard naming for the new converted file.

omc3.scripts.betabeatsrc_output_converter.convert_old_phase(inputdir: Path | str, outputdir: Path | str, suffix: str, plane: str, old_file_name: str = 'phase', new_file_name: str = 'phase_') → None

Looks in the provided directory for expected phase file from BetaBeat.src for a given plane, converts it to the output format used by omc3 and write them to the new location.

The file naming should be getphase(x,y).out, with the following expected columns: NAME, NAME2, S, S1, COUNT, PHASEX, STDPHX, PHXMDL, MUXMDL.

Parameters:

• inputdir (Union[Path, str]) -- Location of the directory with BetaBeat.src output files.

• outputdir (Union[Path, str]) -- Location of the output directory for converted files.

• suffix (str) -- Compensation suffix used in the provided BetaBeat.src output files.

• plane (str) -- the transverse plane for which to look for the output file.

• old_file_name (str) -- the standard naming for the old output file.

• new_file_name (str) -- the standard naming for the new converted file.

omc3.scripts.betabeatsrc_output_converter.convert_old_total_phase(inputdir: Path | str, outputdir: Path | str, suffix: str, plane: str, old_file_name: str = 'phasetot', new_file_name: str = 'total_phase_') → None

Looks in the provided directory for expected total phase file from BetaBeat.src for a given plane, converts it to the output format used by omc3 and write them to the new location.

The file naming should be getphasetot(x,y).out, with the following expected columns: NAME, NAME2, S, S1, COUNT, PHASEX, STDPHX, PHXMDL, MUXMDL.

Parameters:

• inputdir (Union[Path, str]) -- Location of the directory with BetaBeat.src output files.

• outputdir (Union[Path, str]) -- Location of the output directory for converted files.

• suffix (str) -- Compensation suffix used in the provided BetaBeat.src output files.

• plane (str) -- the transverse plane for which to look for the output file.

• old_file_name (str) -- the standard naming for the old output file.

• new_file_name (str) -- the standard naming for the new converted file.

omc3.scripts.betabeatsrc_output_converter.converter_entrypoint(opt: DotDict) → None

Looks for expected BetaBeat.src output files in the provided input directory, converts them to the format used in omc3 and writes the converted files in the provided output directory.

--Required--

• inputdir (str):

  Directory with BetaBeat.src output files.

• outputdir (str):

  Output directory for converted files.

--Optional--

• suffix (str):

  AC dipole compensation suffix used in the provided BetaBeat.src output (‘_free’ for compensation by equation, ‘_free2’ by model).

  choices: ('', '_free', '_free2')

  default: _free

Merge KMOD Results

Script to merge the results from KMOD into one TfsDataFrame. The script takes the kmod-results folders as input and merges the lsa-result tfs-files in these together. The resulting TfsDataFrame is returned, but also written out if an outputdir is given.

BPMWKs (common for both beams) are hereby dropped, to avoid duplicated elements.

Headers of the same name will be overwritten (depending on the alphabetical order of the directory names).

Some sanity checks are performed, e.g. that there is only one entry per element. If IP1 and IP5 results are given for both planes and beams, the luminosity imbalance between these IPs is also calculated and written into the header, as well as logged.

Arguments:

--Required--

• kmod_dirs (Path):

  Path to kmod directories with stored KMOD measurement files,in particular lsa_results.tfs

--Optional--

• outputdir (Path):

  Output directory where to write the result tfs

omc3.scripts.merge_kmod_results.get_kmod_ip_subdirs(kmod_dirs: List[Path]) → List[Path]

Returns the paths to the ip-subdirectories in the kmod result directories.

Parameters:

kmod_dirs (List[Path]) -- Main kmod result directories with IP-subdirectories.

Returns:

(List[Path]) Paths to all subdirectories.

omc3.scripts.merge_kmod_results.get_lumi_imbalance(data_frame: TfsDataFrame) → Tuple[AffineScalarFunc, AffineScalarFunc, AffineScalarFunc]

Calculate the IP1 / IP5 luminosity imbalance. The luminosity is taken as defined in Concept of luminosity, Eq(17): https://cds.cern.ch/record/941318/files/p361.pdf

The calculation of the luminosity imbalance is then:

\[\frac{L_{IP1}}{L_{IP5}}=\frac{\sqrt{\beta_{x1,IP5}+\beta_{x2,IP5}} \cdot\sqrt{\beta_{y1,IP5}+\beta_{y2,IP5}}} {\sqrt{\beta_{x1,IP1}+\beta_{x2,IP1}}\cdot\sqrt{\beta_{y1,IP1}+\beta_{y2,IP1}}}\]

and the error:

\[\begin{split}\sigma_{\frac{L_{IP1}}{L_{IP5}}} = \frac{1}{2}\frac{L_{IP1}}{L_{IP5}} \cdot \sqrt{\sum_{\substack{z \in (x,y) \\ i \in (IP1, IP5) }} {\frac{\sigma^2_{\beta_{z1,i}} + \sigma^2_{\beta_{z2,i}}} {(\beta_{z1,i}+\beta_{z2,i})^2}}}\end{split}\]

Parameters:

data_frame (tfs.TfsDataFrame) -- a TfsDataFrame with the results from a kmod analysis.

Returns:

Tuple with the imbalance, the betas at IPs as ufloats.

omc3.scripts.merge_kmod_results.merge_kmod_results(opt) → TfsDataFrame

Main function to merge K-Mod output. See omc3.scripts.merge_kmod_results.

omc3.scripts.merge_kmod_results.merge_tfs(directories: List[Path], filename: str) → TfsDataFrame

Merge different kmod analysis results from a list of directories into a single TfsDataFrame.

Parameters:

• directories (List[pathlib.Path]) -- list of PosixPath objects to directories holding TFS files with the results of kmod analysis.

• filename (str) -- name of the TFS files to look for in the provided directories.

Returns:

A TfsDataFrame combining all the loaded files from the provided directories.

Update Natural Tune in Lin-Files

Script to update the natural tune in lin files, based on the spectrum data (amps and freqs) and a given frequency interval.

Arguments:

--Required--

• files: List of paths to the spectrum files. The files need to be given without their ‘.lin’/’.amps[xy]’,’.freqs[xy]’ endings. (So usually the path of the TbT-Data file.)

• interval (float): Frequency interval in which the highest peak should be found.

--Optional--

• bpms: List of BPMs which need to be updated. If not given it will be all of them.

• not_found_action (str): Defines what to do, if no line was found in given interval. ‘error’: throws a ValueError; ‘remove’: removes the bpm; ‘ignore’: keeps the old values.

  Choices: ['error', 'remove', 'ignore'] Default: error

• planes (str): Which planes.

  Choices: ('X', 'Y') Default: ['X', 'Y']

• rename_suffix (str): Additional suffix for output lin-file. Will be inserted between filename and extension. If empty, the original file is overwritten - unless they are old files, then the omc3 filename convention will be used.

  Default: None

Write MAD-X Macros

Write out madx scripts for the tracking macros.

Arguments:

--Required--

• outputdir:

  Directory where the observation_points.def will be put.

• twissfile:

  Path to twissfile with observationspoint in the NAME column.

Fake Measurement from Model

Script to generate a pseudo-measurement from a twiss-model.

The model the then generated measurement is compared to can be different from the twiss given, e.g. if the twiss incorporates errors.

Arguments:

--Required--

• twiss (PathOrStrOrDataFrame):

  Twiss dataframe or path to twiss-file.

--Optional--

• model (PathOrStrOrDataFrame):

  Alternative Model (Dataframe or Path) to use. If not given, twiss will be used.

• outputdir (PathOrStr):

  Path to the output directory for the fake measurement.

• parameters (str):

  Optics parameters to use

  choices: ('PHASEX', 'PHASEY', 'BETX', 'BETY', 'DX', 'DY', 'NDX', 'F1010', 'F1001')

  default: ['PHASEX', 'PHASEY', 'BETX', 'BETY', 'DX', 'DY', 'NDX', 'F1010', 'F1001']

• randomize (str):

  Randomize values and/or errors from gaussian distributions. If not randomized, measurement values will be equal to the model values and the errors will be equal to the relative error * measurement.

  choices: ['values', 'errors']

  default: []

• relative_errors (float):

  Relative errors. Either single value for all paramters orlist of values in order of parameters.

  default: [0.0]

• seed (int):

  Set random seed.

omc3.scripts.fake_measurement_from_model.append_model(df: DataFrame, df_model: DataFrame, parameter: str, planes: str = '', beat: bool = False) → DataFrame

Add the values to the measurement.

omc3.scripts.fake_measurement_from_model.create_beta(df_twiss: DataFrame, df_model: DataFrame, parameter: str, relative_error: float, randomize: Sequence[str], headers: Dict)

Create both beta_amp and beta_phase measurements.

omc3.scripts.fake_measurement_from_model.create_coupling(df_twiss: DataFrame, df_model: DataFrame, parameter: str, relative_error: float, randomize: Sequence[str], headers: Dict)

Creates coupling measurements for either the difference or sum RDT.

omc3.scripts.fake_measurement_from_model.create_dispersion(df_twiss: DataFrame, df_model: DataFrame, parameter: str, relative_error: float, randomize: Sequence[str], headers: Dict)

Creates dispersion measurement.

omc3.scripts.fake_measurement_from_model.create_measurement(df_twiss: DataFrame, parameter: str, relative_error: float, randomize: Sequence[str]) → TfsDataFrame

Create a new measurement Dataframe from df_twiss from parameter.

omc3.scripts.fake_measurement_from_model.create_normalized_dispersion(df_disp: DataFrame, df_beta: DataFrame, df_model: DataFrame, parameter: str, headers: Dict)

Creates normalized dispersion from pre-created dispersion and beta dataframes.

omc3.scripts.fake_measurement_from_model.create_phase(df_twiss: DataFrame, df_model: DataFrame, parameter: str, relative_error: float, randomize: Sequence[str], headers: Dict)

Creates both phase advance and total phase measurements.

omc3.scripts.fake_measurement_from_model.create_phase_advance(df_twiss: DataFrame, df_model: DataFrame, parameter: str, relative_error: float, randomize: Sequence[str], headers: Dict)

Creates phase advance measurements.

omc3.scripts.fake_measurement_from_model.create_total_phase(df_twiss: DataFrame, df_model: DataFrame, parameter: str, relative_error: float, randomize: Sequence[str], headers: Dict)

Creates total phase measurements.

omc3.scripts.fake_measurement_from_model.generate(opt) → Dict[str, TfsDataFrame]

Takes a twiss file and writes the parameters in optics_parameters to Output_dir in the format global_correction_entrypoint uses (same format you would get from hole_in_one).

Linfile Cleaning

Performs an automated cleaning of different columns in the lin-file as a standalone script to allow for manual refinement after harpy is done.

The type of cleaning is determined by the number of values in the limit parameter. When no limit is given or a single number is given, auto-cleaning is performed:

All data is assumed to be gaussian distributed around a “true” value, and outliers are cleaned by calculating average and standard deviation of the given data.

The cleaning is done by removing all data-points that are outside of the 1-0.5/n estimated percentile of the data. Where n is the number of (remaining) data-points in each loop, and the process is repeated until n stays constant (or 2 or less data-points remain).

Datapoints with a standard deviation smaller than the given limit are not cleaned. The limit is given in whatever units the data itself is in and is an absolute value.

If two values are given for the limit parameter, all data-points in between these limits are kept and all data-points outside of these limits are cleaned.

Cleaning is done per given file independently i.e. removed BPMs from one file can be present in the next. The columns are iterated on the same file, i.e. the cleaning is done consecutively on the already cleaned data from the last column, yet the moments of the distribution themselves are evaluated per column.

In the end, the lin-files are overwritten with the cleaned ones. If the backup option is activated, a backup of the original file is created, which can be restored via the restore option. In case the restore-flag is given, only the filenames are required. No cleaning is performed with this option.

Arguments:

--Required--

• files (PathOrStr):

  List of paths to the lin-files, including suffix.

--Optional--

• backup:

  Create a backup of the files before overwriting. The backups are numbered, with the highest number being the latest backup.

  action: store_true

• columns (str):

  Columns to clean on.

• keep (str):

  Do not clean BPMs with given names.

• limit (float):

  Two values: Do not clean data-points in between these values. Single value (auto-clean): Do not clean data-points deviating less than this limit from the average.

  default: 0.0

• restore:

  Restore the latest backup of the files. If this parameter is given, no cleaning is performed.

  action: store_true

TODO: also use isolation forest, BUT this probably needs some modification there as well, as it only cleans on TUNE, not on NATTUNE. And it requires an accelerator instance.

omc3.scripts.linfile_clean.clean_columns(files: Sequence[Path | str], columns: Sequence[str], limit: float | None = None, keep: Sequence[str] | None = None, backup: bool = True)

Clean the columns in the given files.

omc3.scripts.linfile_clean.main(opt)

Main function, to parse commandline input and separate restoration from cleaning.

omc3.scripts.linfile_clean.restore_files(files: Sequence[Path | str])

Restore backupped files.