Constants

Specific constants to be used in turn_by_turn, to help with consistency.

IO

This module contains high-level I/O functions to read and write turn-by-turn data objects to and from different formats.

Reading Data

Since version 0.9.0 of the package, data can be loaded either from file or from in-memory structures exclusive to certain codes (for some tracking simulation in MAD-NG or xtrack). Two different APIs are provided for these use cases.

1. To read from file, use the read_tbt function (exported as read at the package’s level). The file format is detected or specified by the datatype parameter.

2. To load in-memory data, use the convert_to_tbt function (exported as convert at the package’s level). This is valid for tracking simulation results from e.g. xtrack or sent back by MAD-NG.

In both cases, the returned value is a structured TbtData object.

Writing Data

The single entry point for writing to disk is the write_tbt function (exported as write at the package’s level). This writes a TbtData object to disk, typically in the LHC SDDS format (by default). The output file extension and format are determined by the datatype argument.

The following cases arise: - If datatype is set to lhc, sps or ascii, the output will be in SDDS format and the file extension will be set to .sdds if not already present. - If datatype is set to madng, the output will be in a TFS file (extension .tfs is recommended). - Other supported datatypes (see WRITERS) will use their respective formats and conventions if implemented.

The datatype parameter controls both the output format and any additional options passed to the underlying writer. Should the noise parameter be used, random noise will be added to the data before writing. A seed can be provided for reproducibility.

Example:

from turn_by_turn import write
write("output.sdds", tbt_data)  # writes in SDDS format by default
write("output.tfs", tbt_data, datatype="madng")  # writes a TFS file in MAD-NG's tracking results format
write("output.sdds", tbt_data, noise=0.01, seed=42)  # reproducibly adds noise before writing

While data can be loaded from the formats of different machines/codes (each through its own reader module), writing functionality is at the moment always done in the LHC’s SDDS format by default, unless another supported format is specified. The interface is designed to be future-proof and easy to extend for new formats.

Supported Modules and Limitations

The following table summarizes which modules support disk reading and in-memory conversion, and any important limitations:

Module	Disk Reading	In-Memory Conversion	Notes / Limitations
lhc	Yes (SDDS, ASCII)	No	Reads LHC SDDS and legacy ASCII files.
sps	Yes (SDDS, ASCII)	No	Reads SPS SDDS and legacy ASCII files.
doros	Yes (HDF5)	No	Reads DOROS HDF5 files.
madng	Yes (TFS)	Yes	In-memory: only via pandas/tfs DataFrame.
xtrack	No	Yes	Only in-memory via xtrack.Line.
ptc	Yes (trackone)	No	Reads MAD-X PTC trackone files.
esrf	Yes (Matlab .mat)	No	Experimental/untested.
iota	Yes (HDF5)	No	Reads IOTA HDF5 files.
ascii	Yes (legacy ASCII)	No	For legacy ASCII files only.
trackone	Yes (MAD-X)	No	Reads MAD-X trackone files.

• Only madng and xtrack support in-memory conversion.

• Most modules are for disk reading only.

• Some modules (e.g., esrf) are experimental or have limited support.

API

turn_by_turn.io.additional_args(datatype: str) → dict[str, Any]

Additional parameters to be added to the reader/writer function.

Parameters:

datatype (str) -- Type of the data.

turn_by_turn.io.convert_to_tbt(file_data: DataFrame | Line, datatype: str = 'xtrack') → TbtData

Convert a pandas or tfs DataFrame (MAD-NG) or a Line (XTrack) to a TbtData object. :param file_data: The data to convert. :type file_data: Union[DataFrame, xt.Line] :param datatype: The type of the data, either ‘xtrack’ or ‘madng’. Defaults to ‘xtrack’. :type datatype: str

Returns:

The converted TbtData object.

Return type:

TbtData

turn_by_turn.io.read_tbt(file_path: str | Path, datatype: str = 'lhc') → TbtData

Calls the appropriate loader for the provided matrices type and returns a TbtData object of the loaded matrices.

Parameters:

• file_path (Union[str, Path]) -- path to a file containing TbtData.

• datatype (str) -- type of matrices in the file, determines the reader to use. Case-insensitive, defaults to lhc.

Returns:

A TbtData object with the loaded matrices.

turn_by_turn.io.write_tbt(output_path: str | Path, tbt_data: TbtData, noise: float = None, seed: int = None, datatype: str = 'lhc') → None

Write a TbtData object’s data to file, in the LHC’s SDDS format.

Parameters:

• output_path (Union[str, Path]) -- path to a the disk location where to write the data.

• tbt_data (TbtData) -- the TbtData object to write to disk.

• noise (float) -- optional noise to add to the data.

• seed (int) -- A given seed to initialise the RNG if one chooses to add noise. This is useful to ensure the exact same RNG state across operations. Defaults to None, which means any new RNG operation in noise addition will pull fresh entropy from the OS.

• datatype (str) -- type of matrices in the file, determines the reader to use. Case-insensitive, defaults to lhc.

Structures

Data structures to be used in turn_by_turn to store turn-by-turn measurement data.

class turn_by_turn.structures.TbtData(matrices: Sequence[DataType], date: datetime = None, bunch_ids: list[int] = None, nturns: int = None)

Object holding a representation of a Turn-by-Turn data measurement. The date of the measurement, the transverse data, number of turns and bunches as well as the bunch IDs are encapsulated in this object.

class turn_by_turn.structures.TrackingData(X: pd.DataFrame, PX: pd.DataFrame, Y: pd.DataFrame, PY: pd.DataFrame, T: pd.DataFrame, PT: pd.DataFrame, S: pd.DataFrame, E: pd.DataFrame)

Object holding multidimensional turn-by-turn simulation data in the form of pandas DataFrames.

classmethod fieldnames() → list[str]

Return a list of the fields of this dataclass.

class turn_by_turn.structures.TransverseData(X: pd.DataFrame, Y: pd.DataFrame)

Object holding measured turn-by-turn data for both transverse planes in the form of pandas DataFrames.

classmethod fieldnames() → list[str]

Return a list of the fields of this dataclass.

Utils

Utility functions for convenience operations on turn-by-turn data objects in this package.

turn_by_turn.utils.add_noise(data: ndarray, noise: float = None, sigma: float = None, seed: int = None) → ndarray

Returns the given data with added noise. Noise is generated as a standard normal distribution (mean=0, standard_deviation=1) with the size of the input data, and scaled by the a factor before being added to the provided data. Said factor can either be provided, or calculated from the input data’s own standard deviation.

Parameters:

• data (np.ndarray) -- your input data.

• noise (float) -- the scaling factor applied to the generated noise.

• sigma (float) -- if provided, then that number times the standard deviation of the input data will be used as scaling factor for the generated noise.

• seed (int) -- a given seed to initialise the RNG.

Returns:

A new numpy array with added noise to the provided data.

turn_by_turn.utils.add_noise_to_tbt(data: TbtData, noise: float = None, sigma: float = None, seed: int = None) → TbtData

Returns a new copy of the given TbT data with added noise. The noise is generated by turn_by_turn.utils.add_noise() from a single rng, i.e. the noise is not repeated on each dataframe.

Parameters:

• data (TbtData) -- your input TbT-data.

• noise (float) -- the scaling factor applied to the generated noise.

• sigma (float) -- if provided, then that number times the standard deviation of the input data will be used as scaling factor for the generated noise.

• seed (int) -- a given seed to initialise the RNG.

Returns:

A copy of the TbtData with noised data on all matrices.

turn_by_turn.utils.all_elements_equal(iterable: Iterable) → bool

Check if all elements in an iterable are equal. WARNING: Does not necissarily work with floating point numbers.

Parameters:

iterable (Iterable) -- an iterable to check.

Returns:

True if all elements are equal, False otherwise.

Return type:

bool

turn_by_turn.utils.generate_average_tbtdata(tbtdata: TbtData) → TbtData

Takes a TbtData object and returns another containing the averaged matrices over all bunches/particles at all used BPMs.

Parameters:

tbtdata (TbtData) -- entry TbtData object from measurements.

Returns:

A new TbtData object with the averaged matrices.

turn_by_turn.utils.get_averaged_data(bpm_names: Sequence[str], matrices: Sequence[TransverseData], plane: str, turns: int) → np.ndarray

Average data from a given plane from the matrices of a TbtData.

Parameters:

• bpm_names (Sequence[str])

• matrices (Sequence[TransverseData]) -- matrices from a TbtData object.

• plane (str) -- name of the given plane to average in.

• turns (int) -- number of turns in the provided data.

Returns:

A numpy array with the averaged data for the given bpms.

turn_by_turn.utils.matrices_to_array(tbt_data: TbtData) → ndarray

Convert the matrices of a TbtData object to a numpy array.

Parameters:

tbt_data (TbtData) -- TbtData object to convert the data from.

Returns:

A numpy array with the matrices data.

turn_by_turn.utils.numpy_to_tbt(names: ~numpy.ndarray, matrix: ~numpy.ndarray, datatype: ~turn_by_turn.structures.TransverseData | ~turn_by_turn.structures.TrackingData = <class 'turn_by_turn.structures.TransverseData'>) → TbtData

Converts turn by turn matrices and names into a TbTData object.

Parameters:

• names (np.ndarray) -- Numpy array of BPM names.

• matrix (np.ndarray) -- 4D Numpy array [quantity, BPM, particle/bunch No., turn No.] quantities in order [x, y].

• datatype (DataType) -- The type of data to be converted to in the matrices. Either TransverseData (which implies reading X and Y fields) or TrackingData (which implies reading all 8 fields). Defaults to TransverseData.

Returns:

A TbtData object loaded with the matrices in the provided numpy arrays.