Constants

Specific constants to be used in turn_by_turn, to help with consistency.

class turn_by_turn.constants.MetaDict[source]

Metadata dictionary, to type-hint known entries. None of the entries are required (total=False).

date

Date of the measurement/creation of the data

Type:

datetime

file

Path to the file the data was loaded from (if available).

Type:

Path | str

machine

Name of the machine the data was measured/simulated on.

Type:

str

source_datatype

The datatype this data was loaded from.

Type:

str

comment

Any comment on the measurement.

Type:

str

IO

This module contains high-level I/O functions to read and write turn-by-turn data objects to and from different formats.

Reading Data

Since version 0.9.0 of the package, data can be loaded either from file or from in-memory structures exclusive to certain codes (for some tracking simulation in MAD-NG or xtrack). Two different APIs are provided for these use cases.

  1. To read from file, use the read_tbt function (exported as read at the package’s level). The file format is detected or specified by the datatype parameter.

  2. To load in-memory data, use the convert_to_tbt function (exported as convert at the package’s level). This is valid for tracking simulation results from e.g. xtrack or sent back by MAD-NG.

In both cases, the returned value is a structured TbtData object.

Writing Data

The single entry point for writing to disk is the write_tbt function (exported as write at the package’s level). This writes a TbtData object to disk, typically in the LHC SDDS format (by default). The output file extension and format are determined by the datatype argument.

The following cases arise: - If datatype is set to lhc, sps or ascii, the output will be in SDDS format and the file extension will be set to .sdds if not already present. - If datatype is set to madng, the output will be in a TFS file (extension .tfs is recommended). - Other supported datatypes (see WRITERS) will use their respective formats and conventions if implemented.

The datatype parameter controls both the output format and any additional options passed to the underlying writer. Should the noise parameter be used, random noise will be added to the data before writing. A seed can be provided for reproducibility.

Example:

from turn_by_turn import write
write("output.sdds", tbt_data)  # writes in SDDS format by default
write("output.tfs", tbt_data, datatype="madng")  # writes a TFS file in MAD-NG's tracking results format
write("output.sdds", tbt_data, noise=0.01, seed=42)  # reproducibly adds noise before writing

While data can be loaded from the formats of different machines/codes (each through its own reader module), writing functionality is at the moment always done in the LHC’s SDDS format by default, unless another supported format is specified. The interface is designed to be future-proof and easy to extend for new formats.

Supported Modules and Limitations

The following table summarizes which modules support disk reading and in-memory conversion, and any important limitations:

+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| Module      | Disk Reading        | In-Memory Conversion  | Notes / Limitations                                           |
+=============+=====================+=======================+===============================================================+
| lhc         | Yes (SDDS, ASCII)   | No                    | Reads LHC SDDS and legacy ASCII files.                        |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| sps         | Yes (SDDS, ASCII)   | No                    | Reads SPS SDDS and legacy ASCII files.                        |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| doros       | Yes (HDF5)          | No                    | Reads DOROS HDF5 files.                                       |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| madng       | Yes (TFS)           | Yes                   | In-memory: only via pandas/tfs DataFrame.                     |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| xtrack      | No                  | Yes                   | In-memory via xtrack.Line + ParticlesMonitor or               |
|             |                     |                       | MultiElementMonitor.                                          |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| ptc         | Yes (trackone)      | No                    | Reads MAD-X PTC trackone files.                               |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| iota        | Yes (HDF5)          | No                    | Reads IOTA HDF5 files.                                        |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| ascii       | Yes (legacy ASCII)  | No                    | For legacy ASCII files only.                                  |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| trackone    | Yes (MAD-X)         | No                    | Reads MAD-X trackone files.                                   |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+
| superkekb   | Yes                 | No                    | Reads SuperKEKB files.                                        |
+-------------+---------------------+-----------------------+---------------------------------------------------------------+

  • In-memory converters: madng, xtrack_line.particle_monitors, xtrack_line.multi_element_monitor.

  • Most modules are for disk reading only.

  • Some modules are experimental or have limited support.

API

turn_by_turn.io.additional_args(datatype: str) → dict[str, Any][source]

Additional parameters to be added to the reader/writer function.

Parameters:

datatype (str) -- Type of the data.

turn_by_turn.io.convert_to_tbt(file_data: DataFrame | xt.Line, datatype: str = 'xtrack') → TbtData[source]

Convert in-memory simulation data to a TbtData object.

Parameters:

  • file_data (Union[DataFrame, Line]) -- The data to convert. For madng this must be a pandas/tfs DataFrame. For xtrack and xtrack_multi_element this must be an xtrack.Line.

  • datatype (str) -- In-memory converter key. One of madng, xtrack, or xtrack_multi_element. Defaults to xtrack.

Returns:

The converted object.

Return type:

TbtData

turn_by_turn.io.read_tbt(file_path: str | Path, datatype: str = 'lhc') → TbtData[source]

Calls the appropriate loader for the provided matrices type and returns a TbtData object of the loaded matrices.

Parameters:

  • file_path (Union[str, Path]) -- path to a file containing TbtData.

  • datatype (str) -- type of matrices in the file, determines the reader to use. Case-insensitive, defaults to lhc.

Returns:

A TbtData object with the loaded matrices.

turn_by_turn.io.write_tbt(output_path: str | Path, tbt_data: TbtData, noise: float | None = None, seed: int | None = None, datatype: str = 'lhc') → None[source]

Write a TbtData object’s data to file, in the LHC’s SDDS format.

Parameters:

  • output_path (Union[str, Path]) -- path to a the disk location where to write the data.

  • tbt_data (TbtData) -- the TbtData object to write to disk.

  • noise (float) -- optional noise to add to the data.

  • seed (int) -- A given seed to initialise the RNG if one chooses to add noise. This is useful to ensure the exact same RNG state across operations. Defaults to None, which means any new RNG operation in noise addition will pull fresh entropy from the OS.

  • datatype (str) -- type of matrices in the file, determines the reader to use. Case-insensitive, defaults to lhc.

Structures

Data structures to be used in turn_by_turn to store turn-by-turn measurement data.

class turn_by_turn.structures.TbtData(matrices: Sequence[DataType], nturns: int, bunch_ids: list[int] | None = None, meta: MetaDict = <factory>)[source]

Object holding a representation of a Turn-by-Turn data measurement. The date of the measurement, the transverse data, number of turns and bunches as well as the bunch IDs are encapsulated in this object.

matrices

Sequence of TransverseData or TrackingData objects. Each entry corresponds to a “bunch” or “particle” (tracking).

Type:

Sequence[DataType]

nturns

Number of turns. Needs to be a positive integer. It is assumed all bunches (and observation points in all entries therein) have the same length in the turn-dimension (columns).

Type:

int

bunch_ids

List of bunch/particle IDs. Will default to [0, 1, ..., nbunches-1] if not set.

Type:

list[int] | None

meta

Dictionary of metadata.

Type:

MetaDict

nbunches

Number of bunches/particles. Automatically set (i.e. cannot be set in the initialization of this object).

Type:

int

class turn_by_turn.structures.TrackingData(X: pd.DataFrame, PX: pd.DataFrame, Y: pd.DataFrame, PY: pd.DataFrame, T: pd.DataFrame, PT: pd.DataFrame, S: pd.DataFrame, E: pd.DataFrame)[source]

Object holding multidimensional turn-by-turn simulation data in the form of pandas DataFrames.

The DataFrames should be N_observation-points x M_turns matrices, and usually contain the BPM/observation-point names as index, while the columns are simply numbered starting from 0. All DataFrames should have the same N x M size.

X

Horizontal position data

Type:

pd.DataFrame

PX

Horizontal momentum data

Type:

pd.DataFrame

Y

Vertical position data

Type:

pd.DataFrame

PY

Vertical momentum data

Type:

pd.DataFrame

T

Negative time difference wrt. the reference particle (multiplied by c)

Type:

pd.DataFrame

PT

Energy difference wrt. the reference particle divided by the ref. momentum (multiplied by c)

Type:

pd.DataFrame

S

Longitudinal position data

Type:

pd.DataFrame

E

Energy data

Type:

pd.DataFrame

classmethod fieldnames() → list[str][source]

Return a list of the fields of this dataclass.

class turn_by_turn.structures.TransverseData(X: pd.DataFrame, Y: pd.DataFrame)[source]

Object holding measured turn-by-turn data for both transverse planes in the form of pandas DataFrames.

The DataFrames should be N_(observation-points) x M_(turns) matrices, and usually contain the BPM/observation-point names as index, while the columns are simply numbered starting from 0. All DataFrames should have the same N x M size.

X

Horizontal position data

Type:

pd.DataFrame

Y

Vertical position data

Type:

pd.DataFrame

classmethod fieldnames() → list[str][source]

Return a list of the fields of this dataclass.

Utils

Utility functions for convenience operations on turn-by-turn data objects in this package.

turn_by_turn.utils.add_noise(data: ndarray, noise: float | None = None, sigma: float | None = None, seed: int | Generator | None = None) → ndarray[source]

Returns the given data with added noise. Noise is generated as a standard normal distribution (mean=0, standard deviation=1) with the size of the input data, and scaled by a factor before being added to the provided data. Said factor can either be provided, or calculated from the input data’s own standard deviation.

Parameters:

  • data (np.ndarray) -- your input data.

  • noise (float | None) -- the scaling factor applied to the generated noise.

  • sigma (float | None) -- if provided, then that number times the standard deviation of the input data will be used as scaling factor for the generated noise.

  • seed (int | np.random.Generator | None) -- Seed for the random number generator. Can be an integer or an np.random.Generator instance.

Returns:

A new numpy array with added noise to the provided data.

turn_by_turn.utils.add_noise_to_tbt(data: TbtData, noise: float | None = None, sigma: float | None = None, seed: int | Generator | None = None) → TbtData[source]

Returns a new copy of the given TbT data with added noise. The noise is generated by turn_by_turn.utils.add_noise() from a single rng, i.e. the noise is not repeated on each dataframe.

Parameters:

  • data (TbtData) -- your input TbT-data.

  • noise (float | None) -- the scaling factor applied to the generated noise.

  • sigma (float | None) -- if provided, then that number times the standard deviation of the input data will be used as scaling factor for the generated noise.

  • seed (int | np.random.Generator | None) -- Seed for the random number generator. Can be an integer or an np.random.Generator instance.

Returns:

A copy of the TbtData with noised data on all matrices.

turn_by_turn.utils.all_elements_equal(iterable: Iterable) → bool[source]

Check if all elements in an iterable are equal. WARNING: Does not necessarily work with floating point numbers.

Parameters:

iterable (Iterable) -- an iterable to check.

Returns:

True if all elements are equal, False otherwise.

Return type:

bool

turn_by_turn.utils.generate_average_tbtdata(tbtdata: TbtData) → TbtData[source]

Takes a TbtData object and returns another containing the averaged matrices over all bunches/particles at all used BPMs.

Parameters:

tbtdata (TbtData) -- input TbtData object from measurements.

Returns:

A new TbtData object with the averaged matrices.

turn_by_turn.utils.get_averaged_data(bpm_names: Sequence[str], matrices: Sequence[TransverseData], plane: str, turns: int) → np.ndarray[source]

Average data from a given plane from the matrices of a TbtData.

Parameters:

  • bpm_names (Sequence[str]) -- Names of the BPMs to average data for.

  • matrices (Sequence[TransverseData]) -- matrices from a TbtData object.

  • plane (str) -- name of the given plane to average in.

  • turns (int) -- number of turns in the provided data.

Returns:

A numpy array with the averaged data for the given bpms.

turn_by_turn.utils.matrices_to_array(tbt_data: TbtData) → ndarray[source]

Convert the matrices of a TbtData object to a numpy array.

Parameters:

tbt_data (TbtData) -- TbtData object to convert the data from.

Returns:

A numpy array with shape (2, n_bpms, nbunches, nturns) containing the matrices data, where the first dimension corresponds to planes (X=0, Y=1).

turn_by_turn.utils.numpy_to_tbt(names: ndarray, matrix: ndarray, datatype: TransverseData | TrackingData = <class 'turn_by_turn.structures.TransverseData'>) → TbtData[source]

Converts turn by turn matrices and names into a TbTData object.

Parameters:

  • names (np.ndarray) -- Numpy array of BPM names.

  • matrix (np.ndarray) -- 4D Numpy array [quantity, BPM, particle/bunch No., turn No.] quantities in order [x, y].

  • datatype (DataType) -- The type of data to be converted to in the matrices. Either TransverseData (which implies reading X and Y fields) or TrackingData (which implies reading all 8 fields). Defaults to TransverseData.

Returns:

A TbtData object loaded with the matrices in the provided numpy arrays.