TFS Files

Main TFS File Handler

Module tfs_files.tfs_pandas

Pandas wrapper to read and write TFS-files.

class tfs_files.tfs_pandas.TfsDataFrame(*args, **kwargs)

Class to hold the information of the built Pandas DataFrame, together with a way of getting the headers of the TFS file. To get a header value do: data_frame[“header_name”] or data_frame.header_name.

exception tfs_files.tfs_pandas.TfsFormatError

Raised when wrong format is detected in the TFS file.

class tfs_files.tfs_pandas.TypeToIdConverter

For symmetry reasons.

tfs_files.tfs_pandas.read_tfs(tfs_path, index=None)

Parses the TFS table present in tfs_path and returns a custom Pandas DataFrame (TfsDataFrame). :param tfs_path: Input filepath :param index: Name of the column to set as index. If not given looks for INDEX_ID-column :return: TFS_DataFrame object

tfs_files.tfs_pandas.write_tfs(tfs_path, data_frame, headers_dict={}, save_index=False)

Writes the Pandas DataFrame data_frame into tfs_path with the headers_dict as headers dictionary. If you want to keep the order of the headers, use collections.OrderedDict. :param tfs_path: Output filepath :param data_frame: Data Frame to save :param headers_dict: Headers of the dataframe, if empty tries to use data_frame.headers :param save_index: bool or string. If True, saves the index of the data_frame to a column identifiable by INDEX_ID (will be loaded automatically by read_tfs). If string, it saves the index of the data_frame to a column named like the string given. Default: False

TFS Helpers

Module tfs_files.tfs_collection

Easy access to tfs-file contents via tfs_pandas.

class tfs_files.tfs_collection.Tfs(*args, **kwargs)

Class to mark attributes as Tfs attributes.

Any parameter given to this class will be passed to the “get_filename()” and “write_to()” methods, together with the plane if “two_planes=False” is not present.

class tfs_files.tfs_collection.TfsCollection(directory, allow_write=False)

Abstract class to lazily load and write TFS files.

The classes that inherit from this abstract class will be able to define TFS files as readable or writable and read or write them just as attribute access or assignments. All attributes will be read and write as Pandas DataFrames.

Example

If “./example” is a directory that contains two TFS file “getbetax.out” and “getbetax.out” with BETX and BETY columns respectively:

>>> class ExampleCollection(TfsCollection)
>>>
>>>    # All TFS attributes must be marked with the Tfs(...) class:
>>>    beta = Tfs("getbeta{}.out")
>>>    # This is a traditional attribute.
>>>    other_value = 7
>>>
>>>    def get_filename(template, plane):
>>>       return template.format(plane)
>>>
>>> example = ExampleCollection("./example")
>>> # Get the BETX column from "getbetax.out":
>>> beta_x_column = example.beta_x.BETX
>>> # Get the BETY column from "getbetay.out":
>>> beta_y_column = example.beta_y.BETY
>>> # The planes can also be accessed as items:
>>> beta_y_column = example.beta["y"].BETY
>>> # This will write an empty DataFrame to "getbetay.out":
>>> example.allow_write = True
>>> example.beta["y"] = DataFrame()

If the file to be loaded is not defined for two planes it can be declared as: coupling = Tfs(“getcouple.out”, two_planes=False) and then accessed as f1001w_column = example.coupling.F1001W. No file will be loaded until the corresponding attribute is accessed and the loaded DataFrame will be buffered, thus the user should expect an IOError if the requested file is not in the provided directory (only the first time but is better to always take it into account!). When a DataFrame is assigned to one attribute it will be set as the buffer value. If the self.allow_write attribute is set to true, an assignment on one of the attributes will trigger the corresponding file write.

clear()

Clear the file buffer.

Any subsequent attribute access will try to load the corresponding file again.

get_filename(*args, **kwargs)

Returns the filename to be loaded or written.

This function will get as parameters any parameter given to the Tfs(…) attributes. It must return the filename to be written according to those parameters. If “two_planes=False” is not present in the Tfs(…) definition, it will also be given the keyword argument plane=”x” or “y”.

read_tfs(filename)

Actually reads the TFS file from self.directory with filename.

This function can be ovewriten to use something instead of tfs_pandas to load the files.

Parameters:filename -- The name of the file to load. Returns:A tfs_pandas instance of the requested file.

write_to(*args, **kwargs)

Returns the filename and DataFrame to be written on assignments.

If this function is overwrote, it will replace get_filename(…) in file writes to find out the filename of the file to be written. It also gets the value assigned as first parameter. It must return a tuple (filename, data_frame).

Module tfs_files.tfs_file_writer

This module contains the class TfsFileWriter which is used to create easily TfsFiles.

Usage1:

import tfs_files.tfs_file_writer as tfs_writer
...
tfs_file_writer = tfs_writer.TfsFileWriter.open("my_file.out")
tfs_file_writer.set_column_width(15) # Specify desired column width
tfs_file_writer.set_outputpath("/x/y/z/") # Specify a directory if desired

tfs_file_writer.add_string_descriptor("NAME", "TWISS")
tfs_file_writer.add_float_descriptor("MASS", 0.938272013)
tfs_file_writer.add_comment("I am a comment")
tfs_file_writer.add_column_names("NAME S BETX ALFX BETY ALFY".split())
tfs_file_writer.add_column_datatypes("%s %le %le %le %le %le".split())
tfs_file_writer.add_table_row("BTVSS.6L2.B1  1.125  131.58734  -1.8991  67.6178 1.6995".split())

tfs_file_writer.write_to_file()

Usage2:

tfs_file_writer = tfs_writer.TfsFileWriter.open("/x/y/z/my_file.out")
#tfs_file_writer.set_outputpath("/x/y/z/") # Don't do this! Outputpath is already in file_name.

... # Add at least column_names, dolumn_datatypes and one table_row
tfs_file_writer.write_to_file()

class tfs_files.tfs_file_writer.TfsFileWriter(file_name, outputpath=None, column_width=20)

This class represents a TFS file. It stores all header lines and the table and write all the content formatted at once by calling the write function.

add_column_datatypes(list_datatypes)

Adds the list of column data types to the table header. If the number of columns is determined already(e.g. by adding column names) and the length of list_datatypes does not match the number of columns a TypeError will be raised.

Parameters:list_datatypes (list) -- Containing the data type(%s, %le) of the columns. Without prefix ‘$’

add_column_names(list_names)

Adds the list of column names to the table header. If the number of columns is determined already(e.g. by adding column data types) and the length of list_names does not match the number of columns a TypeError will be raised.

Parameters:list_names (list) -- Containing the names of the columns. Without prefix ‘*’

add_comment(comment)

Adds the string “# <comment>” to the tfs header.

add_float_descriptor(name, float_value)

Adds the string “@ <name> %le <data>” to the tfs header.

add_header_line(str_value)

Adds the line to the tfs header.

add_int_descriptor(name, int_value)

Adds the string “@ <name> %d <data>” to the tfs header.

add_string_descriptor(name, str_value)

Adds the string “@ <name> %s <data>” to the tfs header.

add_table_row(list_row_entries)

Adds the entries of one row to the table data.

Parameters:list_row_entries (list) -- Values of one row. Datatypes will not be checked. Only length will be checked with the length of column names.

static open(file_name)

This function will create and return a TfsFileWriter object with the given filename. No file will be opened on the file system yet. An actual file will first be created after adding at least column_names, dolumn_datatypes and one table_row and the method call write_to_file().

order_rows(column_name, reverse=False)

Orders the rows according to one of the column names.

write_to_file(formatted=True)

Writes the stored data to the file with the given filename.

Module tfs_files.tfs_utils

Helperfunctions to handle TFS-files easier.

tfs_files.tfs_utils.remove_header_comments_from_files(list_of_files)

Check the files in list for invalid headers (no type defined) and removes them.

tfs_files.tfs_utils.remove_nan_from_files(list_of_files, replace=False)

Remove NAN-Entries from files in list_of_files.

If replace=False a new file with .dropna in it’s name is created, otherwise the file is overwritten.

Module tfs_files.diff_files

Functions to get the difference between two tfs files or dataframes. This is very general, i.e. not as results oriented as getdiff.py.

tfs_files.diff_files.get_diff_two_dataframes(df1, df2, diff_columns, is_error=None, prefix='', index=None, keep_colums=(), out_file=None)

Get the difference of common elements of specific columns between two dataframes.

Merges on index.

Args:

df1 (DataFrame or Path): First dataframe, Minuend df2 (DataFrame or Path): Second dataframe, Subtrahend diff_columns (list of stings): List of columns to get the difference of is_error (list of booleans): defines if the column in question is an error column prefix (str): Prefix for difference columns (default: “”) index (str): index column - most likely needed when reading/writing files keep_colums (list of strings): additional columns to keep in the returned dataframe out_file (Path): if given, writes the result into this file

Returns:

DataFrame containing difference columns and kept columns.