Generic-Parser Modules

Entrypoint Parser

This module contains high-level objects to manage the functionality of generic_parser: entry point parser and decorator.

These allow a function to be decorated as entrypoint. This function will then automatically accept console arguments, config files, json files, kwargs and dictionaries as input and will parse it according to the parameters given to the entrypoint-Decorator.

Terminology:

• Parameter - Items containing info on how to parse Arguments

• Arguments - The input to the wrapped-function

• Options - The parsed arguments and hence the options of the function

Hence, an ArgumentError will be raised in case of something going wrong during parsing, while ParameterErrors will be raised if something goes wrong when adding parameters to the list.

Usage:

To be used as a decorator:

@entrypoint(parameters)
def some_function(options, unknown_options)

Using strict mode (see below):

@entrypoint(parameters, strict=True)
def some_function(options)

It is also possible to use the EntryPoint Class similar to a normal parser:

ep_parser = EntryPoint(parameters)
options, unknown_options = ep_parser.parse(arguments)

Using strict mode (see below):

ep_parser = EntryPoint(parameters, strict=True)
options = ep_parser.parse(arguments)

Parameters:

Parameters need to be a list or a dictionary of dictionaries with the following keys:

name (required): Name of the variable (e.g. later use options.NAME). If ‘params’ is a dictionary, the key will be used as name.

flags (required): Commandline flag(s), e.g. --file

required (optional): bool

default (optional): Default value, if variable not present

help (optional): str

type (optional): Value type (if nargs is given, set to list for dicts!)

choices (optional): choices to choose from (choices need to be of type, if given)

nargs (optional): number of arguments to consume (commandline only, do not use REMAINDER!)

action (optional): either store_true or store_false, will set type to bool and the default to False and True respectively.

Alternatively, you can use the provided EntryPointParameters() class.

Example with EntryPointParameters:

args = EntryPointParameters()
args.add_parameter(name="accel",
                   flags=["-a", "--accel"],
                   help="Which accelerator?",
                   choices=["LHCB1","LHCB2","LHCB5"],
                   default="LHCB1")
args.add_parameter(name="dict",
                   flags=["-d", "--dictionary"],
                   help="File with the BPM dictionary",
                   default="bpm.txt",
                   type=str)

Example with dictionary of dictionaries:

args = EntryPointParameters({
    "accel": dict(
        flags=["-a", "--accel"],
        help="Which accelerator?",
        choices=["LHCB1", "LHCB2", "LHCB5"],
        default="LHCB1"),
    "dict": dict(
        flags=["-d", "--dictionary"],
        help="File with the BPM dictionary",
        default="bpm.txt",
        type=str),
        })

Example with list of dictionaries:

args = [
    dict(
        name="accel",
        flags=["-a", "--accel"],
        help="Which accelerator?",
        choices=["LHCB1", "LHCB2", "LHCB5"],
        default="LHCB1"),
    dict(
        name="dict",
        flags=["-d", "--dictionary"],
        help="File with the BPM dictionary",
        default="bpm.txt",
        type=str),
        ]

The strict option changes the behaviour for unknown parameters: strict=True raises exceptions, strict=False logs debug messages and returns the options. Hence a wrapped function with strict=True must accept one input, with strict=False two. Defaults to False

Arguments:

Arguments to a decorated function can be passed in multiple ways. Let’s take the decorated example function:

@entrypoint(EntryPointParameters({"foo": {}, "bar": {}}), strict=True)
def entry_function(opt):
    print(opt)

Then the different ways to call it are, from python:

entry_function(foo='mark', bar='twain')  # keyword args

entry_function(dict(foo='mark', bar='twain'))  # dictionary
entry_function(entry_dict=dict(foo='mark', bar='twain'))  # dictionary

entry_function(['--foo', 'mark', '--bar', 'twain'])  # commandline args

entry_function(entry_cfg=Path('config.ini'))  # config file
entry_function(entry_cfg=Path('config.ini'), section='section')  # '[section]' in config file


entry_function(entry_json=Path('config.json'))  # json file
entry_function(entry_json=Path('config.json'), section='section')  # '[section]' in json file

or as commandline arguments from a script.py that calls entry_function():

python script.py --foo mark --bar twain
python script.py --entry_cfg config.ini
python script.py --entry_cfg config.ini --section section

class generic_parser.entrypoint_parser.EntryPointParameters(*args, **kwargs)

Helps to build a simple dictionary structure via add_argument functions. You really don’t need that, but old habits die hard.

add_parameter(**kwargs)

Add parameter.

help()

Prints current help. Usable to paste into docstrings.

exception generic_parser.entrypoint_parser.OptionsError

generic_parser.entrypoint_parser.add_params_to_generic(parser, params)

Adds entry-point style parameter to either ArgumentParser, DictParser or EntryPointParameters.

generic_parser.entrypoint_parser.add_to_arguments(args, entry_params=None, **kwargs)

Adds arguments to an existing list or dictionary of arguments. If args is a list, the flags of the names given will be added and entry_params is required.

Parameters:

• args (list,dict) -- Arguments (e.g. from unknown).

• entry_params (list, dict) -- Parameter belonging to the arguments.

Keyword Arguments:

add. (Name and value of the arguments to) --

generic_parser.entrypoint_parser.create_parameter_help(module, param_fun=None)

Print params help quickly but changing the logging format first.

Usage Example::

import amplitude_detuning_analysis create_parameter_help(amplitude_detuning_analysis) create_parameter_help(amplitude_detuning_analysis, “_get_plot_params”)

generic_parser.entrypoint_parser.dict2list_param(param)

Convert dictionary to list and add name by key.

class generic_parser.entrypoint_parser.entrypoint(parameter, strict: bool = False, argument_parser_args: Optional[Mapping] = None, help_printer: Optional[Callable] = None)

Decorator extension of EntryPoint. Implements the __call__ method needed for decorating. Lowercase looks nicer if used as decorator.

generic_parser.entrypoint_parser.list2dict_param(param)

Convert list to dictionary for quicker find.

generic_parser.entrypoint_parser.param_names(params)

Get the names of the parameters, no matter if they are a dict or list of dicts.

generic_parser.entrypoint_parser.save_options_to_config(filepath, opt, unknown=None)

Saves the parsed options to a config file to be used as arguments.

Parameters:

• filepath -- path to write the config file to.

• opt -- parsed known options.

• unknown -- unknown options (only safe for non-commandline parameters).

generic_parser.entrypoint_parser.split_arguments(args, *param_list)

Divide remaining arguments into a list of argument-dicts, fitting to the params in param_list.

Parameters:

• args -- Input arguments, either as list of strings or dict.

• param_list -- list of sets of entry-point parameters (either dict, or list).

Returns:

A list of dictionaries containing the arguments for each set of parameters, plus one more entry for unknown parameters. If the input was a list of argument-strings, the parameters will already be parsed.

Warning

Unless you know what you are doing, run this function only on remaining-arguments from entry point parsing, not on the actual arguments

Warning

Adds each argument only once, to the set of params who claim it first!

Dictionary Parser

This module holds classes to handle different dictionaries as argument containers.

exception generic_parser.dict_parser.ArgumentError

class generic_parser.dict_parser.DictParser(dictionary=None, strict=False)

Provides functions to parse a dictionary.

First, build a dictionary structure with Parameters or Parameter-like dicts as leafs via add_parameter or on init. A similar structured option dictionary with the values as leafs can then be parsed.

add_parameter(param, **kwargs)

Adds an parameter to the parser. If you want it to be an parameter of a sub-dictionary add the ‘loc=subdict.subdict’ keyword to the input.

Parameters:

• param -- Argument to add (either of object of class argument or string defining the name).

• kwargs -- Any of the argument-fields (apart from ‘name’) and/or ‘loc’.

Returns:

This object.

add_parameter_dict(dictionary, loc)

Appends a complete subdictionary to existing argument structure at node ‘loc’.

Parameters:

• loc -- location of the node to append the sub-dictionary.

• dictionary -- The dictionary to append.

Returns:

This object.

parse_arguments(arguments)

Parse a given argument dictionary and return parsed options.

Parameters:

arguments -- Arguments to parse.

Returns:

Options [, Unknown Options].

parse_config_items(items)

Parse a list of (name, value) items, where the values are all strings.

Parameters:

items -- list of (name, value) items.

Returns:

Parsed options.

tree()

Prints the current Parameter-Tree.

class generic_parser.dict_parser.Parameter(name, **kwargs)

Helper Class for DictParser.

exception generic_parser.dict_parser.ParameterError

Entry Datatypes

This module contains advanced datatypes to add as type to any entrypoint or parser.

class generic_parser.entry_datatypes.BoolOrList(value)

A class that behaves like a boolean when possible, otherwise like a list. Hint: list.__new__(list, value) returns an empty list.

class generic_parser.entry_datatypes.BoolOrString(value)

A class that behaves like a boolean when possible, otherwise like a string.

class generic_parser.entry_datatypes.DictAsString(s)

Use dicts in command line like {“key”:value}.

generic_parser.entry_datatypes.get_instance_faker_meta(*classes)

Returns the metaclass that fakes the isinstance() and issubclass() checks.

generic_parser.entry_datatypes.get_multi_class(*classes)

Create a class ‘behaving’ like all classes in classes.

In case a value needs to be converted to a class in this list, it is attempted to cast the input to the classes in the given order (i.e. string-classes need to go to the end, as they ‘always’ succeed).

Utility Tools

Provides utilities to use in other modules.

class generic_parser.tools.DotDict(*args, **kwargs)

Make dict fields accessible by dot notation.

get_subdict(keys, strict=True)

See get_subdict.

class generic_parser.tools.TempStringLogger(module='', level=20)

Temporarily log into a string that can be retrieved by get_log.

Parameters:

• module -- module to log, defaults to the caller file.

• level -- logging level, defaults to INFO.

get_log()

Get the log as string.

generic_parser.tools.get_subdict(full_dict, keys, strict=True)

Returns a sub-dictionary of full_dict containing only keys of keys.

Parameters:

• full_dict -- Dictionary to extract from.

• keys -- keys to extract.

• strict -- If false it ignores keys not in full_dict. Otherwise it crashes on those. Defaults to True.

Returns:

Extracted sub-dictionary.

generic_parser.tools.log_out(stdout=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>, stderr=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>)

Temporarily changes sys.stdout and sys.stderr.

generic_parser.tools.print_dict_tree(dictionary, name='Dictionary', print_fun=<bound method Logger.info of <Logger generic_parser.tools (WARNING)>>)

Prints a dictionary as a tree.

generic_parser.tools.silence()

Suppress all console output. sys.stdout and sys.stderr are rerouted to devnull.

generic_parser.tools.unformatted_console_logging()

Log only to console and only unformatted.