mangadap.par.parset module

Parameter sets are used to control the behavior of internal methods from top-level input. Parameter sets:

provide I/O methods for inspection, documentation, and logging

force parameters to have specific data types, including if the parameter should be callable

specify allowed options for method keywords

provide parameter descriptions

Beyond this and once instantiated, though, they are similar to a normal python dictionary.

ParSet base class

As an example, we can create a bogus parameter set as follows:

from mangadap.par.parset import ParSet
pars = ['test', 'par', 'lst', 'junk']
defaults = ['this', 0, 0.0, None]
options = [['this', 'that'], None, None, None]
dtypes = [str, int, [int, float], None]
descr = ['test parameter description',
         'par parameter description',
         None,
         'this is a rather long description of the junk parameter.  You can include ' \
            'rst-style references like pointing back to the ' \
            ':class:`~mangadap.par.parset.ParSet` class, for when this description is ' \
            'written to an rst table using :func:`~mangadap.par.parset.ParSet.to_rst_table` ' \
            'and included in an rst doc synthesized into html using sphinx.']
p = ParSet(pars, defaults=defaults, options=options, dtypes=dtypes, descr=descr)

Once defined, accessing/showing the values is done similarly to a dictionary, with some convenience printing methods:

>>> p['test']
'that'
>>> p['lst']
0.0
>>> p['junk'] is None
True
>>> p
Parameter  Value  Default  Type        Callable
-----------------------------------------------
test       that   this     str         False
par        3      0        int         False
lst        0.0    0.0      int, float  False
junk       None   None     Undefined   False
>>> p.info()
test
        Value: this
      Default: this
      Options: this, that
  Valid Types: str
     Callable: False
  Description: test parameter description

par
        Value: 0
      Default: 0
      Options: None
  Valid Types: int
     Callable: False
  Description: par parameter description

lst
        Value: 0.0
      Default: 0.0
      Options: None
  Valid Types: int, float
     Callable: False
  Description: None

junk
        Value: None
      Default: None
      Options: None
  Valid Types: Undefined
     Callable: False
  Description: this is a rather long description of the junk parameter.
               You can include rst-style references like pointing back
               to the :class:`~mangadap.par.parset.ParSet` class, for
               when this description is written to an rst table using
               :func:`~mangadap.par.parset.ParSet.to_rst_table` and
               included in an rst doc synthesized into html using
               sphinx.

Restrictions are placed on the allowed values and types for the parameters and new keys cannot be added without doing so explicitly:

>>> p['test'] = 'the other'
...
ValueError: Input value for test invalid: the other.
Options are: ['this', 'that']
>>> p['par'] = 'foo'
...
TypeError: Input value for par has incorrect type: foo.
Valid types are: [<class 'int'>]
>>> p['new'] = 8.3
...
KeyError: 'new is not a valid key for ParSet.'
>>> p.add('new', 8.3, dtype=float)
>>> p
Parameter  Value  Default  Type        Callable
-----------------------------------------------
test       that   this     str         False
par        3      0        int         False
lst        0.0    0.0      int, float  False
junk       None   None     Undefined   False
new        8.3    None     float       False
>>> p['new'] = 8
...
TypeError: Input value for new has incorrect type: 8.
Valid types are: [<class 'float'>]
>>> p['new'] = 8.
>>> p['new']
8.0

There are also a number of IO methods:

To convert to or instantiate from a dictionary:

>>> p.to_dict()
{'test': 'that', 'par': 3, 'lst': 0.0, 'junk': None}
>>> p.data
{'test': 'that', 'par': 3, 'lst': 0.0, 'junk': None}
>>> p.to_dict() is p.data
True
>>> ParSet.from_dict(p.to_dict())
Parameter  Value  Default  Type       Callable
----------------------------------------------
test       that   None     Undefined  False
par        3      None     Undefined  False
lst        0.0    None     Undefined  False
junk       None   None     Undefined  False

To write to or read from an astropy.io.fits.Header:

>>> from astropy.io import fits
>>> hdr = fits.Header()
>>> p.to_header(hdr)
>>> hdr
PAR1    = 'that    '           / ParSet: test
PAR2    =                    3 / ParSet: par
PAR3    =                  0.0 / ParSet: lst
PAR4    = 'None    '           / ParSet: junk
>>> ParSet.from_header(hdr)
Parameter  Value  Default  Type       Callable
----------------------------------------------
test       that   None     Undefined  False
par        3      None     Undefined  False
lst        0.0    None     Undefined  False
junk       None   None     Undefined  False

To write to or read from a configuration file:

>>> print('\n'.join(p.to_config()))
[default]
    # test parameter description
    test = that
    # par parameter description
    par = 3
    lst = 0.0
    # this is a rather long description of the junk parameter.  You can
    # include rst-style references like pointing back to the
    # :class:`~mangadap.par.parset.ParSet` class, for when this
    # description is written to an rst table using
    # :func:`~mangadap.par.parset.ParSet.to_rst_table` and included in an
    # rst doc synthesized into html using sphinx.
    junk = None
>>> ParSet.from_config(p.to_config())
Parameter  Value  Default  Type       Callable
----------------------------------------------
test       that   None     Undefined  False
par        3      None     Undefined  False
lst        0.0    None     Undefined  False
junk       None   None     Undefined  False

Note that in all of the IO methods above, the instantiation method loses essentially all of the differences between the ParSet and a normal dictionary. For this and other reasons, we’ve implemented an abstract class called KeywordParSet.

KeywordParSet class

The KeywordParSet class is derived from ParSet and does two things:

overwrites the add() method so that no new parameters can be added and

overwrites the from_dict() method with the expectation that any class derived from KeywordParSet has an __init__ method that takes a fixed set of keyword arguments.

By overwriting the base class definition, from_dict() takes care of all of the other “from” methods because they in turn use this “from_dict” method to instantiate the object.

All of the parameter-set classes defined and used by the DAP use KeywordParSet as their base. We can rewrite the ParSet example above to use this new base class and construct a relevant demonstration class:

from mangadap.par.parset import KeywordParSet

class DemoPar(KeywordParSet):
    def __init__(self, test=None, par=None, lst=None, junk=None):

        pars = ['test', 'par', 'lst', 'junk']
        values = [test, par, lst, junk]
        defaults = ['this', 0, 0.0, None]
        options = [ ['this', 'that'], None, None, None ]
        dtypes = [ str, int, [int, float], None ]
        descr = ['test parameter description',
                 'par parameter description',
                 None,
                 'this is a rather long description of the junk parameter.  You can include ' \
                    'rst-style references like pointing back to the ' \
                    ':class:`~mangadap.par.parset.ParSet` class, for when this description ' \
                    'is written to an rst table using ' \
                    ':func:`~mangadap.par.parset.ParSet.to_rst_table` ' \
                    'and included in an rst doc synthesized into html using sphinx.']
        super(DemoPar, self).__init__(pars, values=values, defaults=defaults, options=options,
                                      dtypes=dtypes, descr=descr)

The instantiation method for the derived class looks nearly identical to how we originally defined the ParSet instance. However, we can now define the instance using keyword arguments directly, and the ancillary information is now propagated to all the IO methods:

>>> p = DemoPar(par=3, test='that')
>>> p
Parameter  Value  Default  Type        Callable
-----------------------------------------------
test       that   this     str         False
par        3      0        int         False
lst        0.0    0.0      int, float  False
junk       None   None     Undefined   False
>>> p['test'] = 'the other'
...
ValueError: Input value for test invalid: the other.
Options are: ['this', 'that']
>>> p.add('new', 8.3, dtype=float)
...
NotImplementedError: Cannot add parameters to a DemoPar instance.
>>> DemoPar.from_dict(p.to_dict())
Parameter  Value  Default  Type        Callable
-----------------------------------------------
test       that   this     str         False
par        3      0        int         False
lst        0.0    0.0      int, float  False
junk       None   None     Undefined   False
>>> from astropy.io import fits
>>> hdr = fits.Header()
>>> p.to_header(hdr)
>>> DemoPar.from_header(hdr)
Parameter  Value  Default  Type        Callable
-----------------------------------------------
test       that   this     str         False
par        3      0        int         False
lst        0.0    0.0      int, float  False
junk       None   None     Undefined   False
>>> DemoPar.from_config(p.to_config())
Parameter  Value  Default  Type        Callable
-----------------------------------------------
test       that   this     str         False
par        3      0        int         False
lst        0.0    0.0      int, float  False
junk       None   None     Undefined   False

Todo

Add range and length parameters allowing one to define the range allowed for the parameter values and number of elements required (if the parameter is an array)
Allow for a from_par_file classmethod to initialize the parameter set based on a yanny parameter file.
Save the defaults and allow for a revert_to_default function.
Write an __add__ function that will all you to add multiple parameter sets.

class mangadap.par.parset.KeywordParSet(pars, values=None, defaults=None, options=None, dtypes=None, can_call=None, descr=None, cfg_section=None, cfg_comment=None)[source]

Bases: ParSet

An abstract class that uses ParSet as its base.

The main purpose of this class is to redefine the ParSet.from_dict() method and disallow adding new parameters.

add(*args, **kwargs)[source]: Disallow functionality of base class that adds new parameters.

classmethod from_dict(data, ignore_extra=True)[source]: Construct the object using a dictionary.

class mangadap.par.parset.ParDatabase(inp)[source]

Bases: object

Class used as a list of ParSets in a glorified structured numpy array.

Very similar to yanny when converted to a numpy array.

Can be initialized using a list of ParSet objects, or an SDSS parameter file.

Todo

Check that the data types are the same for all ParSet objects in the list
Better handle the NaN values when converting None to a float type
Add from_par_file classmethod?

static _set_dtypes(inp, i)[source]

append(db)[source]

keys()[source]

class mangadap.par.parset.ParSet(pars, values=None, defaults=None, options=None, dtypes=None, can_call=None, descr=None, cfg_section=None, cfg_comment=None)[source]

Bases: object

Generic base class to handle and manipulate a list of operational parameters. A glorified dictionary that constrains and types its components.

Parameters:

pars (list) – A list of keywords for a list of parameter values.
values (list, optional) – Initialize the parameters to these values. If not provided, all parameters are initialized to None or the provided default.
defaults (list, optional) – For any parameters not provided in the values list, use these default values. If not provided, no defaults are assumed.
options (list, optional) – Force the parameters to be one of a list of options. Each element in the list can be a list itself. If not provided, all parameters are allowed to take on any value within the allowed data type.
dtypes (list, optional) – Force the parameter to be one of a list of data types. Each element in the list can be a list itself. If not provided, all parameters are allowed to have any data type.
can_call (list, optional) – Flag that the parameters are callable operations. Default is False.
descr (list, optional) – A list of parameter descriptions. Empty strings by default.
cfg_section (str, optional) – The top-level designation for a configuration section written based on the contents of this parameter set.
cfg_comment (str, optional) – Comment to be placed at the top-level of the configuration section written based on the contents of this parameter set.

Raises:

TypeError – Raised if the input parameters are not lists or if the input keys are not strings.
ValueError – Raised if any of the optional arguments do not have the same length as the input list of parameter keys.

npar

Number of parameters

Type:: int

data

Dictionary with the parameter values

Type:: dict

default

Dictionary with the default values

Type:: dict

options

Dictionary with the allowed options for the parameter values

Type:: dict

dtype

Dictionary with the allowed data types for the parameters

Type:: dict

can_call

Dictionary with the callable flags

Type:: dict

descr

Dictionary with the description of each parameter.

Type:: dict

cfg_section

The top-level designation for a configuration section written based on the contents of this parameter set.

Type:: str

cfg_comment

Comment to be placed at the top-level of the configuration section written based on the contents of this parameter set.

Type:: str

static _config_comment(comment, indent, full_width=72)[source]

Create the list of lines for the description of a given parameter in the configuration file.

Parameters:

comment (str) – The description of the configuration parameter.
indent (str) – The string used to indent the text.
full_width (int, optional) – The full width allowed for each output string in the returned list.

Returns:

List of the strings to write to the output configuration file.

Return type:

list

static _data_string(data, use_repr=True, verbatum=False)[source]

Convert a single datum into a string

Simply return strings, recursively convert the elements of any objects with a __len__ attribute, or use the object’s own __repr__ attribute for all other objects.

Parameters:: data (object) – The object to stringify.

static _data_table_string(data_table, delimiter='print')[source]

Provided the array of data, format it with equally spaced columns and add a header (first row) and contents delimiter.

Parameters:: data_table (numpy.ndarray) – Array of string representations of the data to print.
Returns:: Single long string with the data table.
Return type:: str

_output_string(header=None, value_only=False)[source]

Constructs the short-format table strings for the __repr__ method.

Parameters:

header (str, optional) – String header to provide for the table. This is typically the name of the configuration section.
value_only (bool, optional) – By default, the table includes the parameter key, its current value, the default value, its data type, and if the value can be a callable function. If value_only=True, only the parameter key and current value are returned.

Returns:

Single long string with the parameter table for the __repr__ method.

Return type:

str

static _rst_class_name(p)[source]

_types_list(key)[source]: Return the string names for the specified data types.

_wrap_print(head, output, tcols)[source]

Wrap the contents of an output string for a fixed terminal width. Used for the long-format info() method.

Parameters:

head (str) – The inline header for the output. Can be an empty string, but cannot be None.
output (str) – The main body of the text to write.
tcols (int) – The allowed width for the output.

add(key, value, default=None, options=None, dtype=None, can_call=None, descr=None)[source]

Add a new parameter.

Parameters:

key (str) – Key for new parameter
value (object) – Parameter value, must have a type in the list provided by dtype, if the list is provided
default (object, optional) – Define a default value for the parameter, must have a type in the list provided by dtype, if the list is provided. No default if not provided.
options (list, optional) – List of discrete values that the parameter is allowed to have. Allowed to be anything if not provided.
dtype (list, optional) – List of allowed data types that the parameter can have. Allowed to be anything if not provided.
can_call (bool, optional) – Flag that the parameters are callable operations. Default is False.
descr (str, optional) – Parameter description. Default is that no description is added.

Raises:

ValueError – Raised if the keyword alread exists.

static config_lines(par, section_name=None, section_comment=None, section_level=0, exclude_defaults=False, include_descr=True)[source]

Recursively generate the lines of a configuration file based on the provided ParSet or dict (par).

Parameters:

section_name (str, optional) – Name to give to the top-level of the configuration output.
section_comment (str, optional) – Description to provide for the top-level configuration output.
section_level (int, optional) – The level for the configuration output. Sets the indentation level and the number of square brackets assigned to the section name.
exclude_defaults (bool, optional) – Do not include any parameters that are identical to the defaults.
include_descr (bool, optional) – Include the descriptions of each parameter as comments.

Returns:

The list of the lines to write to a configuration file.

Return type:

list

classmethod from_config(cfg, section_name='default', evaluate=True)[source]

Construct the parameter set using a configuration file.

Parameters:

cfg (str, list) – Either a single string with a file name to read, or a list of configuration-file-style strings with the parameters.
section_name (str, optional) – The configuration file section with the parameters.
evaluate (bool, optional) –
Evaluate the values in the config object before assigning them in the subsequent parameter sets. The parameters in the config file are always read as strings, so this should almost always be true; however, see the warning below.

Warning

When evaluate is true, the function runs eval() on all the entries in the ConfigObj dictionary, done using mangadap.par.util.recursive_dict_evaluate(). This has the potential to go haywire if the name of a parameter unintentionally happens to be identical to an imported or system-level function. Of course, this can be useful by allowing one to define the function to use as a parameter, but it also means one has to be careful with the values that the parameters should be allowed to have. The current way around this is to provide a list of strings that should be ignored during the evaluation, done using mangadap.par.util._eval_ignore().

Returns:

The instance of the parameter set.

Return type:

ParSet

classmethod from_dict(data)[source]

Create a ParSet from a dictionary.

Objects built in this way are nearly identical to a normal dictionary, except that one cannot add keys in the same way.

classmethod from_header(hdr, prefix=None)[source]

Instantiate the ParSet using data parsed from a fits header.

Parameters:

hdr (astropy.io.fits.Header) – Header object with the parameters.
prefix (str, optional) – Prefix of the relevant header keywords, which overwrites the string defined for the class. If None, uses the default for the class.

info(basekey=None)[source]: A long-form version of __repr__ that includes the parameter descriptions.

keys()[source]: Return the list of parameter set keys.

static parse_par_from_hdr(hdr, prefix)[source]

Parse the dictionary of parameters written to a header.

Parameters:

hdr (astropy.io.fits.Header) – Header object to parse.
prefix (str) – The prefix used for the header keywords.

Returns:

A dictionary with the parameter keywords and values.

Return type:

dict

prefix = 'PAR': Class prefix for header keywords when writing the parset to an astropy.io.fits.Header object.

to_config(cfg_file=None, section_name=None, section_comment=None, section_level=0, append=False, quiet=False, exclude_defaults=False, include_descr=True)[source]

Write/Append the parameter set to a configuration file.

Parameters:

cfg_file (str, optional) – The name of the file to write/append to. If None (default), the function will just return the list of strings that would have been written to the file. These lines can be used to construct a configobj.ConfigObj instance.
section_name (str, optional) – The top-level name for the config section. This must be provided if cfg_section is None or any of the parameters are not also ParSet instances themselves.
section_comment (str, optional) – The top-level comment for the config section based on this ParSet.
section_level (int, optional) – The top level of this ParSet. Used for recursive output of nested ParSet objects.
append (bool, optional) – Append this configuration output of this ParSet to the file. False by default. If not appending and the file exists, the file is automatically overwritten.
quiet (bool, optional) – Suppress all standard output from the function.
exclude_defaults (bool, optional) – Do not include any parameters that are identical to the defaults.
include_descr (bool, optional) – Include the descriptions of each parameter as comments.

Raises:

ValueError – Raised if there are types other than ParSet in the parameter list, cfg_section is None, and no section_name argument was provided.

to_dict()[source]: Return a dictionary with the parameters.

Warning

This simply returns a pointer to the internal object dictionary, data.

to_header(hdr, prefix=None, quiet=False)[source]

Write the parameters to a fits header.

Any element that has a value of None or is a ParSet itself is not written to the header.

Parameters:

hdr (astropy.io.fits.Header) – Header object for the parameters. Modified in-place.
prefix (str, optional) – Prefix to use for the header keywords, which overwrites the string defined for the class. If None, uses the default for the class.
quiet (bool, optional) – Suppress print statements.

to_rst_table(parsets_listed=[], header=True, class_link=True, nested=True)[source]

Construct a reStructuredText table with the ParSet data.

This method is mostly meant for documentation purposes, as way of showing the format and default parameters of a given single ParSet or nested set of ParSet objects.

Parameters:

parsets_listed (list, optional) – For nested ParSet objects, this is used to keep a log of ParSet objects that have already been included in the rst table, forcing the table to only appear once.
header (bool, optional) – Include a section header
class_link (bool, optional) – Include an rst-style link to the class instantiation documentation.

Returns:

A list of strings containing each line of the rst table. To print the table:

print('\n'.join(p.to_rst_table()))

where p is a ParSet instance.

Return type:

list

validate_keys(required=None, can_be_None=None)[source]

Validate the keys in the ParSet.

Parameters:

required (list, optional) – A list of required keys.
can_be_None (list, optional) – A list of keys with values that are allowed to be None. All other keys are expected to have defined values.

Raises:

ValueError – Raised if required keys are not present or if keys have None values and are not expected to be.