mangadap.par.parset module¶
Parameter sets are used to control the behavior of internal methods from top-level input. Parameter sets:
provide I/O methods for inspection, documentation, and logging
force parameters to have specific data types, including if the parameter should be callable
specify allowed options for method keywords
provide parameter descriptions
Beyond this and once instantiated, though, they are similar to a normal python dictionary.
ParSet base class¶
As an example, we can create a bogus parameter set as follows:
from mangadap.par.parset import ParSet
pars = ['test', 'par', 'lst', 'junk']
defaults = ['this', 0, 0.0, None]
options = [['this', 'that'], None, None, None]
dtypes = [str, int, [int, float], None]
descr = ['test parameter description',
'par parameter description',
None,
'this is a rather long description of the junk parameter. You can include ' \
'rst-style references like pointing back to the ' \
':class:`~mangadap.par.parset.ParSet` class, for when this description is ' \
'written to an rst table using :func:`~mangadap.par.parset.ParSet.to_rst_table` ' \
'and included in an rst doc synthesized into html using sphinx.']
p = ParSet(pars, defaults=defaults, options=options, dtypes=dtypes, descr=descr)
Once defined, accessing/showing the values is done similarly to a dictionary, with some convenience printing methods:
>>> p['test']
'that'
>>> p['lst']
0.0
>>> p['junk'] is None
True
>>> p
Parameter Value Default Type Callable
-----------------------------------------------
test that this str False
par 3 0 int False
lst 0.0 0.0 int, float False
junk None None Undefined False
>>> p.info()
test
Value: this
Default: this
Options: this, that
Valid Types: str
Callable: False
Description: test parameter description
par
Value: 0
Default: 0
Options: None
Valid Types: int
Callable: False
Description: par parameter description
lst
Value: 0.0
Default: 0.0
Options: None
Valid Types: int, float
Callable: False
Description: None
junk
Value: None
Default: None
Options: None
Valid Types: Undefined
Callable: False
Description: this is a rather long description of the junk parameter.
You can include rst-style references like pointing back
to the :class:`~mangadap.par.parset.ParSet` class, for
when this description is written to an rst table using
:func:`~mangadap.par.parset.ParSet.to_rst_table` and
included in an rst doc synthesized into html using
sphinx.
Restrictions are placed on the allowed values and types for the parameters and new keys cannot be added without doing so explicitly:
>>> p['test'] = 'the other'
...
ValueError: Input value for test invalid: the other.
Options are: ['this', 'that']
>>> p['par'] = 'foo'
...
TypeError: Input value for par has incorrect type: foo.
Valid types are: [<class 'int'>]
>>> p['new'] = 8.3
...
KeyError: 'new is not a valid key for ParSet.'
>>> p.add('new', 8.3, dtype=float)
>>> p
Parameter Value Default Type Callable
-----------------------------------------------
test that this str False
par 3 0 int False
lst 0.0 0.0 int, float False
junk None None Undefined False
new 8.3 None float False
>>> p['new'] = 8
...
TypeError: Input value for new has incorrect type: 8.
Valid types are: [<class 'float'>]
>>> p['new'] = 8.
>>> p['new']
8.0
There are also a number of IO methods:
To convert to or instantiate from a dictionary:
>>> p.to_dict() {'test': 'that', 'par': 3, 'lst': 0.0, 'junk': None} >>> p.data {'test': 'that', 'par': 3, 'lst': 0.0, 'junk': None} >>> p.to_dict() is p.data True >>> ParSet.from_dict(p.to_dict()) Parameter Value Default Type Callable ---------------------------------------------- test that None Undefined False par 3 None Undefined False lst 0.0 None Undefined False junk None None Undefined FalseTo write to or read from an astropy.io.fits.Header:
>>> from astropy.io import fits >>> hdr = fits.Header() >>> p.to_header(hdr) >>> hdr PAR1 = 'that ' / ParSet: test PAR2 = 3 / ParSet: par PAR3 = 0.0 / ParSet: lst PAR4 = 'None ' / ParSet: junk >>> ParSet.from_header(hdr) Parameter Value Default Type Callable ---------------------------------------------- test that None Undefined False par 3 None Undefined False lst 0.0 None Undefined False junk None None Undefined FalseTo write to or read from a configuration file:
>>> print('\n'.join(p.to_config())) [default] # test parameter description test = that # par parameter description par = 3 lst = 0.0 # this is a rather long description of the junk parameter. You can # include rst-style references like pointing back to the # :class:`~mangadap.par.parset.ParSet` class, for when this # description is written to an rst table using # :func:`~mangadap.par.parset.ParSet.to_rst_table` and included in an # rst doc synthesized into html using sphinx. junk = None >>> ParSet.from_config(p.to_config()) Parameter Value Default Type Callable ---------------------------------------------- test that None Undefined False par 3 None Undefined False lst 0.0 None Undefined False junk None None Undefined False
Note that in all of the IO methods above, the instantiation method loses
essentially all of the differences between the
ParSet
and a normal dictionary. For this
and other reasons, we’ve implemented an abstract class called
KeywordParSet
.
KeywordParSet class¶
The KeywordParSet
class is derived from
ParSet
and does two things:
overwrites the
add()
method so that no new parameters can be added andoverwrites the
from_dict()
method with the expectation that any class derived fromKeywordParSet
has an__init__
method that takes a fixed set of keyword arguments.
By overwriting the base class definition,
from_dict()
takes care of all
of the other “from” methods because they in turn use this “from_dict”
method to instantiate the object.
All of the parameter-set classes defined and used by the DAP use
KeywordParSet
as their base. We can
rewrite the ParSet
example above to use
this new base class and construct a relevant demonstration class:
from mangadap.par.parset import KeywordParSet
class DemoPar(KeywordParSet):
def __init__(self, test=None, par=None, lst=None, junk=None):
pars = ['test', 'par', 'lst', 'junk']
values = [test, par, lst, junk]
defaults = ['this', 0, 0.0, None]
options = [ ['this', 'that'], None, None, None ]
dtypes = [ str, int, [int, float], None ]
descr = ['test parameter description',
'par parameter description',
None,
'this is a rather long description of the junk parameter. You can include ' \
'rst-style references like pointing back to the ' \
':class:`~mangadap.par.parset.ParSet` class, for when this description ' \
'is written to an rst table using ' \
':func:`~mangadap.par.parset.ParSet.to_rst_table` ' \
'and included in an rst doc synthesized into html using sphinx.']
super(DemoPar, self).__init__(pars, values=values, defaults=defaults, options=options,
dtypes=dtypes, descr=descr)
The instantiation method for the derived class looks nearly identical
to how we originally defined the ParSet
instance. However, we can now define the instance using keyword
arguments directly, and the ancillary information is now propagated
to all the IO methods:
>>> p = DemoPar(par=3, test='that')
>>> p
Parameter Value Default Type Callable
-----------------------------------------------
test that this str False
par 3 0 int False
lst 0.0 0.0 int, float False
junk None None Undefined False
>>> p['test'] = 'the other'
...
ValueError: Input value for test invalid: the other.
Options are: ['this', 'that']
>>> p.add('new', 8.3, dtype=float)
...
NotImplementedError: Cannot add parameters to a DemoPar instance.
>>> DemoPar.from_dict(p.to_dict())
Parameter Value Default Type Callable
-----------------------------------------------
test that this str False
par 3 0 int False
lst 0.0 0.0 int, float False
junk None None Undefined False
>>> from astropy.io import fits
>>> hdr = fits.Header()
>>> p.to_header(hdr)
>>> DemoPar.from_header(hdr)
Parameter Value Default Type Callable
-----------------------------------------------
test that this str False
par 3 0 int False
lst 0.0 0.0 int, float False
junk None None Undefined False
>>> DemoPar.from_config(p.to_config())
Parameter Value Default Type Callable
-----------------------------------------------
test that this str False
par 3 0 int False
lst 0.0 0.0 int, float False
junk None None Undefined False
Todo
Add range and length parameters allowing one to define the range allowed for the parameter values and number of elements required (if the parameter is an array)
Allow for a from_par_file classmethod to initialize the parameter set based on a yanny parameter file.
Save the defaults and allow for a revert_to_default function.
Write an __add__ function that will all you to add multiple parameter sets.
Copyright © 2019, SDSS-IV/MaNGA Pipeline Group
- class mangadap.par.parset.KeywordParSet(pars, values=None, defaults=None, options=None, dtypes=None, can_call=None, descr=None, cfg_section=None, cfg_comment=None)[source]¶
Bases:
mangadap.par.parset.ParSet
An abstract class that uses
ParSet
as its base.The main purpose of this class is to redefine the
ParSet.from_dict()
method and disallow adding new parameters.
- class mangadap.par.parset.ParDatabase(inp)[source]¶
Bases:
object
Class used as a list of ParSets in a glorified structured numpy array.
Very similar to yanny when converted to a numpy array.
Can be initialized using a list of ParSet objects, or an SDSS parameter file.
Todo
Check that the data types are the same for all ParSet objects in the list
Better handle the NaN values when converting None to a float type
Add from_par_file classmethod?
- class mangadap.par.parset.ParSet(pars, values=None, defaults=None, options=None, dtypes=None, can_call=None, descr=None, cfg_section=None, cfg_comment=None)[source]¶
Bases:
object
Generic base class to handle and manipulate a list of operational parameters. A glorified dictionary that constrains and types its components.
- Parameters
pars (
list
) – A list of keywords for a list of parameter values.values (
list
, optional) – Initialize the parameters to these values. If not provided, all parameters are initialized to None or the provided default.defaults (
list
, optional) – For any parameters not provided in the values list, use these default values. If not provided, no defaults are assumed.options (
list
, optional) – Force the parameters to be one of a list of options. Each element in the list can be a list itself. If not provided, all parameters are allowed to take on any value within the allowed data type.dtypes (
list
, optional) – Force the parameter to be one of a list of data types. Each element in the list can be a list itself. If not provided, all parameters are allowed to have any data type.can_call (
list
, optional) – Flag that the parameters are callable operations. Default is False.descr (
list
, optional) – A list of parameter descriptions. Empty strings by default.cfg_section (
str
, optional) – The top-level designation for a configuration section written based on the contents of this parameter set.cfg_comment (
str
, optional) – Comment to be placed at the top-level of the configuration section written based on the contents of this parameter set.
- Raises
TypeError – Raised if the input parameters are not lists or if the input keys are not strings.
ValueError – Raised if any of the optional arguments do not have the same length as the input list of parameter keys.
- npar¶
Number of parameters
- Type
int
- data¶
Dictionary with the parameter values
- Type
dict
- default¶
Dictionary with the default values
- Type
dict
- options¶
Dictionary with the allowed options for the parameter values
- Type
dict
- dtype¶
Dictionary with the allowed data types for the parameters
- Type
dict
- can_call¶
Dictionary with the callable flags
- Type
dict
- descr¶
Dictionary with the description of each parameter.
- Type
dict
- cfg_section¶
The top-level designation for a configuration section written based on the contents of this parameter set.
- Type
str
- cfg_comment¶
Comment to be placed at the top-level of the configuration section written based on the contents of this parameter set.
- Type
str
- static _config_comment(comment, indent, full_width=72)[source]¶
Create the list of lines for the description of a given parameter in the configuration file.
- Parameters
comment (
str
) – The description of the configuration parameter.indent (
str
) – The string used to indent the text.full_width (
int
, optional) – The full width allowed for each output string in the returned list.
- Returns
List of the strings to write to the output configuration file.
- Return type
list
- static _data_string(data, use_repr=True, verbatum=False)[source]¶
Convert a single datum into a string
Simply return strings, recursively convert the elements of any objects with a
__len__
attribute, or use the object’s own__repr__
attribute for all other objects.- Parameters
data (object) – The object to stringify.
- static _data_table_string(data_table, delimiter='print')[source]¶
Provided the array of data, format it with equally spaced columns and add a header (first row) and contents delimiter.
- Parameters
data_table (numpy.ndarray) – Array of string representations of the data to print.
- Returns
Single long string with the data table.
- Return type
str
- _output_string(header=None, value_only=False)[source]¶
Constructs the short-format table strings for the
__repr__
method.- Parameters
header (
str
, optional) – String header to provide for the table. This is typically the name of the configuration section.value_only (
bool
, optional) – By default, the table includes the parameter key, its current value, the default value, its data type, and if the value can be a callable function. If value_only=True, only the parameter key and current value are returned.
- Returns
Single long string with the parameter table for the
__repr__
method.- Return type
str
- _wrap_print(head, output, tcols)[source]¶
Wrap the contents of an output string for a fixed terminal width. Used for the long-format
info()
method.- Parameters
head (
str
) – The inline header for the output. Can be an empty string, but cannot beNone
.output (
str
) – The main body of the text to write.tcols (
int
) – The allowed width for the output.
- add(key, value, default=None, options=None, dtype=None, can_call=None, descr=None)[source]¶
Add a new parameter.
- Parameters
key (
str
) – Key for new parametervalue (object) – Parameter value, must have a type in the list provided by
dtype
, if the list is provideddefault (object, optional) – Define a default value for the parameter, must have a type in the list provided by
dtype
, if the list is provided. No default if not provided.options (
list
, optional) – List of discrete values that the parameter is allowed to have. Allowed to be anything if not provided.dtype (
list
, optional) – List of allowed data types that the parameter can have. Allowed to be anything if not provided.can_call (
bool
, optional) – Flag that the parameters are callable operations. Default is False.descr (
str
, optional) – Parameter description. Default is that no description is added.
- Raises
ValueError – Raised if the keyword alread exists.
- static config_lines(par, section_name=None, section_comment=None, section_level=0, exclude_defaults=False, include_descr=True)[source]¶
Recursively generate the lines of a configuration file based on the provided ParSet or dict (par).
- Parameters
section_name (
str
, optional) – Name to give to the top-level of the configuration output.section_comment (
str
, optional) – Description to provide for the top-level configuration output.section_level (
int
, optional) – The level for the configuration output. Sets the indentation level and the number of square brackets assigned to the section name.exclude_defaults (
bool
, optional) – Do not include any parameters that are identical to the defaults.include_descr (
bool
, optional) – Include the descriptions of each parameter as comments.
- Returns
The list of the lines to write to a configuration file.
- Return type
list
- classmethod from_config(cfg, section_name='default', evaluate=True)[source]¶
Construct the parameter set using a configuration file.
- Parameters
cfg (
str
,list
) – Either a single string with a file name to read, or a list of configuration-file-style strings with the parameters.section_name (
str
, optional) – The configuration file section with the parameters.evaluate (
bool
, optional) –Evaluate the values in the config object before assigning them in the subsequent parameter sets. The parameters in the config file are always read as strings, so this should almost always be true; however, see the warning below.
Warning
When
evaluate
is true, the function runseval()
on all the entries in the ConfigObj dictionary, done usingmangadap.par.util.recursive_dict_evaluate()
. This has the potential to go haywire if the name of a parameter unintentionally happens to be identical to an imported or system-level function. Of course, this can be useful by allowing one to define the function to use as a parameter, but it also means one has to be careful with the values that the parameters should be allowed to have. The current way around this is to provide a list of strings that should be ignored during the evaluation, done usingmangadap.par.util._eval_ignore()
.
- Returns
The instance of the parameter set.
- Return type
- classmethod from_dict(data)[source]¶
Create a
ParSet
from a dictionary.Objects built in this way are nearly identical to a normal dictionary, except that one cannot add keys in the same way.
- classmethod from_header(hdr, prefix=None)[source]¶
Instantiate the
ParSet
using data parsed from a fits header.- Parameters
hdr (astropy.io.fits.Header) – Header object with the parameters.
prefix (
str
, optional) – Prefix of the relevant header keywords, which overwrites the string defined for the class. If None, uses the default for the class.
- info(basekey=None)[source]¶
A long-form version of __repr__ that includes the parameter descriptions.
- static parse_par_from_hdr(hdr, prefix)[source]¶
Parse the dictionary of parameters written to a header.
- Parameters
hdr (astropy.io.fits.Header) – Header object to parse.
prefix (
str
) – The prefix used for the header keywords.
- Returns
A dictionary with the parameter keywords and values.
- Return type
dict
- prefix = 'PAR'¶
Class prefix for header keywords when writing the parset to an astropy.io.fits.Header object.
- to_config(cfg_file=None, section_name=None, section_comment=None, section_level=0, append=False, quiet=False, exclude_defaults=False, include_descr=True)[source]¶
Write/Append the parameter set to a configuration file.
- Parameters
cfg_file (
str
, optional) – The name of the file to write/append to. If None (default), the function will just return the list of strings that would have been written to the file. These lines can be used to construct a configobj.ConfigObj instance.section_name (
str
, optional) – The top-level name for the config section. This must be provided ifcfg_section
is None or any of the parameters are not alsoParSet
instances themselves.section_comment (
str
, optional) – The top-level comment for the config section based on thisParSet
.section_level (
int
, optional) – The top level of thisParSet
. Used for recursive output of nestedParSet
objects.append (
bool
, optional) – Append this configuration output of thisParSet
to the file. False by default. If not appending and the file exists, the file is automatically overwritten.quiet (
bool
, optional) – Suppress all standard output from the function.exclude_defaults (
bool
, optional) – Do not include any parameters that are identical to the defaults.include_descr (
bool
, optional) – Include the descriptions of each parameter as comments.
- Raises
ValueError – Raised if there are types other than
ParSet
in the parameter list,cfg_section
isNone
, and no section_name argument was provided.
- to_dict()[source]¶
Return a dictionary with the parameters.
Warning
This simply returns a pointer to the internal object dictionary,
data
.
- to_header(hdr, prefix=None, quiet=False)[source]¶
Write the parameters to a fits header.
Any element that has a value of
None
or is aParSet
itself is not written to the header.- Parameters
hdr (astropy.io.fits.Header) – Header object for the parameters. Modified in-place.
prefix (
str
, optional) – Prefix to use for the header keywords, which overwrites the string defined for the class. IfNone
, uses the default for the class.quiet (
bool
, optional) – Suppress print statements.
- to_rst_table(parsets_listed=[], header=True, class_link=True)[source]¶
Construct a reStructuredText table with the
ParSet
data.This method is mostly meant for documentation purposes, as way of showing the format and default parameters of a given single
ParSet
or nested set ofParSet
objects.- Parameters
parsets_listed (
list
, optional) – For nestedParSet
objects, this is used to keep a log ofParSet
objects that have already been included in the rst table, forcing the table to only appear once.header (
bool
, optional) – Include a section headerclass_link (
bool
, optional) – Include an rst-style link to the class instantiation documentation.
- Returns
A list of strings containing each line of the rst table. To print the table:
print('\n'.join(p.to_rst_table()))
where
p
is aParSet
instance.- Return type
list
- validate_keys(required=None, can_be_None=None)[source]¶
Validate the keys in the
ParSet
.- Parameters
required (
list
, optional) – A list of required keys.can_be_None (
list
, optional) – A list of keys with values that are allowed to be None. All other keys are expected to have defined values.
- Raises
ValueError – Raised if required keys are not present or if keys have
None
values and are not expected to be.