mangadap.util.bitmask module¶

Base class for handling bit masks by the DAP.

License:

Copyright (c) 2015, SDSS-IV/MaNGA Pipeline Group: Licensed under BSD 3-clause license - see LICENSE.rst

Source location:

$MANGADAP_DIR/python/mangadap/util/bitmask.py

Imports and python version compliance:

from __future__ import division
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals

import sys
if sys.version > '3':
    long = int
    try:
        from configparser import ConfigParser
    except ImportError:
        print('WARNING: Unable to import configparser!  Beware!')
else:
    try:
        from ConfigParser import ConfigParser
    except ImportError:
        print('WARNING: Unable to import ConfigParser!  Beware!')

import numpy
import os
import textwrap
from pydl.pydlutils.yanny import yanny

Class usage examples:

Here is an example of reading/creating a TemplateLibrary, then creating a plot that shows the full spectrum (blue) and the unmasked spectrum (green):

# Imports
import numpy
from mangadap.drpfits import DRPFits
from mangadap.proc.TemplateLibrary import TemplateLibrary, TemplateLibraryBitMask
from matplotlib import pyplot

# Define the DRP file
drpf = DRPFits(7495, 12703, 'CUBE')

# Build the template library
tpl_lib = TemplateLibrary('M11-MILES', drpf=drpf, directory_path='.')
# Writes: ./manga-7495-12703-LOGCUBE_M11-MILES.fits

# Initialize the mask object
tplbm = TemplateLibraryBitMask()

# Refactor the mask for the first template spectrum using the bitmask
unmasked = numpy.invert(tplbm.flagged(tpl_lib.hdu['MASK'].data[0,:]))

# Plot the full spectrum (blue)
pyplot.plot(tpl_lib.hdu['WAVE'].data, tpl_lib.hdu['FLUX'].data[0,:])
# Plot the unmasked pixels (green; points are connected even if there are gaps
pyplot.plot(tpl_lib.hdu['WAVE'].data[unmasked], tpl_lib.hdu['FLUX'].data[0,unmasked])
# Show the plot
pyplot.show()

Revision history:

01 Jun 2015: Original implementation by K. Westfall (KBW)
07 Oct 2015: (KBW) Added a usage case
29 Jan 2016: (KBW) Changed attributes of BitMask and
added functionality to print a description of the bits.  Convert
mangadap.proc.templatelibrary.TemplateLibraryBitMask to
new format where the bits are read from a configuration file.
17 Feb 2016: (KBW) Minor edit to documentation
16 Mar 2016: (KBW) Moved TemplateLibraryBitMask to
mangadap.proc.templatelibrary.TemplateLibraryBitMask;
moved HDUList_mask_wavelengths to
mangadap.proc.util.HDUList_mask_wavelengths().
27 Mar 2016: (KBW) Added BitMask.from_ini_file() and
BitMask.from_par_file() class methods.  Added
BitMask._fill_sequence() static method.  This allows for
BitMask objects to be declared directly from the files,
and allows the bit values to take on any number.
05 Apr 2016: (KBW) Added parameters to initialization of
BitMask objects to clean up some of the derived class
initialization.
11 May 2016: (KBW) Switch to using pydl.pydlutils.yanny
instead of internal yanny reader
29 Jul 2016: (KBW) Change asarray to atleast_1d

class mangadap.util.bitmask.BitMask(keys=None, descr=None, ini_file=None, par_file=None, par_grp=None)[source]¶

Bases: object

Generic class to handle and manipulate bitmasks. The input list of bit names (keys) must be unique, except that values of ‘NULL’ are ignored. The index in the input keys determines the bit value; ‘NULL’ keys are included in the count. For example:

>>> from mangadap.util.bitmask import BitMask
>>> keys = [ 'key1', 'key2', 'NULL', 'NULL', 'key3' ]
>>> bm = BitMask(keys)
>>> bm.info()
         Bit: key1 = 0

         Bit: key2 = 1

         Bit: key3 = 4

Parameters:	keys (str, list, numpy.ndarray) – (Optional) List of keys (or single key) to use as the bit name. Each key is given a bit number ranging from 0..N-1. If not provided, one of either ini_file or par_file must be provided. descr (str, list, numpy.ndarray) – (Optional) List of descriptions (or single discription) provided by `info()` for each bit.
Raises:	`ValueError` – Raised if more than 64 bits are provided. `TypeError` – Raised if the provided keys do not have the correct type.

nbits¶

Number of bits

Type:	int

bits¶

A dictionary with the bit name and value

Type:	dict

descr¶

List of bit descriptions

Type:	numpy.ndarray

max_value¶

The maximum valid bitmask value given the number of bits.

Type:	int

static _fill_sequence(keys, vals, descr)[source]¶

The instantiation of BitMask does not include the value of the bit, it just assumes that the bits should be in sequence such that the first key has a value of 0, and the last key has a value of N-1. Unfortunately, not all the bits the DAP follow a sequence. This function finds the range of the bits and then slots in NULL keywords and empty descriptions where necessary to fill in the full complement of bits. NULL keywords are ignored by the BitMask object.

This is a static method because it doesn’t depend on any of the attributes of BitMask.

Parameters:	keys (list or str) – Bit names vals (list or int) – Bit values descr (list or str) – Description of each bit
Returns:	Three arrays with the filled keys, values, and descriptions.
Return type:	numpy.ndarray
Raises:	`ValueError` – Raised if a bit value is less than 0.

consolidate(value, flag_set, consolidated_flag)[source]¶: Consolidate a set of flags into a single flag.

flagged(value, flag=None)[source]¶

Determine if a bit is on in the provided bitmask value. The function can be used to determine if any individual bit is on or any one of many bits is on.

Parameters:	value (uint, int, array) – Bitmask value. It should be less than or equal to `max_value`; however, that is not checked. flag (string, array) – (Optional) Bit names to check. If None, then it checks if any bit is on.
Returns:	Boolean flags that the provided flags (or any flag) is on for the provided bitmask value.
Return type:	bool
Raises:	`KeyError` – Raised by the dict data type if the input flag is not one of the valid `flags`. `TypeError` – Raised if the provided flag does not contain one or more strings.

flagged_bits(value)[source]¶

Return the list of flagged bit names for a single bit value.

Parameters:	value (int, uint) – Bitmask value. It should be less than or equal to `max_value`; however, that is not checked.
Returns:	List of flagged bit value keywords.
Return type:	list
Raises:	`KeyError` – Raised by the dict data type if the input flag is not one of the valid `flags`. `TypeError` – Raised if the provided flag does not contain one or more strings.

classmethod from_ini_file(f)[source]¶

Define the object using a ini configuration file. The sections of the configuration file define the keys, and each section is expected to have value and descr components that define the bit value and provided a description of the bit. An example ini file might look like this:

[NO_DATA]
 value = 0
 descr = Pixel has no data

[INVALID_DATA]
 value = 1
 descr = Pixel value is invalid

See mangadap.proc.templatelibrary.TemplateLibraryBitMask for an example that uses this function.

Parameters:	f (str) – File name to use for defining the `BitMask`.
Returns:	Object with bitmasks defined by the ini file.
Return type:	`BitMask`
Raises:	`FileNotFoundError` – Raised if the input file does not exist.

classmethod from_par_file(f, name)[source]¶

Define the object using an SDSS-style parameter file. This has been tailored to work with the sdssMaskbits.par file in IDLUTILS; however, it can work with other like files.

See mangadap.drpfits.DRPFitsBitMask for an example that uses this function.

Parameters:	f (str) – File name to use for defining the `BitMask`. name (str) – The designation of the bits to assign. For example, in `mangadap.drpfits.DRPFitsBitMask` this is ‘MANGA_DRP3PIXMASK’.
Returns:	Object with bitmasks defined by the parameter file.
Return type:	`BitMask`
Raises:	`FileNotFoundError` – Raised if the input file does not exist.

info()[source]¶: Print the list of bits and, if available, their descriptions.

keys()[source]¶

Return a list of the bits; ‘NULL’ keywords are ignored.

Returns:	List of bit keywords.
Return type:	list

minimum_dtype(asuint=False)[source]¶: Return the smallest int datatype that is needed to contain all the bits in the mask. Output as an unsigned int if requested.

Warning

uses int16 if the number of bits is less than 8 and asuint=False because of issue astropy.io.fits has writing int8 values.

toggle(value, flag)[source]¶

Toggle a bit in the provided bitmask value.

Parameters:	value (uint, int, array) – Bitmask value. It should be less than or equal to `max_value`; however, that is not checked. flag (list, numpy.ndarray, or str) – Bit name(s) to toggle.
Returns:	New bitmask value after toggling the selected bit.
Return type:	uint
Raises:	`KeyError` – Raised by the dict data type if the input flag is not one of the valid `flags`. `Exception` – Raised if the provided flag is not a string.

turn_off(value, flag)[source]¶

Ensure that a bit is turned off in the provided bitmask value.

Parameters:	value (uint or array) – Bitmask value. It should be less than or equal to `max_value`; however, that is not checked. flag (list, numpy.ndarray, or str) – Bit name(s) to turn off.
Returns:	New bitmask value after turning off the selected bit.
Return type:	uint
Raises:	`KeyError` – Raised by the dict data type if the input flag is not one of the valid `flags`. `Exception` – Raised if the provided flag is not a string.

turn_on(value, flag)[source]¶

Ensure that a bit is turned on in the provided bitmask value.

Parameters:	value (uint or array) – Bitmask value. It should be less than or equal to `max_value`; however, that is not checked. flag (list, numpy.ndarray, or str) – Bit name(s) to turn on.
Returns:	New bitmask value after turning on the selected bit.
Return type:	uint
Raises:	`KeyError` – Raised by the dict data type if the input flag is not one of the valid `flags`. `Exception` – Raised if the provided flag is not a string.