mangadap.proc.reductionassessments module¶

Class that performs a number of assessments of a DRP file needed for handling of the data by the DAP. These assessments need only be done once per DRP data file.

License:

Copyright (c) 2015, SDSS-IV/MaNGA Pipeline Group: Licensed under BSD 3-clause license - see LICENSE.rst

Source location:

$MANGADAP_DIR/python/mangadap/proc/reductionassessments.py

Imports and python version compliance:

from __future__ import division
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals

import sys
import warnings
if sys.version > '3':
    long = int

import glob
import os
import time
import logging
import numpy

from scipy import sparse
from pydl.goddard.astro import airtovac
from astropy.io import fits

from ..drpfits import DRPFits
from ..par.parset import ParSet
from ..config.defaults import dap_source_dir, default_dap_common_path
from ..config.defaults import default_dap_file_name
from ..util.covariance import Covariance
from ..util.geometry import SemiMajorAxisCoo
from ..util.fileio import init_record_array, rec_to_fits_type, write_hdu
from ..util.log import log_output
from ..util.parser import DefaultConfig
from .util import select_proc_method

Class usage examples:

Add examples!

Revision history:

24 Mar 2016: Implementation begun by K. Westfall (KBW)
11 May 2016: (KBW) Switch to using
pydl.goddard.astro.airtovac instead of internal function
19 May 2016: (KBW) Added loggers and quiet keyword arguments
to ReductionAssessment, removed verbose
23 Feb 2017: (KBW) Use DAPFitsUtil read and write functions.
27 Feb 2017: (KBW) Use DefaultConfig.
17 May 2017: (KBW) Added ability to use a response function
for the flux statistics.

class mangadap.proc.reductionassessments.ReductionAssessment(method_key, drpf, pa=0.0, ell=0.0, method_list=None, dapsrc=None, dapver=None, analysis_path=None, directory_path=None, output_file=None, hardcopy=True, symlink_dir=None, clobber=False, checksum=False, loggers=None, quiet=False)[source]¶

Bases: object

Object used to perform and store assessments of a DRP file needed for handling of the data by the DAP. These assessments need only be done once per DRP data file.

See compute() for the provided data.

Parameters:

method_key (str) – Keyword selecting the assessment method to use.
drpf (mangadap.drpfits.DRPFits) – DRP file (object) to use for the assessments.
pa (float) – (Optional) On-sky position angle of the major axis used to calculate elliptical, semi-major-axis coordinates, defined as the angle from North through East and denoted $\phi_0$. Default is 0.0.
ell (float) – (Optional) Ellipticity defined as $\varepsilon=1-b/a$, based on the semi-minor to semi-major axis ratio ($b/a$) of the isophotal ellipse used to calculate elliptical, semi-major-axis coordinates. Default is 0.0.
method_list (list) – (Optional) List of ReductionAssessmentDef objects that define the parameters required to perform the assessments of the DRP file. The method_key must select one of these objects. Default is to construct list using available_reduction_assessments().
dapver (str) – (Optional) DAP version, which is used to define the default DAP analysis path. Default is defined by mangadap.config.defaults.default_dap_version()
analysis_path (str) – (Optional) The path to the top level directory containing the DAP output files for a given DRP and DAP version. Default is defined by mangadap.config.defaults.default_analysis_path().
directory_path (str) – (Optional) The exact path for the output file. Default is defined by mangadap.config.defaults.default_dap_common_path().
output_file (str) – (Optional) The name of the file for the computed assessments. The full path of the output file will be directory_path/output_file. Default is defined by mangadap.config.defaults.default_dap_file_name().
hardcopy (bool) – (Optional) Flag to write the data to a fits file. Default is True.
symlink_dir (str) – (Optional) Create a symlink to the file in this directory. Default is for no symlink.
clobber (bool) – (Optional) If the output file already exists, this will force the assessments to be redone and the output file to be overwritten. Default is False.
checksum (bool) – (Optional) Compare the checksum when reading the data.
loggers (list) – (Optional) List of logging.Logger objects to log progress; ignored if quiet=True. Logging is done using mangadap.util.log.log_output(). Default is no logging.
quiet (bool) – (Optional) Suppress all terminal and logging output. Default is False.

method¶

Parameters defining the method to use for the reduction assessments.

Type:	`ReductionAssessmentDef`

drpf¶

DRP file (object) with which the template library is associated for analysis

Type:	`mangadap.drpfits.DRPFits`

pa¶

On-sky position angle of the major axis used to calculate elliptical, semi-major-axis coordinates, defined as the angle from North through East and denoted $\phi_0$.

Type:	float

ell¶

Ellipticity defined as $\varepsilon=1-b/a$, based on the semi-minor to semi-major axis ratio ($b/a$) of the isophotal ellipse used to calculate elliptical, semi-major-axis coordinates.

Type:	float

directory_path¶

The exact path for the output file. Default is defined by mangadap.config.defaults.default_dap_common_path().

Type:	str

output_file¶

The name of the file for the computed assessments. The full path of the output file will be directory_path/output_file. Default is defined by _set_paths().

Type:	str

hardcopy¶

Flag to keep a hardcopy of the data by writing the data to a fits file.

Type:	bool

symlink_dir¶

Symlink created to the file in this directory

Type:	str

hdu¶

HDUList with the data, with columns as described above.

Type:	astropy.io.fits.hdu.hdulist.HDUList

correlation¶

Covariance matrix for the mean flux measurements, if calculated.

Type:	`mangadap.util.covariance.Covariance`

loggers¶

List of logging.Logger objects to log progress; ignored if quiet=True. Logging is done using mangadap.util.log.log_output().

Type:	list

quiet¶

Suppress all terminal and logging output.

Type:	bool

_define_method(method_key, method_list=None, dapsrc=None)[source]¶

Select the assessment method from the provided list. Used to set method; see mangadap.proc.util.select_proc_method().

Parameters:

method_key (str) – Keyword of the selected method. Available methods are provided by available_reduction_assessments()
method_list (list) – (Optional) List of ReductionAssessmentDef objects that define the parameters required to assess the reduced data.
dapsrc (str) – (Optional) Root path to the DAP source directory. If not provided, the default is defined by mangadap.config.defaults.dap_source_dir().

_initialize_primary_header(hdr=None)[source]¶

_per_spectrum_dtype()[source]¶: Construct the record array data type for the output fits extension.

_set_paths(directory_path, dapver, analysis_path, output_file)[source]¶

Set the I/O paths. Used to set directory_path and output_file. If not provided, the defaults are set using, respectively, mangadap.config.defaults.default_dap_common_path() and mangadap.config.defaults.default_dap_file_name().

Parameters:	directory_path (str) – The exact path to the DAP reduction assessments file. See `directory_path`. dapver (str) – DAP version. analysis_path (str) – The path to the top-level directory containing the DAP output files for a given DRP and DAP version. output_file (str) – The name of the file with the reduction assessments. See `compute()`.

compute(drpf, pa=None, ell=None, dapver=None, analysis_path=None, directory_path=None, output_file=None, hardcopy=True, symlink_dir=None, clobber=False, loggers=None, quiet=False)[source]¶

Compute and output the main data products. The list of HDUs are:

PRIMARY : Empty apart from the header information.

SPECTRUM : Extension with the main, per-spectrum measurements; see below.

CORREL : The correlation matrix between the SIGNAL measurements provided in the SPECTRUM extension. The format of this extension is identical to the nominal output of the mangadap.util.covariance.Covariance object; see mangadap.util.covariance.Covariance.write().

The SPECTRUM extension contains the following columns:

DRP_INDEX : Array coordinates in the DRP file; see mangadap.drpfits.DRPFits.spatial_index. For RSS files, this is a single integer; for CUBE files, it is a vector of two integers.

SKY_COO : On-sky X and Y coordinates. Coordinates are sky-right offsets from the object center; i.e., positive X is along the direction of positive right ascension. See mangadap.drpfits.DRPFits.mean_sky_coordinates.

ELL_COO : Elliptical (semi-major axis) radius and azimuth angle from N through East with respect to the photometric position angle; based on the provided ellipticity parameters. See mangadap.util.geometry.SemiMajorAxisCoo.

FGOODPIX : Fraction of good pixels in each spectrum.

MINEQMAX : Flag that min(flux) = max(flux) in the spectrum; i.e., the spaxel has no data.

SIGNAL, VARIANCE, SNR : Per pixel means of the flux, flux variance, and signal-to-noise. The VARIANCE and SNR columns use the inverse variance provided by the DRP. See mangadap.drpfits.DRPFits.flux_stats().

Parameters:

drpf (mangadap.drpfits.DRPFits) – DRP file (object) to use for the assessments.
pa (float) – (Optional) On-sky position angle of the major axis used to calculate elliptical, semi-major-axis coordinates, defined as the angle from North through East and denoted $\phi_0$. Default is 0.0.
ell (float) – (Optional) Ellipticity defined as $\varepsilon=1-b/a$, based on the semi-minor to semi-major axis ratio ($b/a$) of the isophotal ellipse used to calculate elliptical, semi-major-axis coordinates. Default is 0.0.
dapver (str) – (Optional) DAP version, which is used to define the default DAP analysis path. Default is defined by mangadap.config.defaults.default_dap_version()
analysis_path (str) – (Optional) The path to the top level directory containing the DAP output files for a given DRP and DAP version. Default is defined by mangadap.config.defaults.default_analysis_path().
directory_path (str) – (Optional) The exact path for the output file. Default is defined by mangadap.config.defaults.default_dap_common_path().
output_file (str) – (Optional) The name of the file for the computed assessments. The full path of the output file will be directory_path/output_file. Default is defined by mangadap.config.defaults.default_reduction_assessments_file().
hardcopy (bool) – (Optional) Flag to write the data to a fits file. Default is True.
symlink_dir (str) – (Optional) Create a symlink to the file in this directory. Default is for no symlink.
clobber (bool) – (Optional) If the output file already exists, this will force the assessments to be redone and the output file to be overwritten. Default is False.
loggers (list) – (Optional) List of logging.Logger objects to log progress; ignored if quiet=True. Default is no logging.
quiet (bool) – (Optional) Suppress all terminal and logging output. Default is False.

Raises:

ValueError – Raise if no DRPFits object is provided or if the output file is undefined.

static default_paths(plate, ifudesign, method_key, directory_path=None, drpver=None, dapver=None, analysis_path=None, output_file=None)[source]¶

Set the default directory and file name for the output file.

Parameters:	plate (int) – Plate number of the observation. ifudesign (int) – IFU design number of the observation. method_key (str) – Keyword designating the method used for the reduction assessments. directory_path (str) – (Optional) The exact path to the DAP reduction assessments file. Default set by `mangadap.config.defaults.default_dap_common_path()`. drpver (str) – (Optional) DRP version. Default set by `mangadap.config.defaults.default_drp_version()`. dapver (str) – (Optional) DAP version. Default set by `mangadap.config.defaults.default_dap_version()`. analysis_path (str) – (Optional) The path to the top-level directory containing the DAP output files for a given DRP and DAP version. Default set by `mangadap.config.defaults.default_analysis_path()`. output_file (str) – (Optional) The name of the file with the reduction assessments. Default set by `mangadap.config.defaults.default_dap_file_name()`.
Returns:	Two strings with the path for the output file and the name of the output file.
Return type:	str

file_name()[source]¶: Return the name of the output file.

file_path()[source]¶: Return the full path to the output file.

info()[source]¶

class mangadap.proc.reductionassessments.ReductionAssessmentDef(key, waverange=None, response_func=None, covariance=False)[source]¶

Bases: mangadap.par.parset.ParSet

Class with parameters used to define how the reduction assessments are performed. At the moment this is just a set of parameters that define how the S/N is calculated.

See mangadap.par.parset.ParSet for attributes.

Todo

Allow for different ways of calculating covariance?

Parameters:

key (str) – Keyword to distinguish the assessment method.
waverange (numpy.ndarray, list) – A two-element vector with the starting and ending wavelength (angstroms in VACUUM) within which to calculate the signal-to-noise
response_func (array-like) – A two-column array with a response function to use for the S/N calculation. The columns must br the wavelength and amplitude of the response function, respectively.
covariance (str) – Type of covariance measurement to produce

mangadap.proc.reductionassessments.available_reduction_assessments(dapsrc=None)[source]¶

Return the list of available reduction assessment methods. To get a list of default methods provided by the DAP do:

from mangadap.proc.reductionassessments import available_reduction_assessments
rdx_methods = available_reduction_assessments()
print(rdx_methods)

Each element in the rdx_methods list is an instance of ReductionAssessmentDef, which is printed using the ParSet base class representation function.

New methods can be included by adding ini config files to $MANGADAP_DIR/python/mangadap/config/reduction_assessments. See an example file at $MANGADAP_DIR/python/mangadap/config/example_ini/example_reduction_assessment_config.ini.

Parameters:	dapsrc (str) – (Optional) Root path to the DAP source directory (i.e., $MANGADAP_DIR). If not provided, the default is defined by `mangadap.config.defaults.dap_source_dir()`.
Returns:	A list of `ReductionAssessmentDef()` objects, each defining a separate assessment method.
Return type:	list
Raises:	`NotADirectoryError` – Raised if the provided or default dapsrc is not a directory. `OSError/IOError` – Raised if no reduction assessment configuration files could be found. `KeyError` – Raised if the assessment method keywords are not all unique. `NameError` – Raised if either ConfigParser or ExtendedInterpolation are not correctly imported. The latter is a Python 3 only module!

mangadap.proc.reductionassessments.validate_reduction_assessment_config(cnfg)[source]¶

Validate the mangadap.util.parser.DefaultConfig object that provides the reduction-assessment method parameters.

Parameters:	cnfg (`mangadap.util.parser.DefaultConfig`) – Object with the reduction-assessment method parameters needed by `mangadap.proc.reductionassessments.ReductionAssessmentDef`.
Returns:	Booleans that specify how the reduction assessment should be constructed. The flags specify to use (1) the wavelength range, (2) a bandpass filter parameter file, or (3) a file with a filter response function.
Return type:	bool
Raises:	`KeyError` – Raised if required keyword does not exist. `ValueError` – Raised if keys have unacceptable values. `FileNotFoundError` – Raised if a file is specified but could not be found.