mangadap.survey.drpcomplete module

Defines a class used to produce and provide an interface for data required to create The DAP Datacube Configuration File.

The drpcomplete file/data is primarily required for the survey-level Execution of the DAP. However, it does provide useful information regarding the completed DRP files.

class mangadap.survey.drpcomplete.DRPComplete(platelist=None, ifudesignlist=None, drpall=None, drpver=None, redux_path=None, dapver=None, analysis_path=None, directory_path=None, readonly=False)[source]

Bases: object

Database with information needed by the DAP to analyze the completed DRP files.

This class searches the defined paths for files resulting from the 3D phase of the MaNGA DRP, and then collates the information necessary to run those files through the MaNGA DAP. The necessary metadata are pulled from the DRPall file; see update().

Parameters:

platelist (str, list, optional) – List of plates to search for. Default is to search the full DRP path.
ifudesignlist (str, list, optional) – List of IFUs to search for. Default is to search the full DRP path.
drpall (str, optional) – The full path to the DRPall fits file. Default is set by mangadap.config.manga.drpall_file().
drpver (str, optional) –
DRP version, which is:
- used to define the default DRP redux path
- used in the name of the drpcomplete fits file
- included as a header keyword in the output file
Default is defined by mangadap.config.manga.drp_version().
redux_path (str, optional) – The path to the top-level directory containing the DRP output files. Default is defined by mangadap.config.manga.drp_redux_path().
dapver (str, optional) –
DAP version, which is:
- used to define the default DAP analysis path
- included as a header keyword in the output drpcomplete fits file
Default is defined by mangadap.config.manga.dap_version().
analysis_path (str, optional) – The path to the top level directory for the DAP output files. Default is defined by mangadap.config.manga.dap_analysis_path()
directory_path (str, optional) – Direct path for the output file. Default is a directory called common within the analysis_path directory.
readonly (bool, optional) – Flag that the drpcomplete fits file is only opened for reading, not for updating. If True, many of the attributes will be set to None.

platelist

List of plates to search for, see above.

Type:: list

ifudesignlist

List of IFU designs to search for, see above.

Type:: list

drpall

The DRPall file, see above.

Type:: str

drpver

DRP version, see above.

Type:: str

redux_path

Path to the top level of directory for the DRP output, see above.

Type:: Path

dapver

DAP version, see above.

Type:: str

analysis_path

Path to the top level directory for the DAP output files, see above.

Type:: Path

directory_path

Direct path for the output file.

Type:: Path

readonly

Flag that the drpcomplete fits file is only opened for reading, not for updating.

Type:: bool

hdu

Fits data with binary table data.

Type:: astropy.io.fits.HDUList

nobs

Number of observations in the file

Type:: int

_all_data_exists()[source]

Determines if the data for all the plates/ifudesigns selected in the current compilation is already present in the current drpcomplete fits file.

Returns:: Flag that all data has already been collated for the requested plate/ifudesign list.
Return type:: bool

_confirm_access(reread=False)[source]

Check the drpcomplete fits file at file_path() is accessible, and read the data if not yet read.

Parameters:: reread (bool, optional) – Force the file to be re-read
Returns:: Flag that hdu is read and valid.
Return type:: bool

_find_completed_reductions(mindesign=19, combinatorics=False, on_disk=False)[source]

Search the DRP path for reduced CUBE files.

Function allows for one or both of platelist and ifudesignlist to be None. The behavior is:

If both are None, all available CUBE files are used to create platelist and ifudesignlist.

If one is None, all available CUBE files within the provided list of the one that is not None are used to create platelist and ifudesignlist. E.g., if platelist =[7443] and ifudesignlist is None, all CUBE files with plate=7443 are chosen.

If both are not None and they have different lengths, or the same length and combinatorics is True, all available CUBE files within the constraints of both lists are selected. E.g., if platelist =[7443,7495] and ifudesignlist =[12704], the CUBE files with (plate,ifudesign)=[(7443,12704),(7459,12704)] are chosen.

If both are not None and they have the same length and combinatorics is False, all available CUBE files in the matched lists are chosen. E.g. if platelist =[7443,7495] and ifudesignlist =[12704,12703], the CUBE files with (plate,ifudesign)=[(7443,12704),(7459,12703) are chosen

Parameters:

mindesign (int, optional) – Minimum bundle design to consider. For example, to ignore all the 7-fiber bundles, set mindesign=19 to only select the bundles with 19 or more fibers.
combinatorics (bool, optional) – Create platelist and ifudesignlist by determining all possible combinations of the input values. See above.
on_disk (bool, optional) – When searching for available files to analyze, search the DRP directory path instead of using the data in the DRPall file.

Returns:

Two lists with the available plates and ifudesigns for which to collect data.

Return type:

tuple

_find_modes()[source]

Using the provided list of CUBE DRP files, find if the RSS mode is also available.

Warning

This assumes everything in platelist and ifudesignlist has a ‘CUBE’ file.

Currently only two modes are possible:

only ‘CUBE’ file are available

both ‘CUBE’ and ‘RSS’ files are available

Returns:: Array of modes for each input DRP file.
Return type:: numpy.ndarray

_match_drpall()[source]

Find the data needed for the DRP complete database in the DRPall file.

If a plate-ifudesign combination is not found in the DRPall file, the other parameter values are set to -9999 and the MaNGA ID is set to NULL.

If the plate-ifudesign combination is found, the columns in the plateTargets files that the code expects to find are 'plate', 'ifudsgn', 'mangaid', 'objra', 'objdec'; the values that will be replaced with 'NULL' (str) or -9999 (int or float) if they do not exist are 'nsa_version', 'nsa_nsaid', 'mngtarg1', 'mngtarg3', 'z', 'nsa_elpetro_ba', 'nsa_elpetro_phi', 'nsa_elpetro_th50_r', 'nsa_vdisp'.

Returns:: 14 numpy.ndarray arrays with the MaNGA ID, object right ascension, object declination, catalog ID, index of the entry in the catalog, catalog version, ID of object in the catalog, main MaNGA survey target bitmask, ancillary MaNGA survey target bitmask, velocity, velocity dispersion, ellipticity, position angle, and effective radius.
Return type:: tuple

_read()[source]: Read the data in the existing file at file_path().

_read_fix_data(fix_file)[source]

Read “fix” data from the provided file.

Fix data are generally corrections to the metadata provides by the NSA for specific MaNGA observations.

Parameters:: fix_file (str) – SDSS parameter file with the fix data.
Returns:: A yanny structure with the data fixes. Returns None and raises a warning if the file does not exist.
Return type:: yanny

_read_photometry_fix()[source]: Wrapper for _read_fix_data() and photometry_fix_file() that returns the photometry fix data.

_read_redshift_fix()[source]: Wrapper for _read_fix_data() and redshift_fix_file() that returns the redshift fix data.

can_analyze(row=None)[source]

Determine if the DAP can analyze a plate-ifu entry in the database.

The selection is currently:

MaNGAID != ‘NULL’

MANGA_TARGET1 > 0 or MANGA_TARGET3 > 0

VEL > -500

Parameters:: row (int, optional) – The specific row to test. By default, return a boolean vector for all the database rows.
Returns:: Either a single boolean or boolean numpy.ndarray flagging that DAP can (True) or cannot (False) analyze the data associated with the database entry (or entries).

entry_index(plate, ifudesign, reread=False)[source]

Find the index of the row with the parameter data for the specified plate and ifudesign.

Warning

This is very inefficient if you’re looking for multiple entries…

Parameters:

plate (int) – Plate number
ifudesign (int) – IFU design
reread (bool, optional) – Force the database to be re-read

Returns:

Index of the row in data with the data for the given plate and ifudesign.

Return type:

int

Raises:

ValueError – Raised if the given plate and ifudesign were not found.

property file_name: Return the name of the DRP complete database.

property file_path: Return the full pat to the DRP complete database.

update(platelist=None, ifudesignlist=None, combinatorics=False, force=False, alldrp=False, on_disk=False, quiet=False)[source]

Update the DRP complete file.

If platelist and/or ifudesignlist are provided, the existing platelist and ifudesignlist attributes are replaced.

If platelist and ifudesignlist do not have the same length or combinatorics is True, the lists are expanded to include all combinations of their elements.

If platelist and ifudesignlist are None or alldrp is True, all available plates/ifudesigns are collected from within the DRP directory structure.

If the result of file_path() does not exist, it is created.

If the result of file_path() does exist, the available plates and ifudesigns in the file are compared against the list (provided or collected) to update. If the lists differ, the drpcomplete fits file is re-created from scratch. If all the plates and ifudesigns are available, nothing is done, unless force=True.

Parameters:

platelist (str, list, optional) – List of plates to include in the drpcomplete fits file.
ifudesignlist (str, list, optional) – List of ifudesigns to include in the drpcomplete fits file.
combinatorics (bool, optional) – Determine all combinations of the entered plates and ifudesigns.
force (bool, optional) – Overwrite any existing drpcomplete fits file with a new one built from scratch.
alldrp (bool, optional) – Find the full list of available DRP files.
on_disk (bool, optional) – When searching for available files to analyze, search the DRP directory path instead of using the data in the DRPall file.
quiet (bool, optional) – Suppress terminal output

Raises:

ValueError – Raised if drpcomplete fits file was opened in read-only mode.

write(platelist, ifudesignlist, modes, mangaid, objra, objdec, catid, catindx, trg_version, trg_id, manga_trg1, manga_trg3, vel, veldisp, ell, pa, Reff, drpver=None, redux_path=None, dapver=None, analysis_path=None, overwrite=True)[source]

Write the drpcomplete fits binary table.

Header keywords are:

VERSDRP: DRP version

RDXPTH: DRP reduction path

VERSDAP: DAP version

SISPTH: DAP analysis path

Binary table columns:

PLATE: Plate number

IFUDESIGN: IFU design

MODES: Modes available

MODES=1: only ‘CUBE’ file is available

MODES=2: both ‘CUBE’ and ‘RSS’ files are available

MANGAID: MaNGA ID (same as ‘CATID-CATINDX’)

OBJRA: Object right ascension

OBJDEC: Object declination

CATID: Catalog ID used for target selection

CATINDX: Index in catalog with target data

TRG_VERSION: Version of the catalog (e.g., NSA)
used in targetting (def=’NULL’ if not available)

TRG_ID: Target ID in catalog (def=-9999 if not available)

MANGA_TARGET1: Main MaNGA survey target bitmask

MANGA_TARGET3: Ancillary MaNGA survey target bitmask

VEL: Systemic velocity from catalog (def=-9999.0 if not available)

VDISP: Velocity dispersion from catalog (def=-9999.0 if not available)

ELL: Ellipticity (1-b/a) from catalog (def=-9999.0 if not available); Elliptical Petrosian value from NSA

PA: Position angle from catalog (def=-9999.0 if not available); Elliptical Petrosian value from NSA

REFF: Effective radius from catalog (def=-9999.0 if not available); Elliptical Petrosian value from NSA

Parameters:

platelist (list) – List of plates
ifudesignlist (list) – List of IFU designs
modes (numpy.ndarray) – Mode values, see above
mangaid (numpy.ndarray) – MaNGA IDs
objra (numpy.ndarray`_) – Object right ascensions
objdec (numpy.ndarray`_) – Object declinations
catid (numpy.ndarray`_) – Catalog ID used for target selection
catindx (numpy.ndarray`_) – Index in catalog with target data
trg_version (numpy.ndarray`_) – Version of the catalog (e.g., NSA) used in targetting
trg_id (numpy.ndarray`_) – Target ID in catalog
manga_trg1 (numpy.ndarray`_) – Main MaNGA survey target bitmask
manga_trg3 (numpy.ndarray`_) – Ancillary MaNGA survey target bitmask
vel (numpy.ndarray`_) – Velocity from catalog, if available
veldisp (numpy.ndarray`_) – Velocity dispersion from catalog, if available
ell (numpy.ndarray`_) – Ellipticity (1-b/a) from catalog, if available
pa (numpy.ndarray`_) – Position angle from catalog, if available
Reff (numpy.ndarray`_) – Effective radius from catalog, if available
drpver (str, optional) – DRP version, see above.
redux_path (Path, optional) – Path to the top level of directory for the DRP output, see above.
dapver (str, optional) – DAP version, see above.
analysis_path (Path, optional) – Path to the top level directory for the DAP output files, see above.
overwrite (bool, optional) – Overwrite any existing file.

Raises:

AttributeError – Raised if drpcomplete fits file was opened in read-only mode.
FileExistsError – Raised if the drpcomplete file exists and overwrite=False.

write_config(ofile, plate=None, ifudesign=None, index=None, sres_ext=None, sres_fill=None, covar_ext=None, reread=False, overwrite=True)[source]

Write a config file with the data used to instantiate a mangadap.datacube.manga.MaNGADataCube datacube for analysis.

Parameters:

ofile (str, Path) – Output file name.
plate (int, optional) – Plate number.
ifudesign (int, optional) – IFU design.
index (int, optional) – Index of the row in data with the data to return.
sres_ext (str, optional) – The extension to use when constructing the spectral resolution vectors for the MaNGA datacubes. See mangadap.datacube.manga.MaNGADataCube.spectral_resolution().
sres_fill (bool, optional) – Fill masked spectral-resolution data by simple linear interpolation.
covar_ext (str, optional) – Extension in the MaNGA DRP CUBE file to use as the single spatial correlation matrix for all wavelength channels.
reread (bool, optional) – Force the database to be re-read
overwrite (bool, optional) – Overwrite any existing parameter file

Raises:

IOError – Raised if the parameter file already exists and overwrite is False.
ValueError – Raised if - the row with the data is unknown because either index is not defined or one or both of plate and ifudesign is not defined. - index does not exist in the data array.