How do I use the MaNGA Catalog DRPall File?
The DRPall file
The DRPall file is a summary file from the MaNGA Data Reduction Pipeline (DRP). This table contains target and observational information (e.g., number of exposures, IFU size, identifiers), as well as galaxies properties taken from the NSA catalog, such as redshift and photometry for all the galaxies in the current data release. More information about this file can be found here. The datamodel for the DRPall file can be found here.
Below we give examples of how to access and use the DRPall data with python, IDL, Marvin, and CAS. The python examples have been tested with python version 3.7 and IDL with version 8.8.
Accessing the DRPall file with Python
import numpy as np #importing the needed packages
from astropy.io import fits
drpall = fits.open('drpall-v3_1_1.fits') #assumes you are in the same directory as the DRPall file
tbdata = drpall[1].data
# Print column names
tbdata.columns.names
Find the redshift of a galaxy:
In this example, we will find the redshift for a galaxy with the MaNGA ID 12-193481, which just happens to be the galaxy used in the other tutorials. In the DRPall file, there is a column called 'mangaid' that contains the MaNGA IDs for all the observed MaNGA galaxies. The redshift in DRPall comes from the NSA catalog and is the column labelled 'nsa_z'.
ind = np.where(tbdata['mangaid'] == '12-193481') #finding the index for our galaxy using the MaNGA ID
print('redshift =', tbdata['nsa_z'][ind][0]) #printing the redshift of this index corresponding to our galaxy
# should print 'redshift = 0.0402719'
Find galaxies that match a specific ancillary catalog:
Now we will find which galaxies in DRPall were part of a specific ancillary catalog. For this we use the MaNGA Target 3 bitmask (column name 'mngtarg3') which is described and has a table with the different bits listed here. We are interested in the Dwarf Galaxies, which has the bitname 'Dwarf' and bit number '14'.
# select the column for the MaNGA Target 3 bitmask targets = tbdata['mngtarg3'] # select rows from the Dwarf Galaxies with MaNGA ancillary catalog (which is bit 14) dwarf_indices = np.where(targets & 1 << 14)[0] print(dwarf_indices) # Result: #[ 3921 4134 7009 7104 7121 7260 7276 7668 7760 7793 7794 8072 # 8280 8447 8522 8796 8810 8871 8905 8907 9007 9061 9417 9602 # 9757 10134 10255 10283 10284 10419 10506]
Identifying unique galaxies:
Here we will find the unique galaxies observed with MaNGA. Please note this caveat about some duplicate galaxies with different MaNGA IDs. We will use the MaNGA Target bitmasks 1 and 3 (mngtarg1 and mngtarg3)
# Find all main and ancillary target data cubes
cube_bools = (tbdata['mngtarg1'] != 0) | (tbdata['mngtarg3'] != 0)
cubes = tbdata[cube_bools]
# Find galaxies excluding those from the Coma, IC342, M31, and globular cluster ancillary programs (bits 19,20,21,27)
targ3 = tbdata['mngtarg3']
galaxies = tbdata[cube_bools & ((targ3 & 1<<19) == 0) & ((targ3 & 1<<20) == 0) & ((targ3 & 1<<21) == 0) & ((targ3 & 1<<27) == 0) ]
print('Number of galaxies', len(galaxies))
# Result: Number of galaxies 10296
# Get unique galaxies
uniq_vals, uniq_idx=np.unique(galaxies['mangaid'], return_index=True)
uniq_galaxies = galaxies[uniq_idx]
print('Unique galaxies', len(uniq_galaxies))
# Result: Unique galaxies 10160
Note that this number of unique galaxies does not take into account the data quality. Once galaxies with DRPQUAL3 flags of critical or unusual are removed the number of unique, high quality galaxies in DR17 is 10010.
Accessing the DRPall file with IDL
The DRPall file can be read in using, for example, MRDFITS. This will put the data into an IDL structure.
drpall=mrdfits('drpall-v3_1_1.fits',1,hdr) ;Binary table with 99 columns and 11273 rows
; Print column names
help,drpall,/str
Find the redshift of a galaxy:
In this example, we will find the redshift for a galaxy with the MaNGA ID 12-193481, which just happens to be the galaxy used in the other tutorials. In the DRPall file, there is a column called 'mangaid' that contains the MaNGA IDs for all the observed MaNGA galaxies. The redshift in DRPall comes from the NSA catalog and is the column labelled 'nsa_z'.
ind = where(drpall.mangaid eq '12-193481') ;finding the index of our galaxy print,'redshift =', drpall[ind].nsa_z ;should print 'redshift = 0.040271900'
Identifying unique galaxies:
Here we will find the unique galaxies observed with MaNGA. Please note this caveat about some duplicate galaxies with different MaNGA IDs. We will use the MaNGA Target bitmasks 1 and 3 (mngtarg1 and mngtarg3)
; Find all main and ancillary target data cubes cube_bools = (drpall.mngtarg1 ne 0) or (drpall.mngtarg3 ne 0) ; Trim to only these data cubes cubes = drpall[where(cube_bools)] print,'Number of data cubes: ', n_elements(cubes) ;Result: 11236 ; Find galaxies excluding those from the Coma, IC342, M31, and globular cluster ancillary programs (bits 19,20,21,27) galaxies = drpall[where(cube_bools and ((drpall.mngtarg3 and 2L^19L+2L^20L+2L^21+2L^27) eq 0))] ; Select unique galaxies mangaid = galaxies.mangaid uqmangaid = mangaid[uniq(mangaid[sort(mangaid)])] print,'Number of unique galaxies: ', n_elements(uqmangaid) ;Result: 10160
Note that this number of unique galaxies does not take into account the data quality. Once galaxies with DRPQUAL3 flags of critical or unusual are removed the number of unique, high quality galaxies in DR17 is 10010.
Accessing the DRPall file with Marvin
Select Main Sample Galaxies
To select the Main Sample galaxies (Primary + Secondary + Color Enhanced samples), we need to download the DR17 drpall file file and put it in the expected $SAS_BASE_DIR directory. For more information on the location of your SAS_BASE_DIR environment variable, see the Marvin documentation.
Let’s open the DRPall file:
from marvin import config
from marvin.utils.general.general import get_drpall_table
config.setRelease('DR17')
data = get_drpall_table()
The Main Sample consists of the Primary, Secondary, and Color-Enhanced Samples, which correspond to MNGTARG1 bits 10, 11, and 12, respectively.
import numpy as np primary = data['mngtarg1'] & 2**10 secondary = data['mngtarg1'] & 2**11 color_enhanced = data['mngtarg1'] & 2**12 main_sample = np.logical_or.reduce((primary, secondary, color_enhanced)) plateifus = data['plateifu'][main_sample]
Accessing the DRPall file with CAS
The DRPall file exists as a table in Catalog Archive Server (CAS). You can query the DRPall table under the Query tab in CASjobs. For example, to find all galaxies with redshift > 0.05 and stellar mass <1e10:
SELECT mangaid, objra, objdec, nsa_z, nsa_sersic_mass FROM dbo.mangaDrpAll where nsa_z > 0.05 and nsa_sersic_mass < 1e10
