Astropy: Tables

Documentation

For more information about the features presented below, you can read the astropy.table docs.

Creating tables

In [1]:
import numpy as np
from astropy.table import Table
In [2]:
t1 = Table()
t1['name'] = ['source 1', 'source 2', 'source 3']
t1['flux'] = [1.2, 2.2, 3.1]
In [3]:
t1
Out[3]:
<Table length=3>
nameflux
str8float64
source 11.2
source 22.2
source 33.1
In [4]:
print(t1)
  name   flux
-------- ----
source 1  1.2
source 2  2.2
source 3  3.1
In [5]:
t1['size'] = [1,5,4]
t1
Out[5]:
<Table length=3>
namefluxsize
str8float64int64
source 11.21
source 22.25
source 33.14
In [6]:
t1['size']
Out[6]:
<Column name='size' dtype='int64' length=3>
1
5
4
In [7]:
np.array(t1['size'])
Out[7]:
array([1, 5, 4])
In [8]:
t1['size'][0]
Out[8]:
1

Iterating over tables

It is possible to iterate over rows or over columns. To iterate over rows, simply iterate over the table itself:

In [9]:
for row in t1:
    print(row)
  name   flux size
-------- ---- ----
source 1  1.2    1
  name   flux size
-------- ---- ----
source 2  2.2    5
  name   flux size
-------- ---- ----
source 3  3.1    4

Rows can act like dictionaries, so you can access specific columns from a row:

In [10]:
for row in t1:
    print(row['name'])
source 1
source 2
source 3

You can also access rows by accessing a numerical item in the table:

In [11]:
row = t1[0]

Iterating over columns is also easy:

In [12]:
for colname in t1.columns:
    column = t1[colname]
    print(column)
  name  
--------
source 1
source 2
source 3
flux
----
 1.2
 2.2
 3.1
size
----
   1
   5
   4

Accessing specific rows from a column object can also be done with the item notation:

In [13]:
for colname in t1.columns:
    column = t1[colname]
    print(column[0])
source 1
1.2
1

Joining tables

In [14]:
from astropy.table import join
In [15]:
t2 = Table()
t2['name'] = ['source 1', 'source 3']
t2['flux2'] = [1,9]
In [16]:
t3 = join(t1, t2, join_type='outer')
t3
Out[16]:
<Table masked=True length=3>
namefluxsizeflux2
str8float64int64int64
source 11.211
source 22.25--
source 33.149
In [17]:
np.mean(t3['flux2'])
Out[17]:
5.0

Grouping and Aggregation

In [18]:
from astropy.table import Table
obs = Table.read("""name    obs_date    mag_b  mag_v
                    M31     2012-01-02  17.0   17.5
                    M31     2012-01-02  17.1   17.4
                    M101    2012-01-02  15.1   13.5
                    M82     2012-02-14  16.2   14.5
                    M31     2012-02-14  16.9   17.3
                    M82     2012-02-14  15.2   15.5
                    M101    2012-02-14  15.0   13.6
                    M82     2012-03-26  15.7   16.5
                    M101    2012-03-26  15.1   13.5
                    M101    2012-03-26  14.8   14.3
                    """, format='ascii')
In [19]:
obs_by_name = obs.group_by('name')
In [20]:
obs_by_name
Out[20]:
<Table length=10>
nameobs_datemag_bmag_v
str4str10float64float64
M1012012-01-0215.113.5
M1012012-02-1415.013.6
M1012012-03-2615.113.5
M1012012-03-2614.814.3
M312012-01-0217.017.5
M312012-01-0217.117.4
M312012-02-1416.917.3
M822012-02-1416.214.5
M822012-02-1415.215.5
M822012-03-2615.716.5
In [21]:
for group in obs_by_name.groups:
    print(group)
    print("")
name  obs_date  mag_b mag_v
---- ---------- ----- -----
M101 2012-01-02  15.1  13.5
M101 2012-02-14  15.0  13.6
M101 2012-03-26  15.1  13.5
M101 2012-03-26  14.8  14.3

name  obs_date  mag_b mag_v
---- ---------- ----- -----
 M31 2012-01-02  17.0  17.5
 M31 2012-01-02  17.1  17.4
 M31 2012-02-14  16.9  17.3

name  obs_date  mag_b mag_v
---- ---------- ----- -----
 M82 2012-02-14  16.2  14.5
 M82 2012-02-14  15.2  15.5
 M82 2012-03-26  15.7  16.5

In [22]:
obs_by_name.groups.aggregate(np.mean)
WARNING: Cannot aggregate column 'obs_date' with type '<U10' [astropy.table.groups]
Out[22]:
<Table length=3>
namemag_bmag_v
str4float64float64
M10115.013.725
M3117.017.4
M8215.715.5

Masked tables

In [23]:
t4 = Table(masked=True)
t4['name'] = ['source 1', 'source 2', 'source 3']
t4['flux'] = [1.2, 2.2, 3.1]
In [24]:
t4['flux'].mask = [1,0,1]
t4
Out[24]:
<Table masked=True length=3>
nameflux
str8float64
source 1--
source 22.2
source 3--

Writing data

In [25]:
t3.write('test.fits', overwrite=True)
In [26]:
t3.write('test.vot', format='votable', overwrite=True)
WARNING: W31: ?:?:?: W31: NaN given in an integral field without a specified null value [astropy.io.votable.converters]

Reading data

You can download the data used below here.

In [27]:
t4 = Table.read('data/2mass.tbl', format='ascii.ipac')
In [28]:
t4
Out[28]:
<Table masked=True length=929>
radecclonclaterr_majerr_minerr_angdesignationj_mj_cmsigj_msigcomj_snrh_mh_cmsigh_msigcomh_snrk_mk_cmsigk_msigcomk_snrph_qualrd_flgbl_flgcc_flgndetgal_contammp_flgdistanglej_hh_kj_k
degdegarcsecarcsecdegmagmagmagmagmagmagmagmagmag
float64float64str12str13float64float64int64str16float64float64float64float64float64float64float64float64float64float64float64float64str3str3str3str3str6int64int64float64float64float64float64float64
274.429506-13.87054718h17m43.08s-13d52m13.97s0.080.084518174308-135213916.3050.1420.1436.714.0480.1070.10813.613.2570.0660.06616.5CAA2221110ss06665500975.080151256.4482.2570.7913.048
274.423821-13.8697418h17m41.72s-13d52m11.06s0.060.069018174171-135211014.8020.0580.05926.712.6350.0590.0650.111.7680.0450.04665.2AAA2221110ss66666600993.752042256.8782.1670.8673.034
274.424587-13.73962918h17m41.90s-13d44m22.66s0.080.084518174190-134422616.328------14.3450.0590.0610.413.4050.0460.04714.4UAA0220110cc00366600995.726698284.113--0.94--
274.433933-13.76950218h17m44.14s-13d46m10.21s0.080.084518174414-134610216.2810.0980.0996.814.0570.0350.03613.512.9560.0320.03321.8CAA22211100006556600942.627418278.2522.2241.1013.325
274.437013-13.88569818h17m44.88s-13d53m08.51s0.090.094518174488-135308515.171------14.4120.1520.1529.813.7420.0950.09510.6UBA6220220cc00556600964.105389252.93--0.67--
274.433996-13.75244618h17m44.16s-13d45m08.81s0.080.089018174415-134508816.54------14.5190.0830.0838.813.6040.0430.04412.0UBA0220110cc00566600953.230532281.908--0.915--
274.418138-13.7721518h17m40.35s-13d46m19.74s0.080.089018174035-134619717.98------14.610.0430.0448.113.4560.0560.05713.8UBA02201100000164500996.047248277.25--1.154--
274.433695-13.89904918h17m44.09s-13d53m56.58s0.060.069018174408-135356513.0110.0210.024139.010.9170.020.021243.810.0130.0170.019328.3AAA22211100066666600990.166399250.4662.0940.9042.998
274.425482-13.7714918h17m42.12s-13d46m17.36s0.080.0813518174211-134617316.086------13.7090.0650.06618.612.5030.0440.04533.1UAA62201200c00555500970.896919277.582--1.206--
................................................................................................
274.81801-14.00124518h19m16.32s-14d00m04.48s0.180.16118191632-140004416.240.1130.1135.615.5310.1640.1642.515.252------CDU22011000006060000809.817146149.610.709----
274.822709-14.03725418h19m17.45s-14d02m14.11s0.070.074518191745-140214115.9990.0970.0987.014.0090.0320.03310.013.0770.0350.03616.4CAA22211100006265600931.339773152.7791.990.9322.922
274.880758-13.9995618h19m31.38s-13d59m58.42s0.060.069018193138-135958414.1630.0350.03737.811.1790.020.021135.69.7650.0170.019347.1AAA22211100055666600935.512452137.7622.9841.4144.398
274.652526-14.05510618h18m36.61s-14d03m18.38s0.060.069018183660-140318315.0350.0520.05419.413.0990.040.04127.512.2540.0410.04141.7AAA222111c0056666600908.109808190.6821.9360.8452.781
274.760586-13.99992718h19m02.54s-13d59m59.74s0.080.089018190254-135959716.3290.1220.1235.514.4880.0670.0676.413.6170.0510.05211.1CCA22211100006061600724.557553163.2271.8410.8712.712
274.831132-14.02002718h19m19.47s-14d01m12.10s0.080.084518191947-140112016.203------13.2380.020.02120.412.0160.0230.02443.6UAA02201100000666600891.347132149.27--1.222--
274.972435-13.76037418h19m53.38s-13d45m37.35s0.120.111018195338-134537317.472------16.755------14.4130.0840.0844.8UUD00200100000000600964.82893379.963------
274.870009-13.81777518h19m28.80s-13d49m03.99s0.080.084518192880-134903916.933------14.5140.0640.0656.312.9570.0410.04118.4UCA02201100000266600592.99805893.69--1.557--
274.735323-13.94157518h18m56.48s-13d56m29.67s0.140.144518185647-135629616.643------14.88------14.2910.1160.1176.0UUC00200100000000400498.524438165.968------
274.866294-13.84177818h19m27.91s-13d50m30.40s0.080.084518192791-135030415.615------13.9110.0750.07510.912.7650.1340.13421.9UAE0220110cc00554500591.97725102.147--1.146--

Practical Exercises

These exercises use the all-sky ROSAT catalog which you can download from here.

Level 1

Try and find a way to make a table of the ROSAT point source catalog that contains only the RA, Dec, and count rate. Hint: you can see what methods are available on an object by typing e.g. t. and then pressing <TAB>. You can also find help on a method by typing e.g. t.remove_column?.

Level 2

Make an all-sky equatorial plot of the ROSAT sources, with all sources shown in black, and only the sources with a count rate larger than 2. shown in red (bonus points if you use an Aitoff projection!)

Level 3

Try and write out the ROSAT catalog into a format that you can read into another software package. For example, try and write out the catalog into CSV format, then read it into a spreadsheet software package (e.g. Excel, Google Docs, Numbers, OpenOffice).