Abacus Cosmos

A suite of cosmological N-body simulations

Overview

We provide example Python 3 code to load the FoF and Rockstar halo catalogs and particle subsamples. The main file is Halos.py; we also provide Halotools.py, which is a thin wrapper around Halos.py that puts the catalogs in halotools format, where HODs can easily be applied.

All the code is available on the AbacusCosmos GitHub. Please file an issue there if you find a problem with the code.

Using these interfaces is not necessary; all of the data formats are documented in Data Specifications. But the example code might help you get started using the catalogs faster or help with implementing your own code to parse the catalogs.

The generally recommended usage is to clone the repository and add it to your PYTHONPATH environment variable. This will allow you to import AbacusCosmos from anywhere. Typically, this will involve placing a command like the following in your ~/.bashrc file:

export PYTHONPATH="/path/to/AbacusCosmos/:$PYTHONPATH"

Yuan, Eisenstein, & Garrison has also developed a decorated HOD package called GRAND-HOD that incorporates multiple HOD generalizations, such as assembly bias and velocity bias. The GRAND-HOD repository contains details on its usage and code examples.

Examples

The following examples are adapted from this Jupyter notebook.

Halos.py

from AbacusCosmos import Halos

import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np

/home/lgarrison/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters

Load a single halo Rockstar catalog with particle subsamples:

# The path to the catalog will be different on your system
cat = Halos.make_catalog_from_dir(dirname='/mnt/gosling2/bigsim_products/AbacusCosmos_1100box_products/AbacusCosmos_1100box_00_products/AbacusCosmos_1100box_00_rockstar_halos/z0.300',
                                  load_subsamples=True, load_pids=False)
halos = cat.halos
halo_1000 = halos[1000]
subsamples = cat.subsamples

for field in sorted(halo_1000.dtype.fields):
    print(field, ':', halo_1000[field])

A : [53.04891   5.399098 -9.398761]
A2 : [39.850292  6.907639 -7.766779]
J : [-1.9693516e+13  2.7946513e+13 -1.4426672e+13]
N : 155
N_SO : 983
Voff : 29.62147
Xoff : 22.592842
alt_N : [135 121  91  29]
alt_N_SO : [983 983 983  48]
alt_m : [5.0370770e+12 4.5147137e+12 3.3953630e+12 1.0820388e+12]
alt_m_SO : [3.6677385e+13 3.6677385e+13 3.6677385e+13 1.7909607e+12]
b_to_a : 0.46212736
b_to_a2 : 0.38154382
bulkvel : [ -663.75305    93.82869 -1657.28   ]
bullock_spin : 0.07133085
c_to_a : 0.33044723
c_to_a2 : 0.27497157
child_r : 0.43996406
corevel : [ -682.8728    108.45962 -1700.5497 ]
desc : 0
energy : -9.7564e+16
flags : 25
halfmass_radius : 154.01212
id : 298710
kin_to_pot : 0.7260747
klypin_rs : 49.53651
m : 5783311000000.0
m_SO : 36677385000000.0
m_pe_b : 5295078600000.0
m_pe_d : 3507298200000.0
mgrav : 4775895600000.0
min_bulkvel_err : 984.3729
min_pos_err : 0.00025078392
min_vel_err : 1050.1628
n_core : 98
num_child_particles : 206
num_p : 206
p_start : 73900406
parent_id : 298727
pos : [426.93628   15.228214 237.38449 ]
r : 375.43213
rs : 55.43358
rvmax : 237.90572
spin : 0.054060824
subsamp_len : 18
subsamp_start : 40967
vel : [ -682.8728    108.45962 -1700.5497 ]
vmax : 303.44504
vmax_r : 0.43149778
vrms : 326.53113

The subsample particles should be near the halo location of [426.93628 15.228214 237.38449 ], and indeed they are:

particles_for_halo_1000 = subsamples[halo_1000['subsamp_start']:halo_1000['subsamp_start']+halo_1000['subsamp_len']]
print(particles_for_halo_1000['pos'])

[[426.9143     15.245714  237.90285  ]
 [427.37714    15.368571  237.59572  ]
 [427.23428    15.095714  237.58429  ]
 [426.94427    15.0671425 237.53714  ]
 [426.8857     15.225714  237.47571  ]
 [426.9057     15.158571  237.44286  ]
 [426.75427    15.267143  237.44142  ]
 [427.12286    15.108572  237.42     ]
 [427.1643     15.288571  237.35858  ]
 [426.79858    15.330001  237.34572  ]
 [426.9586     15.265714  237.34143  ]
 [426.9943     15.207143  237.32713  ]
 [426.67285    15.554285  237.28142  ]
 [426.94858    15.187143  237.28     ]
 [427.09717    15.387142  237.17857  ]
 [426.53857    15.372857  237.11143  ]
 [426.64716    15.35      236.78427  ]
 [427.06       15.085714  236.72856  ]]

Filter out subhalos:

Note that the above halo is actually a subhalo since its parent_id is not -1. Let’s see what fraction of Rockstar halos are actually subhalos:

(halos['parent_id'] != -1).mean()

0.09474006531180443

So about 10% are subhalos. For some analyses you might want to only include top-level parent halos, since Rockstar halo masses always include substructure mass:

print('# halos before subhalo filtering:', len(halos))
halos = halos[halos['parent_id'] == -1]
print('# halos after subhalo filtering:', len(halos))

# halos before subhalo filtering: 9565499
# halos after subhalo filtering: 8659263

Load all redshifts of a FoF catalog:

cats_by_z = Halos.make_catalogs(sim_name='emulator_1100box_planck_00',
                                products_dir='/mnt/gosling2/bigsim_products/emulator_1100box_planck_products/',
                                redshifts='all', load_phases=False, load_subsamples=False, halo_type='FoF')

We can plot the evolution of the halo mass function with redshift:

fig, ax = plt.subplots()
for z in sorted(cats_by_z.keys()):
    cat = cats_by_z[z]
    bin_edges, bin_centers, hist = Halos.halo_mass_hist(cat.halos[0]['N'])
    ax.loglog(bin_centers, hist, label='$z = {:.2f}$'.format(z))
ax.set_xlabel('Halo mass [# of particles]')
ax.set_ylabel('Number of halos')
ax.legend()

<matplotlib.legend.Legend at 0x7fe5c60e2ba8>

png

Halotools.py

from AbacusCosmos import Halotools
import halotools

import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np

Load two different cosmologies at redshift 0.1:

cats = Halotools.make_catalogs(sim_name='AbacusCosmos_720box', cosmologies=[0,1], redshifts=0.1,
                                products_dir='/mnt/gosling2/bigsim_products/AbacusCosmos_720box_products/',
                                phases=None, halo_type='Rockstar', load_halo_ptcl_catalog=False)

Generate mock galaxy catalogs for both:

for cat in cats:
    # First apply an arbitrary mass cut to make the example run faster
    cat.halo_table = cat.halo_table[cat.halo_table['halo_N'] >= 100]
    # Make an approximate concentration column for mock purposes
    cat.halo_table['halo_conc'] = cat.halo_table['halo_rvir'] / cat.halo_table['halo_klypin_rs']

from halotools.empirical_models import PrebuiltHodModelFactory
models = []
for cat in cats:
    model = PrebuiltHodModelFactory('zheng07', redshift=cats[0].redshift, concentration_key='halo_conc')
    model.populate_mock(cat)
    models += [model]

Compute the 2PCF on the mock galaxies:

import Corrfunc
bins = np.logspace(-1,1.5,25)
bin_centers = (bins[:-1] + bins[1:])/2.
tpcfs = []
for model in models:
    gals = model.mock.galaxy_table
    results = Corrfunc.theory.xi(X=gals['x'], Y=gals['y'], Z=gals['z'],
                            boxsize=model.mock.BoxSize, nthreads=4,
                            binfile=bins)
    tpcfs += [results]

plt.loglog(bin_centers, tpcfs[0]['xi'], label=cats[0].SimName)
plt.loglog(bin_centers, tpcfs[1]['xi'], label=cats[1].SimName)
plt.legend()
plt.xlabel(r'galaxy separation $s$ [Mpc/h]')
plt.ylabel(r'$\xi(s)$')

Text(0,0.5,'$\\xi(s)$')

png

cats = Halotools.make_catalogs(sim_name='AbacusCosmos_720box', cosmologies=[0,1], redshifts=0.1,
                                products_dir='/mnt/gosling2/bigsim_products/AbacusCosmos_720box_products/',
                                phases=None, halo_type='Rockstar', load_halo_ptcl_catalog=True)

halos = cats[1].halo_table
subsamples = cats[1].halo_ptcl_table
i = 999

halos[i]

<Row index=999>

halo_idhalo_corevel [3]halo_bulkvel [3]halo_m_SOhalo_child_rhalo_vmax_rhalo_mgravhalo_vmaxhalo_rvmaxhalo_rshalo_klypin_rshalo_vrmshalo_J [3]halo_energyhalo_spinhalo_alt_m [4]halo_alt_m_SO [4]halo_Xoffhalo_Voffhalo_b_to_ahalo_c_to_ahalo_A [3]halo_b_to_a2halo_c_to_a2halo_A2 [3]halo_bullock_spinhalo_kin_to_pothalo_m_pe_bhalo_m_pe_dhalo_halfmass_radiushalo_num_phalo_num_child_particleshalo_p_starthalo_deschalo_flagshalo_n_corehalo_subsamp_starthalo_subsamp_lenhalo_min_pos_errhalo_min_vel_errhalo_min_bulkvel_errhalo_Nhalo_alt_N [4]halo_N_SOhalo_alt_N_SO [4]halo_mvirhalo_rvirhalo_xhalo_yhalo_zhalo_vxhalo_vyhalo_vzhalo_upidhalo_hostid
int64float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32int64int64int64int64int64int64int64int64float32float32float32int32int32int32int32float32float32float32float32float32float32float32float32int64int64
569976-152.74799 .. -599.93256-152.74799 .. -599.932569653903000000.00.000278078920.26705223918326100000.0195.099290.0342442020.0133995790.013399579203.21617-592084000000.0 .. 550002360000.0-7822789500000000.00.02085905918326100000.0 .. 390288570000.09653903000000.0 .. 528037500000.013.8211318.2015190.65585090.51455605-1.9109313 .. 25.466890.94717130.5979646414.867451 .. -12.6810510.0234053450.70826471339113500000.0390288570000.00.059088998207207327698500255350850174.5048728e-05751.38617751.3861712680 .. 34841841 .. 461446363500000.00.2025542131.88268945.794632145.67741-152.74799859.7607-599.93256569974569974
subsamples[halos['halo_subsamp_start'][i]:halos['halo_subsamp_start'][i] + halos['halo_subsamp_len'][i]]

<Table length=17>

xyzvxvyvzpid
float32float32float32float32float32float32int64
31.5762146.07626145.95335-469.4303693.1775-122.8415761637243587
31.92966345.918232145.87573-362.6751732.6623-49.721591620647597
31.86607745.79948145.8-356.45993859.6168-504.52791622709673
31.87823346.225872145.76727-213.51036829.18066-650.76791628931922
31.87449345.611534145.7364-301.61993885.6658-222.101941631011277
32.0736645.440414145.7093-425.00992585.4169-160.406971612335919
31.86888346.23335145.69434-276.39355731.1998-451.88151616498956
31.7978245.83595145.68967-87.74398830.643-365.59991631009831
31.90254645.819115145.66815-235.446351061.7021-548.39991628939112
31.99137745.540466145.66254-490.8179774.6148-296.135961628934790
31.80623245.791065145.65599-242.66695549.7709-659.450871624786154
31.91283245.77517145.6242-315.32993913.0858-748.56581631006954
31.7501345.52457145.6242-677.27386952.84485-346.862951579159738
31.83615545.76582145.62047-153.55197686.87085-652.59591628931911
31.98670245.853714145.57465-225.20955884.75183-67.2703861604057349
31.8829145.802288145.5615750.72699793.80884-788.32481626864073
31.92218245.79013145.34088-227.585951125.5907-667.67691628931917