Example: On-Off Observation#

This example demonstrates how to process On-Off observation data with HiFAST. In this mode, the telescope observes a target source (“On”) and then a nearby empty region of the sky (“Off”). Subtracting the “Off” spectrum from the “On” spectrum removes background noise and instrumental effects, leaving the clean signal from the source.

This page presents two different methods for processing On-Off data, which primarily differ in how they handle the removal of instrumental standing waves.

Download Sample Data#

First, download the sample dataset from this link: https://pan.cstcloud.cn/s/LdEsxTHRY

Method 1: Sine-fitting for Standing Waves#

This method is suitable when the standing wave phase is stable over time. It involves fitting and subtracting a sine wave from the spectrum.

1. Separate, Calibrate, and Subtract

The first script uses hifast.pos_swi to separate the On-source, Off-source, and noise diode observations. It then performs a temperature calibration and subtracts the Off-source spectrum from the On-source spectrum.

 1#!/bin/bash
 2
 3# enable **/
 4shopt -s globstar
 5
 6# only need input chunk *0001.fits
 7files="$(ls RAW_data/*/*/*M01*W_0001.fits)"
 8# check files
 9printf "${files}"
10echo
11
12# commands
13#     "python -m hifast.sep file.hdf5 ..." only supports one file
14# run "python -m hifast sep -p 3 file1.hdf5 file2.hdf5 file3.hdf5  ..." run multiple files and 3 files in parallel.
15python -m hifast pos_swi -p 5 \
16    $files \
17   -d 4 -m 4 -n 36 \
18   --frange 1378  1410  \
19   --smooth gaussian --s_sigma 5 \
20   --noise_mode high \
21   --t_src 120 --t_ref 120 --t_change 30 --n_repeat 1 \
22   --only-off True \
23   --outdir output/%[project]s/%[date]s

2. Manual Baseline and Standing Wave Removal

The next step is to manually remove the baseline and fit the standing wave using a Jupyter notebook.

Note

The notebook for this step is available for download but is not rendered on this page to keep the layout clean.

3. Final Corrections

The final step is to perform flux calibration and a coordinate system correction.

python -m hifast.flux
python -m hifast.multi --fc True

Method 2: FFT for Standing Waves#

This method uses a Fast Fourier Transform (FFT) approach to remove the standing wave. This can be more effective if the standing wave phase is not stable.

1. Initial Calibration

First, calibrate the data in the same way as for mapping, without separating the On and Off observations yet.

 1#!/bin/bash
 2
 3# enable **/
 4shopt -s globstar
 5# 1. Temperature calibration
 6echo "Temperature calibration:"
 7# only need input chunk *0001.fits
 8files="$(ls RAW_data/*/*/*M01*W_0001.fits)"
 9# check files
10printf "${files}"
11echo
12# commands
13#     "python -m hifast.sep file.hdf5 ..." only supports one file
14# run "python -m hifast sep -p 3 file1.hdf5 file2.hdf5 file3.hdf5  ..." run multiple files and 3 files in parallel.
15python -m hifast sep -p 5 \
16    $files \
17   -d 4 -m 4 -n 36 \
18   --frange 1369  1419  \
19   --smooth gaussian --s_sigma 2 \
20   --check_cal A --pcal_vary_lim_bin 0.02 \
21   --merge_pcals True --method_merge median --method_interp nearest \
22   --save_pcals True \
23   --outdir 'output2/%(project)s/%(date)s'
24
25# RA-DEC
26# # only need beam 01 specs_T.hdf5 file
27python -m hifast radec **/*-M01*-specs_T.hdf5

2. FFT Standing Wave Removal

Next, use the FFT method to remove the standing wave from each spectrum.

 1#!/bin/bash
 2
 3# enable **/
 4shopt -s globstar
 5
 6files="$(ls output2/*/202*/*M01*-specs_T.hdf5)"
 7# check fpatten
 8printf "${files}"
 9echo
10
11# using hifast.sh to run multi files
12# commands
13commands=$(cat <<EOF
14# limit frange again
15python -m hifast.bld  | --nproc 5 \
16                        --frange 1369 1419 \
17                        --method PLS-asym2 --lam 1e9 \
18                        --njoin 40 \
19                        --s_method_freq gaussian --s_sigma_freq 3 \
20                        --exclude_type auto2 \
21                        --post_method poly-asym2 --post_deg 3 \
22                        --post_s_method_freq gaussian --post_s_sigma_freq 3 \
23                        --post_exclude_type auto2
24python -m hifast.sw   | -c conf/S2-sw.ini --nobld True
25python -m hifast.flux  |  
26EOF
27)
28# check commands
29echo "commands: ""$commands"
30# run multiple files
31hifast.sh "$files" -c "$commands" -n 4

3. Separate and Subtract

Now, use hifast.pos_swi_2 to separate the On and Off observations and perform the subtraction.

 1#!/bin/bash
 2
 3# enable **/
 4shopt -s globstar
 5
 6# only need input chunk *0001.fits
 7files="$(ls -v output2/*/202*/*M01*-sw_nobld-flux.hdf5)"
 8# check files
 9printf "${files}"
10echo
11# commands
12python -m hifast pos_swi_2 -p 5 \
13    $files \
14   --frange 1378  1400  \
15   --t_src 120 --t_ref 120 --t_change 30 --n_repeat 1 \
16   --only-off True
17

4. Manual Baseline Removal

Finally, manually remove any remaining baseline residuals in a Jupyter notebook.

Note

The notebook for this step is available for download but is not rendered on this page to keep the layout clean.

5. Final Correction

The final step is to perform the coordinate system correction.

python -m hifast.multi --fc True