OM data reduction with SAS: step by step fast mode data processing

Introduction The processing chain omfchain, used to obtain light curves from OM fast mode data is a concatenation of several SAS tasks run consecutively. All these steps are described in detail. Expected Outcome OM threads describe how to process OM data using the corresponding chains within SAS. They give also some advice and hints on the checkings to be done on output SAS products to assess the quality of these output products. SAS Tasks to be Used omfchain All tasks invoked by omfchain are described below. Prerequisites Before running any of the OM chains users should check in the ODF what type of data they have: images, fast mode, and/or spectral images obtained with the grisms. Then the corresponding chain(s) can be run. Running a chain that does not correspond to the type of data will give a fatal error and it may produce some confusion to the user. A proper set up of SAS is mandatory. SAS Startup Thread Useful Links SAS documentation SAS User Guide Caveats

---

Procedure

The data package received by the investigator contains OM data which normally do not necessitate further processing for the purpose of calibration. However, a user may want to apply the most recent calibrations, or to change some default parameters to e.g. improve the source detection on his/her data. It may not be necessary to run the complete chains, but just some tasks. All this can be done interactively.

The SAS_ODF environment variable shall be set to the summary file produced by odfingest, or to a directory containing the data, and access to calibration files shall be set through cifbuild. In addition to the fits files containing the science data, the ODF needs to contain the OM house-keeping files:

   0644_0125321001_OMX00000NPH.FIT - non periodic house-keeping
   0644_0125321001_OMX00000PEH.FIT - periodic house-keeping

and spacecraft files of the following form:

   0644_0125321001_SCX00000SUM.SAS - ASCII observation summary file
   0644_0125321001_SCX00000TCS.FIT - Spacecraft Time correlation file
   0644_0125321001_SCX00000ATS.FIT - Spacecraft Attitude file

Invoking omichain or omfchain will automatically start the processing of all OM data in the working directory. The duration of the process will depend on the number of exposures and windows and at the end we shall obtain the final processed files. No intermediate file will be preserved.

In the standard automatic SSC pipeline processing, the temporary image files are re-used and thus overwritten. In the task by task processing, they can be distinguished so that intermediate stage output can be looked at if desired. In the next examples we have used the same naming system than omfchain. This allows us to maintain some intermediate files which can be helpful in understanding the whole process.

Some of the default parameters used by individual tasks can be tuned if necessary.

Example of fast mode data processing

In order to process a single exposure containing fast mode data as

   0644_0125321001_OMS00202FAE.FIT

the following files are necessary (they are normally part of the ODF):

   0644_0125321001_OMS00200WDX.FIT - Exposure priority window file
   0644_0125321001_OMS00200THX.FIT - Exposure tracking history file
   0644_0125321001_OMS00202FAE.FIT - Exposure fast mode data file (fast mode window)

and the process can be run task by task in the following way. Note that output products of the different tasks have been given the names that omfchain would have applied.

It is assumed throughout this example that the ODF data are in: /path_to_your_data/
and we are running the tasks and writing their results in a subdirectory: /path_to_your_data/fast/

step 1

The OM tracking data are processed for later use, if applicable.

omprep set=/path_to_your_data/0644_0125321001_OMS00200THX.FIT \
 pehset=/path_to_your_data/0644_0125321001_OMX00000PEH.FIT \
 nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT \
 wdxset=/path_to_your_data/0644_0125321001_OMS00200WDX.FIT \
 outset=/path_to_your_data/fast/tmp_tracking \
 modeset=3

As for image data, if there is no THX file, then set=DUMMYTHX.FIT. omprep will generate a dummy file needed for the rest of the chain, with zero drift in it.

step 2

A plot of the tracking data, useful to assess the S/C stability during the exposure, is generated. Note that tracking data may not be good enough, in which case no plot is produced.

omdrifthist set=/path_to_your_data/fast/tmp_tracking \
 plotfile=/path_to_your_data/fast/P0125321001OMS002TSHPLT0000.PS \
 trackradius=0.5 \
 hardcopy=yes

step 3

The count rates of tracking stars are written into a file.

omthconv thxset=/path_to_your_data/fast/tmp_tracking \
 nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT \
 outset=/path_to_your_data/fast/P0125321001OMX002TSTRTS0000.FIT \
 modeset=1

step 4

Processing of the fast mode data starts here.

omprep set=/path_to_your_data/0644_0125321001_OMS00202FAE.FIT \
 pehset=/path_to_your_data/0644_0125321001_OMX00000PEH.FIT \
 nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT \
 wdxset=/path_to_your_data/0644_0125321001_OMS00200WDX.FIT \
 outset=/path_to_your_data/fast/F0125321001OMS002EVLIST2000.FIT \
 modeset=1

In its second run, omprep is invoked for fast data (modeset=1) and the FAE raw event data is transformed into a modified event list to be used as input for evselect

step 5

Starting the processing of the detected events.

evselect table=/path_to_your_data/fast/F0125321001OMS002EVLIST2000.FIT \
 withimageset=yes \
 imageset=/path_to_your_data/fast/F644_0125321001_OMS00202FIMI.FIT \
 xcolumn=RAWX \
 ycolumn=RAWY

The pseudo-image corresponding to the fast mode OSW has been created by this first run of evselect.

step 6

Correcting for drift.

omfastshift nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT \
 set=/path_to_your_data/fast/F0125321001OMS002EVLIST2000.FIT \
 thxset=/path_to_your_data/fast/tmp_tracking \
 interpolate=yes

The X- and Y- coordinates of the photon events are corrected for spacecraft drift. New columns are added to the event list with the corrected values.

Although no real flat field correction exists for OM, nor is it necessary, the processing requires such a file which can be generated using the task omflatgen as follows. (This task can also be run at the beginning of the processing).

step 7.0

omflatgen outset=0644_0125321001_OMX00000FFX.FIT

The output flatfield (primary extension) will be set to unity.

step 7

mfastflat set=/path_to_your_data/fast/F0125321001OMS002EVLIST2000.FIT \
 slewflatset=/path_to_your_data/0644_0125321001_OMX00000FFX.FIT \
 fastimgset=/path_to_your_data/fast/P0125321001OMS002IMAGE_2000.FIT \
 oswflatset=/path_to_your_data/fast/F0125321001OMS002FLAFLD2000.FIT

Here again, the system is prepared to apply a subset of the omflatgen generated flat field to the fast mode window data, taking into account the spacecraft drift. The flat field is set to one, and therefore this correction has no real effect. The task generates the tracking shifted F0125321001OMS002FLAFLD2000.FIT (only for the fast mode window) and the corrected pseudo image image.fit.

step 8

omdetect nsigma=6 \
 set=/path_to_your_data/fast/P0125321001OMS002IMAGE_2000.FIT \
 regionfile=/path_to_your_data/fast/F0125321001OMS002REGION2000.ASC \
 outset=/path_to_your_data/fast/P0125321001OMS002SWSRLI2000.FIT

The output region will allow the user to check the proper detection of the source in the small fast window. The PSF information is used to parameterize the detected source.

step 9

omatt set=/path_to_your_data/fast/P0125321001OMS002IMAGE_2000.FIT \
 sourcelistset=/path_to_your_data/fast/P0125321001OMS002SWSRLI2000.FIT \
 ppsoswset=/path_to_your_data/fast/P0125321001OMS002SIMAGE2000.FIT \
 usecat=no rotateimage=yes tolerance=3

Astrometry is applied as for image data. The pseudo-image is north aligned too.

step 10

omregion set=/path_to_your_data/fast/P0125321001OMS002SWSRLI2000.FIT \
 srcnumber=1 srcradius=-6 nfwhm=3 bkginner=1.2 bkgouter=2.5 \
 bkgfile=/path_to_your_data/fast/F0125321001OMS002BGDREG2000.FIT \
 srcfile=/path_to_your_data/fast/F0125321001OMS002SRCREG2000.FIT

These regions will be used by evselect to filter out the event list, extracting the corresponding photon events for the source and the background. Optional parameters can be used to fine-tune the definition of the regions.

step 11

Source events

evselect table=/path_to_your_data/fast/F0125321001OMS002EVLIST2000.FIT \
 expression='((WIN_FLAG .eq. 0) .and. \
    (region(/path_to_your_data/fast/F0125321001OMS002SRCREG2000.FIT, CORR_X, CORR_Y)))' \
 xcolumn=CORR_X ycolumn=CORR_Y \
 rateset=/path_to_your_data/fast/F0125321001OMS002SCRATE2000.FIT \
 timecolumn=TIME timebinsize=10 maketimecolumn=yes withrateset=yes

step 12

Background events

evselect table=/path_to_your_data/fast/F0125321001OMS002EVLIST2000.FIT \
 expression='((WIN_FLAG .eq. 0) .and. \
    (region(/path_to_your_data/fast/F0125321001OMS002BGDREG2000.FIT, CORR_X, CORR_Y)))' \
 xcolumn=CORR_X ycolumn=CORR_Y \
 rateset=/path_to_your_data/fast/F0125321001OMS002BGRATE2000.FIT \
 timecolumn=TIME timebinsize=10 maketimecolumn=yes withrateset=yes

And finally the light curve can be obtained and plotted. The photometric corrections are also applied (coincidence loss, dead time, PSF, magnitude conversion).

step 13

omlcbuild srcregionset=/path_to_your_data/fast/F0125321001OMS002SRCREG2000.FIT \
 bkgregionset=/path_to_your_data/fast/F0125321001OMS002BGDREG2000.FIT \
 srcrateset=/path_to_your_data/fast/F0125321001OMS002SCRATE2000.FIT \
 bkgrateset=/path_to_your_data/fast/F0125321001OMS002BGRATE2000.FIT \
 sourcelistset=/path_to_your_data/fast/P0125321001OMS002SWSRLI2000.FIT \
 wdxset=/path_to_your_data/0644_0125321001_OMS00200WDX.FIT \
 outset=/path_to_your_data/fast/P0125321001OMS002TIMESR2000.FIT

step 14

lcplot set=/path_to_your_data/fast/P0125321001OMS002TIMESR2000.FIT \
 binsize=1 plotdevice=/PS \
 plotfile=/path_to_your_data/fast/F0125321001OMS002TIMESR2000.PS \
 bkgdyscale=no

Last Updated: 16 April 2010

---

Caveats None

---