OM data reduction with SAS: processing image data, step by step

Introduction The thread explains how to process OM images to extract photometry for all detected sources. Expected Outcome OM threads describe how to process OM data using the corresponding chains within SAS. They give also some advice and hints on the checkings to be done on output SAS products to assess the quality of these output products. SAS Tasks to be Used omichain Prerequisites Before running any of the OM chains users should check in the ODF what type of data they have: images, fast mode, and/or spectral images obtained with the grisms. Then the corresponding chain(s) can be run. Running a chain that does not correspond to the type of data will give a fatal error and it may produce some confusion to the user. A proper set up of SAS is mandatory. SAS Startup Thread Useful Links SAS documentation SAS User Guide Caveats

---

Procedure

The data package received by the investigator contains OM data which normally do not necessitate further processing for the purpose of calibration. However, a user may want to apply the most recent calibrations, or to change some default parameters to e.g. improve the source detection on his/her data. It may not be necessary to run the complete chains, but just some tasks. All this can be done interactively.

The SAS_ODF environment variable shall be set to the summary file produced by odfingest, or to a directory containing the data, and access to calibration files shall be set through cifbuild.

In addition to the fits files containing the OM science image data, the ODF needs to contain the OM house-keeping files:

  0644_0125321001_OMX00000NPH.FIT - non periodic house-keeping
  0644_0125321001_OMX00000PEH.FIT - periodic house-keeping

and some spacecraft files of the following form:

  0644_0125321001_SCX00000SUM.SAS - ASCII observation summary file
  0644_0125321001_SCX00000TCS.FIT - Spacecraft Time correlation file
  0644_0125321001_SCX00000ATS.FIT - Spacecraft Attitude file

Invoking omichain or omfchain will automatically start the processing of all OM data in the working directory. The duration of the process will depend on the number of exposures and windows and at the end we shall obtain the final processed files.

In the standard automatic SSC pipeline processing, the temporary image files are re-used and thus overwritten. In the task by task processing, they can be distinguished so that intermediate stage output can be looked at if desired. In the next examples we have used the same naming system than omichain. This allows us to maintain some intermediate files which can be helpful in understanding the whole process.

Currently the chains can accept parameters to limit the processing to a given filter or a given exposure. Some of the default parameters used by individual tasks can also be tuned. (See online SAS help pages for detailed instructions for each task parameters)

Example of image data processing

To process task by task a single exposure in image mode as

  0644_0125321001_OMS00600IMI.FIT

the following files are necessary (they are normally part of the ODF):

  0644_0125321001_OMS00600WDX.FIT - Exposure priority window file
  0644_0125321001_OMS00600THX.FIT - Exposure tracking history file
  0644_0125321001_OMS00600IMI.FIT - Exposure image file

and the process will be run in the following way. Note that output products of the different tasks have been given the names that omichain would have applied.

It is assumed throughout this example that the ODF data are in: /path_to_your_data/
and we are running the tasks and writing their results in a subdirectory: /path_to_your_data/proc/

step 1

The OM tracking data are processed for later use, if applicable.

omprep set=/path_to_your_data/0644_0125321001_OMS00600THX.FIT \
 nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT \
 pehset=/path_to_your_data/0644_0125321001_OMX00000PEH.FIT \
 wdxset=/path_to_your_data/0644_0125321001_OMS00600WDX.FIT \
 outset=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FIT \
 modeset=3

In case there is no THX file, then set=DUMMYTHX.FIT. omprep will generate a dummy file needed for the rest of the chain, with zero drift in it.

step 2

A plot of the tracking data, useful to assess the S/C stability during the exposure, is generated. Note that tracking data may not be good enough, in which case no plot is produced.

omdrifthist set=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FIT \
 plotfile=/path_to_your_data/proc/P0125321001OMS006TSHPLT000.ps

step 3

The count rates of tracking stars are written into a file.

omthconv thxset=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FIT \
 nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT \
 outset=/path_to_your_data/proc/P0125321001OMS006TSTRTS000.FIT

step 4

Processing of the image data starts here. Some parameters are extracted for insertion into the FITS header.

omprep set=/path_to_your_data/0644_0125321001_OMS00600IMI.FIT \
 nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT \
 pehset=/path_to_your_data/0644_0125321001_OMX00000PEH.FIT \
 wdxset=/path_to_your_data/0644_0125321001_OMS00600WDX.FIT \
 outset=/path_to_your_data/proc/I0125321001OMS006IMAGE10000.FIT

step 5

Bad pixels in the image are flagged.
omcosflag set=/path_to_your_data/proc/I0125321001OMS006IMAGE10000.FIT \
 thxset=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FIT

Although no real flat field correction exists for OM, nor is it necessary, the processing requires such a file which can be generated using the task omflatgen as follows. (This task can also be run at the beginning of the processing)

step 6.0

A flat field is generated.

omflatgen outset=/path_to_your_data/proc/P06440125321001OM000FLAFLD0000.FIT

The output flatfield (primary extension) will be set to unity.

step 6

The flat field is applied to the data.

omflatfield set=/path_to_your_data/proc/I0125321001OMS006IMAGE10000.FIT \
 thxset=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FIT \
 inorbitflatset=/path_to_your_data/proc/P06440125321001OM000FLAFLD0000.FIT \
 tsflatset=/path_to_your_data/proc/I0125321001OMS006FLATF0000.FIT \
 outset=/path_to_your_data/proc/I0125321001OMS006IMAGE20000.FIT

step 7

A very important correction is applied here: modulo-8 fixed pattern is removed.

ommodmap set=/path_to_your_data/proc/I0125321001OMS006IMAGE20000.FIT \
 mod8product=yes \
 mod8set=/path_to_your_data/proc/I0125321001OMS006MOD80000.FIT \
 outset=/path_to_your_data/proc/P0125321001OMS006IMAGE_0000.FIT \
 nsig=3 \
 nbox=16

step 8

A searching algorithm is used to find the sources present in the image. Aperture photometry is performed on all sources to obtain corrected count rates.

omdetect nsigma=2 minsignificance=3 detectextended=1 \
 set=/path_to_your_data/proc/P0125321001OMS006IMAGE_0000.FIT \
 regionfile=/path_to_your_data/proc/I0125321001OMS006REGION0000.ASC \
 levelimage=/path_to_your_data/proc/I0125321001OMS006LEVELIMAGE0000.FIT \
 outset=/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT

Note that all functions from this step on can be accomplished with the interactive omsource task.

step 9

Image quality information is stored for later use on mosaiced images for additional source detection. This task is skipped if the data windows are not suitable for mosaicing (e.g. unique window in all exposures)

omqualitymap srclistset=/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT \
 set=/path_to_your_data/proc/P0125321001OMS006IMAGE_0000.FIT \
 outset=/path_to_your_data/proc/P0125321001OMS006IMAGE_0000.FIT \
 mode=setqualityimage

step 10

The count rates of all sources found with omdetect are converted to magnitudes in the OM instrumental system.

ommag set=/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT

step 11

Astrometry corrections are performed here to derive the coordinates of all detected sources. A sky image (North on top) is produced. This is just cosmetical, no photometric measurements should be made on this image (SIMAGE file)

omatt set=/path_to_your_data/proc/P0125321001OMS006IMAGE_0000.FIT \
 sourcelistset=/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT \
 ppsoswset=/path_to_your_data/proc/P0125321001OMS006SIMAGE0000.FIT \
 usecat=F rotateimage=T

Note that currently the usage of a catalogue for source cross correlation is disabled. The detected sources can be overlaid on the OSW as follows:

step 11a

The success of the detection can be checked.

implot set=/path_to_your_data/proc/P0125321001OMS006SIMAGE0000.FIT \
 withsrclisttab=yes \
 srclisttab=/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT \
 device='/XW'

A similar, perhaps easier result can be obtained by displaying the image with ds9 and loading the corresponding region produced by omdetect.

step 12

After all images for a given filter have been processed, they can be combined to form a mosaic. If the images are not suitable then the task is skipped.

ommosaic imagesets="/path_to_your_data/proc/list of sky images" \
 mosaicedset=/path_to_your_data/proc/myfield_in_myfilter.fit

step 13

After processing all exposures (with one or several filters), we can combine their source lists. Color corrections will be applied to derive standard UBV magnitudes.

omsrclistcomb sourcelistsets="/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT \
    /path_to_your_data/proc/any_other_SWSRLI_file" \
 nsigma=3 \
 outset=/path_to_your_data/proc/P0125321001OMCOMBOBSMLI0000.FIT

Note the list, surrounded by quotes, of SWSRLI files.

Further processing

Starting with SAS 9, an additional detection can be done on mosaiced images. This will allow us to find fainter sources. The optional parameter processmosaicedimages(=yes) in omichain has to be used.

If the observation contains multiple exposures with the same filter then the detected sources can be checked for variability using, in SAS 10, the task omvariability. Please refer to the on-line documentation for more details.

Last Updated: 16 April 2010

---

Caveats None

---