Research & Science Home ESA Public Web Site Sci-Tech Portal      XMM-Newton Public Web Site XMM-Newton Sci-Tech Portal
Astrophysics Missions Planetary Exploration Missions Solar Terrestrial Science Missions Fundamental Physics Missions Science Faculty

XMM-Newton SOC
Home Page
Conferences & Meetings
News
Latest News
News Archive
Newsletter
Quarterly Status Report
General User Support
XMM-Newton Helpdesk
Users Group
Proposers Info
Documents & Manuals
AO-14 Information
OTAC Results
Target Visibility Tool
Target Search Tool
Observers Info
ToO Alert
ToO Details
Proposal Enhancement
Long Term Plan
Short Term Schedule
Scheduled Observations Search
Scheduling Guided Tour
Observation Log Browser
Data Analysis
SAS News
What is SAS?
How to use SAS
SAS Workshops
SAS Version Changes
Download and Install SAS
Science Simulator (SciSim)
Archive, Pipeline & Catalogues
XMM-Newton Science Archive
Pipeline Processing System
Observation & Data Status
Time Correlation Fix
Serendipitous Source Catalogue
OM Serendipitous Source Catalogue
Slew Survey Catalogue
Latest Slew Results
Browsing Interface for RGS Data
Live Radiation Monitor Plots
Radiation Monitor Files
Calibration & Background
Calibration
Background Analysis
SOC Info
XMM-Newton Contact Details
How to get to ESAC
ESA Research Fellowships
ESAC Trainee Projects
ESA Young Graduate Trainees
About XMM-Newton
Image Gallery
XMM-Newton Technical Details
XMM-Newton SOC Overview
XMM-Newton 10th Anniversary
Publications
Publications
Large Programmes
Publication Guidelines
Source Naming Convention
Communicating your Results
Other Links
Project Related Links
Research Projects
Other X-ray Missions
Space Weather Links
Where is XMM-Newton Now?
Search the XMM-Newton website
Demos and Guided tour of the XMM-Newton Science Archive (XSA)



Demos

Several videos showing the basics of XSA are available:

Guided Tour

Description of the main capabilities of the XSA interface, users are encouraged to explore by themselves many more interesting possibilities.
You may click on the images below to view them full size.

1. Starting the XSA
2. Logging in
3. Help
4. Searching data
  4.1. Basic search for observations by target name/coordinates
  4.2. Search for observations using a list of target names/coordinates
  4.3. Search for sources in EPIC/OM/Slew catalogues by source name/coordinates
  4.4. Search for sources in EPIC/OM/Slew catalogues using a list of source names/coordinates
  4.5. Other search options
5. Results display
6. Saving, printing and browsing the results lists and/or query configurations
7. Options to retrieve data
  7.1.Direct data download
  7.2.Retrieve data through the shopping basket
8. Interactive generation of scientific products
  8.1.Interactive generation of EPIC sources spectra and light curves
  8.2.Process data "on-the-fly" to have the most updated data product
  8.3.Filter of bad events from an EPIC event list prior to retrieval
9. Request Monitor
10. Exit



1. Starting the XSA

  • From the XSA homepage (http://xmm.esac.esa.int/xsa/) click on "Start the XMM-Newton Science Archive (XSA) Interface" link, and then on the "Start XSA Browser" button. A pop-up interface should then be opened.
    Starting may take up to several seconds, depending on the speed of the link, the speed of your computer etc.

    To run the XSA Archive through your web browser it must be Java compliant, if not please install the latest Java2 following the instructions provided here.
    Java must be turned on! (you may have it disabled by default in your browser...)

  • Alternatively you can run XSA as a Java Desktop application from http://xmm.esac.esa.int/xsa/ clicking on Start the "XMM-Newton Science Archive (XSA) Webstart". The pop-up interface should then be opened.
    To run XSA as a Desktop application you must have Java Web Start enabled. Details on what Java Web Start is can be found here.

  • If you have problems, go to the XSA FAQ. Please check the FULL LIST of options for running the XSA.

2. Logging in

To retrieve XSA data it is necessary to login, to do so click on the Login/Register button. If you are already registered then enter your username and password. Note that only the first eight characters are recognized. The case is important only when entering the password.

You can then either click on the button Login & Proceed to go back to the screen you came from, or Login & stay here to remain on this page to change your details/password.

If you are not yet registered you can do it via the Register As New User button, and enter the mandatory details. We suggest to choose as username your initial followed by your last name. Passwords must contain at least 6 characters, at least 2 must be alphabetic and at least one numeric or special. Confirmation of registration will be sent by e-mail, so it is ensured that your e-mail address is correct.


3. Help!

In general, any box that changes colour when the mouse moves over can be clicked on to give help. There is a list of Frequently Asked Questions (FAQ) that deals with solutions to common technical or installation problems and another one offering step-by-step recipes to accomplish basic query and/or retrieval tasks. If you still feel lost, consider contacting the XMM-Newton HelpDesk.


4. Searching data

You should now be on the 'Query Specification' screen. Text or numbers can be introduced in the white boxes
  1. the top-left widget indicates whether the archive browser is busy doing something. Any actions (e.g., a query) can be interrupted by clicking on the Cancel query button
  2. the top-right white box briefly indicates the status of the ongoing action. In the case of a query, for instance, it will change from Idle to Conducting Query to Generating List of Entries. If a name resolver is used, this box will indicate the coordinates of the searched (resolved) object(s)
The "Query specification" screen is divided in 8 sub-panels which allow different combinations of search criteria. With the button Results Display you can select the output of your query. With the MULTIPLE option it is possible to select:
  • Observations and Exposures
  • Observations and EPIC Catalogue sources
  • Observations and OM Catalogue sources
  • Observations and XID source identifications

4.1 - Basic search for observations by target name/coordinates


In the "Principal Search Criteria" sub-panel (see Fig.1) you can select different options:

  • Observation identifier (or a list of observation identifiers through the File with Observation ID list widget)
  • Target name. The name can be resolved by Simbad or NED. The name given by the proposer can also be entered. Strings can be used to look for all occurrences of that string in the database, either exact or not. For instance, to search for all the objects of the NGC catalogue, you may enter 'ngc'.
  • Coordinates (Equatorial, Galactic, Ecliptic). It is possible to search in a cone or in a box, or to check whether the given position falls in an observed field-of-view (FOV)
  • Observation Status (Only processed observations means to select observations for which PPS products are available in the XSA)
  • Observation availability (i.e., observations can be proprietary, therefore only browsing is possible except for the PI)
  • Observation mode (pointed or slew observation)

Whenever the option Any is selected in a selection box, the corresponding parameter is not used for the query. Similarly, parameters belonging to a closed panel are not used for a query.

The results of a query will be sorted according to different parameters depending on the Sort criteria selected.

After introducing your inputs click the button "Execute Query" and you will be directed to the "Results" panel



Fig.1 - "Principal search criteria" sub-panel of the "Query Specification" panel of XSA

4.2. - Search for observations using a list of target names/coordinates


Still in the Principal Search Criteria sub-panel (see Fig.1) of the Query Specification panel it is possible to search on a list of either target names (to be resolved by NED or SIMBAD, or the name as originally given by the observer) or target coordinates.

In order to search on a list of target names:

  • 1. go to the Query Specification panel
  • 2. open the Principal Search Criteria sub-panel
  • 3. select:
    • "Search Target By" = "Name"
    • "Name" = "for NED", or "for SIMBAD" or "Given by Proposer"
    • Target in FoV, or Radius or Box, as appropriate
  • 4. write the file name (full path) in the File With Target List widget (the button Locate File besides the widget can be used to browse your local disk)
  • 5. click on Execute query

In order to search on a list of coordinates:

  • 1. go to the Query Specification panel
  • 2. open the Principal Search Criteria sub-panel
  • 3. select:
    • "Search Target By" = "Equatorial" or "Galactic" or "Ecliptic"
    • Target in FoV, or Radius or Box, as appropriate
  • 4. write the file name (full path) in the File With Target List widget (the button Locate File besides the widget can be used to browse your local disk)
  • 5. click on Execute query

The file for a search on a target list shall contain one target name per line. The file for a search on a coordinate list shall have one object coordinates per line. Several formats are allowed for equatorial coordinates, click on the RA, Dec blue boxes to see the formats accepted.
For ecliptic and galactic coordinates, inputs are expected in degrees (two numbers per line).

4.3 - Search for sources in the EPIC, OM and Slew Survey source catalogues by source name/coordinates


For using other search criteria one should close the Principal Search Criteria sub-panel and open the "XMM-Newton EPIC Catalogue", the "XMM-Newton OM Source Catalogue" or the "XMM-Newton Slew Survey Source Catalogue" sub-panels

  • Opening "XMM-Newton EPIC Catalogue" (see Fig.2), allows to search the content of the XMM-Newton EPIC source catalogue. Queries are possible both on quantities related to the observations, where the sources were detected, and/or on source properties.
  • To query the catalogue for sources by name or coordinates introduce your inputs as for the Principal Search Criteria
  • To select also by the source properties the other buttons can be used. Some of the buttons (e.g. Counts, Hardness ...) activate pop-up windows, which contain the actual widgets to be filled to perform queries on the corresponding quantities. Click Close to close the window, and Clear to remove any inputs from the widgets. Once a value has been introduced in a widget, this remains set even if the window is closed. In this case, the corresponding Open button in the XMM-Newton EPIC Source Catalogue sub-panel turns to dark grey.
  • After introducing the search parameters click on the button Execute query



    Fig.2 - "XMM-Newton EPIC Source Catalogue" sub-panel in the query specification XSA screen


  • Opening the "XMM-Newton OM Source Catalogue" sub-panel (see Fig.3), it is possible to search the content of the XMM-Newton OM source catalogue. Queries are possible both on quantities related to the observations, where the sources where detected (target name, coordinates, observation date, etc), and on source properties. Among the latter, one may mention magnitudes/fluxes for each OM filter)
  • After introducing the search parameters click on the button Execute query



    Fig.3 - "XMM-Newton OM Source Catalogue" sub-panel in the query specification XSA screen


  • Opening the "XMM-Newton Slew Surve Source Catalogue" sub-panel (see Fig.4), it is possible to search the content of the most updated version of the Slew Survey Source Catalogue
  • After introducing the search parameters click on the button Execute query



    Fig.4 - "XMM-Newton Slew Survey Source Catalogue" sub-panel in the query specification XSA screen


4.4. - Search for sources in EPIC/OM/Slew catalogues using a list of target names/coordinates


The search on a list of targets or coordinates in the 2XMMi, OM and Slew Catalogues can be performed in a similar way as the search of observations using a list of targets/coordinates.

4.5. - Other search options


Other sub-panels offer more search possibilities.

  • Opening the "Proposal" sub-panel (see Fig.5), it is possible to search on
    • Target Type
    • Proposal identifier
    • PI name
    • Proposal category
    • Proposal program
    • String in proposal abstract

    probably the most interesting for the scientific use of XSA is the possibility of selecting observations by Target Type, as classified according to the four digit numeric code of the "HEASARC Object Classification" (see http://heasarc.gsfc.nasa.gov/W3Browse/class_help.html).

    Searching by object Class one can select the objects of a given Class that were observed as targets by XMM-Newton (bearing in mind that the class assignments for the same object made by different PIs, may not always be identical, and the class codes may not always be present, correct, or complete).



    Fig.5 - "Proposal" sub-panel in the query specification XSA screen


  • The other sub-panels are
    • the "Orbit and Data Processing" sub-panel, which allows to search on Revolution number, Data processing configuration, etc,
    • the "Exposure" sub-panel, which allows to search per Instrument Combination of Modes/Filters, etc.
    • the "XID program observations" sub-panel, which allows to search the content of the XID follow-up program (currently only INT/WFC images are available). Queries are possible both on quantities related to the XMM-Newton observation, of which the XID observation is the follow-up, and on the XID observation itself.

    It is also possible to use simultaneously several panels, for instance Observations in Revolution > 1500, with EPIC PN in Full Frame mode, near the position of NGC 253.

    After introducing the search parameters click on the button Execute query.

5. Results display

After clicking the button Execute Query the panel Latest Results is opened, displaying the list of observations, exposures and/or sources, matching your search criteria (see Fig.6). In order to create a file with the content of this panel:

If you want to leave this panel and start another selection click on the Query Specification button at the top left.



Fig.6 - "Latest Results" panel in the query specification XSA screen


With the XSA, it is currently possible to create different lists as a result of a query:

  • an observations list, i.e. a list of all XMM-Newton observations matching the query. The records of the observation catalogue are white
  • an exposures list, i.e. a list of all XMM-Newton exposures matching the query. The records of the exposure catalogues are blue
  • an EPIC source catalogue list, i.e. a list of all the sources belonging to the XMM-Newton EPIC source catalogue matching the query. Dark green fields are expanded in a pop-up window, displaying additional quantities, when the cursor is located upon them. For instance: if the cursor is moved to the field displaying the total pn count rate, a window appears, where the count rates in different energy bands are shown
  • an XMM-OM source list, i.e. a list of all the sources belonging to the XMM-Newton OM UV source catalogue matching the query. Moving the mouse on dark yellow fields pops-up a small window with magnitudes or fluxes for the different OM filters bands
  • an XMM-Newton Slew Survey source list, i.e., a list of all the sources belonging to the XMM-Newton Slew Survey source catalogue matching the query
  • a XID product catalogue, i.e. a list of all the XID program observations. The records of the XID product catalogue are magenta.

It is also possible to select a combined list of Observations and one of the other types (Exposures or EPIC or OM or Slew Survey sources).

Once in the Latest Results panel, you can switch from one type of list to another using the top right selection menu (Observations, Exposures, (EPIC) Sources, OM Sources, Slew Sources, XID; see Fig.7) and then clicking the Apply button. The amount of information displayed in the results display for each of these options can be selected using the right part of that menu. For instance, you may choose the instruments, whose exposures you want to be displayed in an exposures list.



Fig.7 - Selection menu in the Results Display screen


By default, an icon of the corresponding EPIC field image is displayed besides each record of the resulting observations list. You can change this to display the background light curve during the observation (obtained by combining the corresponding PPS products) by selecting Bckg. Light Curve in the selection menu of the results display screen (see Fig.7 above). Analogously, to the right of each exposure information the corresponding icon is displayed. The results lists for the XMM-Newton EPIC source catalogue displays icons corresponding to the full energy bandpass image around the source and to the Digital Sky Survey finding chart, the spectrum and the Light Curve of the source. Clicking on one of such icons will start an image viewing tool, from which a FITS viewer tool can be in turn launched (clicking on Interactive Image Analysis). Images can be displayed as well through ds9 or Aladin.
For the XMM-OM and the Slew Survey source catalogues there are also icons displayed for each source.

To the left of each record of an observation list (see Fig.6), additional buttons allow to get information on the corresponding observation:

  • Exposure, allows to expand the observation into its constituent exposures
  • Sources, shows all the EPIC sources detected in that observation
  • Details, leads to a page containing technical information on the observation scheduling and on the proposal to which the observation belongs
  • Articles, provides links to refereed journal papers which made use of the observation data. Ultimately, this link allows to reach the corresponding ADS entries.
  • Query Other Archives, opens a pop-up window to search the target in a choice of archives in different wavelengths
  • XID observations, displays the XID observation taken as follow-up of the XMM-Newton observation

6. Saving, printing and browsing the results lists and/or query configurations

  • To save/print a file containing the parameters defining a query click on the Save Session item under the File menu to the top left of the interface. Saved configurations can be later uploaded through the Load Session from File item in the same menu.
  • To save/print the results of a query click on the Print/Save Results to open a menu of options. The option "Save Results (Custom one-line-format)" gives great flexibility for selecting the results to be saved.

You can browse through different queries performed during an open XSA session through the small blue arrows at the upper left corner, in the Query Specification or Latest Results panels. The parameters employed for each query are visualized by dragging the cursor on the small downward arrow besides the configuration counter (see Fig.8)



Fig.8 - Example of the query browser in the XSA user interface. By placing the cursor close to the downward arrow
besides the query counter, the parameters employed for each of the queries done during a given XSA session are listed


All configurations are lost once the XSA session is closed.

7. Options to retrieve data

You may retrieve data from the XSA directly downloading them or via a "Shopping Basket".
Both methods allow the same retrieval options:

  • ODF, if you want to retrieve the Observation Data Files
  • PPS, if you want to retrieve the Pipeline Products
  • Source-specific Spectra, Timing or Images if you want to retrieve the Pipeline Products of the EPIC sources
  • Custom if you want to define a customized sub-set of PPS products for each observation/exposure (e.g.: only the EPIC event lists, the RGS spectra and/or the PPS cross-correlation products). In order to specify the content of the customized set, click on the Define Custom button. The "Custom Definition" window (see Fig.10) allows to choose individual datasets and/or "groups" (event lists, images, source lists, light curves, spectra and/or finding charts)
  • The option MULTIPLE can be used to combine any of the above options in a single retrieval request


Fig.10 - The "Custom Dataset Definition" panel


7.1 Direct data download


Use the direct download button besides each observation or exposure catalogue record (see Fig.9). This button activates directly an ftp session, downloading data products corresponding to that observation or exposure.



Fig.9 - XSA results panel. The Retrieve button for direct download is located besides each observation (white) or exposure (blue) record.
For the data covered by proprietary rights, the icon display is disabled.


7.2 Retrieve data through the shopping basket


With this method you will transfer the records of the observations/exposures/XSA source lists, whose data you want to retrieve, into a "shopping basket" . This method is envisaged for complex, multi-observation or multi-exposure retrieval requests. Once the shopping basket contains all the datasets you intend to retrieve, these can be transferred to a secure ftp area.

7.2.1 Selecting the data and moving it to the shopping basket

To select data for retrieval click on the check box to the left of each record of the results list, and then click on Move selected to basket at the top of the panel. The observation will disappear from the results screen. To see what is in your shopping basket, click the Shopping basket button. The observations, exposures or sources that you intended to move shall now have disappeared from the Latest Results panel and appeared in the Shopping basket panel.

It is possible to move to the shopping basket entire observations (with all its displayed exposure/source records), individual exposures, or sources. Exposures pertaining to different observations can be freely selected and combined in the shopping basket. If you want to request ALL the observations that your query produced you can click on the Move All to Basket button. This will move all the observations into the Shopping Basket and it will also take you to the Shopping Basket so that you can make the request straight away.

7.2.2 In the shopping basket

The shopping basket contains the instantaneous list of all the observation, exposure and XSA source records, whose data are available for you to retrieve. You can delete items from the shopping basket by clicking on the check box to the left of them and then clicking the Delete selected box to the top right.

When records corresponding to observation, exposure and XSA source catalogues are simultaneously present in the shopping basket, the button Next in the top left corner allows to select the type of products you want to retrieve for each group (observations, exposures and source products).

A quota on the volume of retrievable observations per user exists if the shopping basket is used as retrieval method. This quota is currently set to 2 GBytes per user. The quota is refreshed every three days. If you think you need an increase of your own quota, please do not hesitate to contact the XMM-Newton HelpDesk, so that we can examine which the best way to help you. No quota exists on the direct download (mechanism 7.1. above).

When you are happy with your retrieval choice, you can retrieve the corresponding dataset(s) by clicking on the Submit Request button.

7.2.3. Retrieval

The last step prompts the overlay of the Request summary panel. In this panel the number of requested observations, the size of the dataset and the remaining quota are reported. You can go back to the Query specification or to the Latest Results screen if you want to change your query or shopping basket selection, respectively. You can also choose the preferred mean of delivery, and the compression format, and whether you prefer the PPS products as individual files (which can be browsed in the ftp area) or as a single tarfile (ODFs are always distributed as an individual gzipped tarfile). Confirm your request with the button at the bottom of the panel. A request ID is shown. This means that the generation of the dataset has started. If ftp has been chosen, an e-mail will be sent to the request originator as soon as the data are available on the ftp area for retrieval.

8. Interactive generation of scientific products

8.1 Interactive production of spectra and light curves for EPIC sources


EPIC spectra and light curves are data products and can be downloaded like other PPS products. Spectra and light curves can also be generated interactively from the XSA user interface (together with the appropriate spectral response matrices) through the following steps:

  1. perform a query, with Result Display = Exposures
  2. click on an EPIC exposure image icon to define regions, from which spectra and light curve can be extracted.
  3. click on Interactive Image Analysis in the pop'd-up display window
  4. click Spectra/Light on the Survey Product Display Tool window (see Fig.9). Response matrices are produced alongside source and background spectra.
  5. define:
    • Create=Spectra or Create=Light Curve
    • Binsize=n, where n is the light curve bin size in seconds (for Light Curve only)
  6. click with the left mouse button on the image (see Fig.10). A green circle appears
  7. modify the size of the circle by dragging one of highlighted corners with the right mouse button
  8. change Select source to Select background
  9. repeat steps 6. and 7. above (the circle is now blue)
  10. you may type selectlib expressions in the Expression widget, if you wish (standard data screening expression are already there by default)
  11. click Run

You will receive an e-mail, notifying the availability of the requested scientific products for ftp retrieval.



Fig.11 - The XSA image viewer. Its sub-panel is activated, which allows the interactive generation of spectra and light curves. In the example: the green circle indicates the region,
from which the source spectrum is being extracted; the blue circle indicates the region, from which the background spectrum is being extracted



Fig.12 - Sub-panel to set the interactive scientific products extraction parameters


Users are recommended to check the SAS documentation and threads and read the current SAS watchout items if they intend to use this functionality.

8.2 Process data "on-the-fly" to have the most updated data products


The XSA contains data processed with different versions of the data reduction software and calibrations. Although all the data are periodically reprocessed with the same SAS version and the same calibration files, a certain degree of inhomogeneity is unavoidable, as the XSA is the science archive of an operational mission, whose content is being continuously upgraded.

In order to alleviate this inhomogeneity, the XSA user interface provides a tool to process Observation Data Files with the latest public version of software and calibrations, with no need to install SAS locally, through the following steps:

  1. select the observation whose data you want to process, by clicking on the check box besides the observation record
  2. click the button Move Selected to Basket
  3. move to the "shopping basket" by clicking the button Shopping Basket at the top of the Latest Results panel
  4. [at this point you are requested to log in as a registered XSA user if you have not done it yet. If you do not own an XSA account yet, create a new one]
  5. select Observation Products=Data Processing
  6. select one of the following items in the menu besides each record in the shopping basket:
    • MOS will process MOS data with emproc
    • EPN will process PN data with epproc
    • RGS will process RGS data with rgsproc
    • OM Images will process OM image data with omichain
    • ALL will process all the instruments data
  7. click on the button Submit Request
  8. click on the button Confirm in the new panel (Confirmation of Request), that appears automatically.
An e-mail is sent, notifying the availability of processed data for ftp retrieval.

8.3 Filter of bad events from an EPIC event list prior to retrieval


This can be done following these steps

  1. select the observation whose data you want to process, by clicking on the check box besides the observation record
  2. click the button Move Selected to Basket
  3. move to the "shopping basket" by clicking the button Shopping Basket at the top of the Latest Results panel
  4. [at this point you are requested to log in as a registered XSA user if you have not done it yet. If you do not own an XSA account yet, create a new one]
  5. select Observation Products=Data Processing
  6. select event list for the Observation you want to process
  7. in the pop'd-up window, select the file corresponding to the event lists you wish to retrieve
  8. in order to define the data selection expression, you may follow either one of the following ways:
    • write the selection expression in the SAS Expression widget
    or:
    • click on Background GTI Filtering
    • fill the widgets, corresponding to the data filtering criteria you intend to apply
    • click on Generate SAS Expression
  9. click OK
  10. click on the button Submit Request
  11. click on the button Confirm in the new panel (Confirmation of Request), that appears automatically.
An e-mail will be received, notifying the availability of the requested scientific products for ftp retrieval.


9. Request Monitor

You can monitor your shopping basket download or data processing requests, through a specific "Request Monitor" tool. To start it, click on the Request Monitor button in the top row of each user interface panel.

The status of each process is color-coded.

10. Exit

Just click on the 'File' menu at the top left of the applet and select Exit. It is recommended to exit when not used. Note that when you exit all selections will be lost.



   Copyright 2014© European Space Agency. All rights reserved.
This page was last updated on 1 January, 1970.