Demos and Guided tour of the XMM-Newton Science Archive (XSA)
Notice
Due to programmed maintenance, XMM-Newton web service, Science Archive and some SOC computing services may be unavailable from Friday, 21 June, 13:30 until Sunday, 23 June, 20:00 (GMT).
During this period no ToO requests can be processed.
Demos
Several videos showing the basics of XSA are available:
Description of the main capabilities of the XSA interface, users are encouraged to
explore by themselves many more interesting possibilities.
You may click on the images below to view them full size.
From the XSA homepage
(http://xmm.esac.esa.int/xsa/)
click on "Start the XMM-Newton Science Archive (XSA)
Interface" link, and then on the
"Start XSA Browser" button. A pop-up interface should then be opened.
Starting may take
up to several seconds, depending on the speed of the link, the speed of your
computer etc.
To run the XSA Archive through your web browser it must be Java compliant,
if not please install the latest Java2 following the instructions provided
here.
Java must be turned on! (you may have it disabled by default in your browser...)
Alternatively you can run XSA as a Java Desktop application from
http://xmm.esac.esa.int/xsa/
clicking on Start the "XMM-Newton Science Archive (XSA) Webstart".
The pop-up interface should then be opened.
To run XSA as a Desktop application you must have Java Web Start enabled.
Details on what Java Web Start is can be found
here.
If you have problems, go to the
XSA FAQ. Please check the
FULL LIST of options for running the XSA.
2. Logging in
To retrieve XSA data it is necessary to login, to do so click on the
Login/Register button.
If you are already registered then enter your username and password.
Note that only the first eight characters are recognized. The case is
important only when entering the password.
You can then either click on the button Login & Proceed
to go back to the screen you came from, or Login & stay here
to remain on this page to change your details/password.
If you are not yet registered you can do it via the Register As New User
button, and enter the mandatory details. We suggest to choose as username
your initial followed by your last name. Passwords must contain at least 6 characters,
at least 2 must be alphabetic and at least one numeric or special.
Confirmation of registration will be sent by e-mail, so it is ensured
that your e-mail address is correct.
3. Help!
In general, any box that changes colour when the mouse moves over can be
clicked on to give help.
There is a list of Frequently Asked Questions (FAQ) that deals with
solutions to common technical or installation problems and another one
offering step-by-step recipes to
accomplish basic query and/or retrieval tasks.
If you still feel lost, consider contacting
the XMM-Newton HelpDesk.
4. Searching data
You should now be on the 'Query Specification' screen.
Text or numbers can be introduced in the white boxes
the top-left widget indicates whether the archive browser is
busy doing something. Any actions
(e.g., a query) can be interrupted by
clicking on the Cancel query button
the top-right white box briefly indicates the status of
the ongoing action. In the case of a query, for instance,
it will change from Idle to Conducting Query to
Generating List of Entries. If a name resolver is
used, this box will indicate the coordinates of the
searched (resolved) object(s)
The "Query specification" screen is divided in 8 sub-panels which
allow different combinations of search criteria. With the button
Results Display you can select the output of your query.
With the MULTIPLE option it is possible to select:
Observations and Exposures
Observations and EPIC Catalogue sources
Observations and OM Catalogue sources
Observations and XID source identifications
4.1 - Basic search for observations by target name/coordinates
In the "Principal Search Criteria" sub-panel (see Fig.1) you
can select different options:
Observation identifier (or a list of observation identifiers through
the File with Observation ID list widget)
Target name. The name can be resolved by Simbad or NED. The name
given by the proposer can also be entered. Strings can be used to
look for all occurrences of that string in the database, either exact
or not. For instance, to search for all the objects of the NGC catalogue,
you may enter 'ngc'.
Coordinates (Equatorial, Galactic, Ecliptic). It is possible to search
in a cone or in a box, or to check whether the given position falls in
an observed field-of-view (FOV)
Observation Status (Only processed observations means to select
observations for which PPS products are available in the XSA)
Observation availability (i.e., observations can be proprietary,
therefore only browsing is possible except for the PI)
Observation mode (pointed or slew observation)
Whenever the option Any is selected in a selection box, the
corresponding parameter is not used for the query. Similarly, parameters
belonging to a closed panel are not used for a query.
The results of a query will be sorted according to different parameters
depending on the Sort criteria selected.
After introducing your inputs click the button "Execute Query"
and you will be directed to the "Results" panel
Fig.1 - "Principal search criteria" sub-panel of the "Query Specification"
panel of XSA
4.2. - Search for observations using a list of target names/coordinates
Still in the Principal Search Criteria sub-panel (see Fig.1) of the
Query Specification panel it is possible to search on a list of either
target names (to be resolved by NED or SIMBAD, or the name as originally given
by the observer) or target coordinates.
In order to search on a list of target names:
1. go to the Query Specification panel
2. open the Principal Search Criteria sub-panel
3. select:
"Search Target By" = "Name"
"Name" = "for NED", or "for SIMBAD" or "Given by Proposer"
Target in FoV, or Radius or Box, as appropriate
4. write the file name (full path) in the File With Target List widget
(the button Locate File besides the widget can be used to browse your
local disk)
5. click on Execute query
In order to search on a list of coordinates:
1. go to the Query Specification panel
2. open the Principal Search Criteria sub-panel
3. select:
"Search Target By" = "Equatorial" or "Galactic"
or "Ecliptic"
Target in FoV, or Radius or Box, as appropriate
4. write the file name (full path) in the File With Target List widget
(the button Locate File besides the widget can be used to browse your
local disk)
5. click on Execute query
The file for a search on a target list shall contain one target name per line. The
file for a search on a coordinate list shall have one object coordinates per line.
Several formats are allowed for equatorial coordinates, click on the RA, Dec blue
boxes to see the formats accepted.
For ecliptic and galactic coordinates, inputs are expected in degrees (two
numbers per line).
4.3 - Search for sources in the EPIC, OM and Slew Survey
source catalogues by source name/coordinates
For using other search criteria one should close the Principal Search
Criteria sub-panel and open the "XMM-Newton EPIC Catalogue",
the "XMM-Newton OM Source Catalogue" or the "XMM-Newton Slew
Survey Source Catalogue" sub-panels
Opening "XMM-Newton EPIC Catalogue" (see Fig.2), allows to
search the content of the
XMM-Newton EPIC source catalogue. Queries are possible both on
quantities related to the observations, where the sources were detected,
and/or on source properties.
To query the catalogue for sources by name or coordinates introduce
your inputs as for the Principal Search Criteria
To select also by the source properties the other buttons can be used.
Some of the buttons (e.g. Counts, Hardness ...) activate
pop-up windows, which contain the actual widgets to be filled to perform
queries on the corresponding quantities. Click Close to close
the window, and Clear to remove any inputs from the widgets.
Once a value has been introduced in a widget, this remains set even if
the window is closed. In this case, the corresponding Open button
in the XMM-Newton EPIC Source Catalogue sub-panel turns to dark
grey.
After introducing the search parameters click on the button Execute
query
Fig.2 - "XMM-Newton EPIC Source Catalogue" sub-panel in the query
specification XSA screen
Opening the "XMM-Newton OM Source Catalogue" sub-panel (see Fig.3),
it is possible to search the content of the
XMM-Newton OM source catalogue. Queries are possible both on quantities
related to the observations, where the sources where detected (target
name, coordinates, observation date, etc), and on source properties. Among
the latter, one may mention magnitudes/fluxes for each OM filter)
After introducing the search parameters click on the button
Execute query
Fig.3 - "XMM-Newton OM Source Catalogue" sub-panel in the query
specification XSA screen
Opening the "XMM-Newton Slew Surve Source Catalogue" sub-panel
(see Fig.4), it is possible to search the content of the most updated version
of the
Slew Survey Source Catalogue
After introducing the search parameters click on the button
Execute query
Fig.4 - "XMM-Newton Slew Survey Source Catalogue" sub-panel in the
query specification XSA screen
4.4. - Search for sources in EPIC/OM/Slew catalogues using a list
of target names/coordinates
Opening the "Proposal" sub-panel (see Fig.5), it is possible
to search on
Target Type
Proposal identifier
PI name
Proposal category
Proposal program
String in proposal abstract
probably the most interesting for the scientific use of XSA is the
possibility of selecting observations by Target Type, as classified
according to the four digit numeric code of the "HEASARC Object Classification"
(see
http://heasarc.gsfc.nasa.gov/W3Browse/class_help.html).
Searching by object Class one can select the objects of a given Class that were
observed as targets by XMM-Newton (bearing in mind that the class assignments
for the same object made by different PIs, may not always be identical, and the
class codes may not always be present, correct, or complete).
Fig.5 - "Proposal" sub-panel in the query
specification XSA screen
The other sub-panels are
the "Orbit and Data Processing" sub-panel, which allows to
search on Revolution number, Data processing configuration, etc,
the "Exposure" sub-panel, which allows to search per Instrument
Combination of Modes/Filters, etc.
the "XID program observations" sub-panel, which allows to search
the content of the
XID follow-up program (currently only INT/WFC images are available).
Queries are possible both on quantities related to the XMM-Newton observation,
of which the XID observation is the follow-up, and on the XID observation
itself.
It is also possible to use simultaneously several panels, for instance Observations
in Revolution > 1500, with EPIC PN in Full Frame mode, near the position of NGC 253.
After introducing the search parameters click on the button Execute query.
5. Results display
After clicking the button Execute Query the panel Latest Results
is opened, displaying the list of observations, exposures and/or sources, matching
your search criteria (see Fig.6). In order to create a file with the content of
this panel:
If you want to leave this panel and start another selection click on the Query
Specification button at the top left.
Fig.6 - "Latest Results" panel in the query specification XSA screen
With the XSA, it is currently possible to create different lists as a
result of a query:
an observations list, i.e. a list of all XMM-Newton
observations matching the query. The records of the observation
catalogue are white
an exposures list, i.e. a list of all XMM-Newton
exposures matching the query. The records of the exposure catalogues are
blue
an EPIC source catalogue list, i.e. a list of all
the sources belonging to the
XMM-Newton EPIC source catalogue matching the query. Dark green
fields are expanded in a pop-up window, displaying additional quantities,
when the cursor is located upon them. For instance: if the cursor is moved
to the field displaying the total pn count rate, a window appears, where
the count rates in different energy bands are shown
an XMM-OM source list, i.e. a list of all the sources
belonging to the
XMM-Newton OM UV source catalogue matching the query. Moving the mouse
on dark yellow fields pops-up a small window with magnitudes or fluxes for
the different OM filters bands
a XID product catalogue, i.e. a list of all the
XID program observations. The records of the XID product catalogue
are magenta.
It is also possible to select a combined list of Observations and one of the
other types (Exposures or EPIC or OM or Slew Survey sources).
Once in the Latest Results panel, you can switch from one type of
list to another using the top right selection menu (Observations, Exposures,
(EPIC) Sources, OM Sources, Slew Sources, XID; see Fig.7) and then
clicking the Apply button. The amount of information displayed in
the results display for each of these options can be selected using the right
part of that menu. For instance, you may choose the instruments, whose
exposures you want to be displayed in an exposures list.
Fig.7 - Selection menu in the Results Display screen
By default, an icon of the corresponding EPIC field image is displayed
besides each record of the resulting observations list. You can change
this to display the background light curve during the observation (obtained
by combining the corresponding PPS products) by selecting Bckg. Light
Curve in the selection menu of the results display screen (see Fig.7
above). Analogously, to the right of each exposure information the
corresponding icon is displayed. The results lists for the XMM-Newton
EPIC source catalogue displays icons corresponding to the full energy
bandpass image around the source and to the Digital Sky Survey finding
chart, the spectrum and the Light Curve of the source. Clicking on one
of such icons will start an image viewing tool, from which a FITS viewer
tool can be in turn launched (clicking on Interactive Image Analysis).
Images can be displayed as well through
ds9 or
Aladin.
For the XMM-OM and the Slew Survey source catalogues there are also icons
displayed for each source.
To the left of each record of an observation list (see Fig.6), additional
buttons allow to get information on the corresponding observation:
Exposure, allows to expand the observation into its
constituent exposures
Sources, shows all the EPIC sources detected in that
observation
Details, leads to a page containing technical information
on the observation scheduling and on the proposal to which the
observation belongs
Articles, provides links to refereed journal papers which
made use of the observation data. Ultimately, this link allows to
reach the corresponding
ADS entries.
Query Other Archives, opens a pop-up window to search
the target in a choice of archives in different wavelengths
XID observations, displays the XID observation taken as
follow-up of the XMM-Newton observation
6. Saving, printing and browsing the results lists and/or query
configurations
To save/print a file containing the parameters defining a query
click on the Save Session item under the File
menu to the top left of the interface. Saved configurations can
be later uploaded through the Load Session from File
item in the same menu.
To save/print the results of a query click on the
Print/Save Results to open a menu of options. The option
"Save Results (Custom one-line-format)" gives great
flexibility for selecting the results to be saved.
You can browse through different queries performed during an open XSA
session through the small blue arrows at the upper left corner, in the
Query Specification or Latest Results panels. The
parameters employed for each query are visualized by dragging the
cursor on the small downward arrow besides the configuration counter
(see Fig.8)
Fig.8 - Example of the query browser in the XSA user interface.
By placing the cursor close to the downward arrow besides the query
counter, the parameters employed for each of the queries done during
a given XSA session are listed
All configurations are lost once the XSA session is closed.
7. Options to retrieve data
You may retrieve data from the XSA directly downloading them or
via a "Shopping Basket".
Both methods allow the same retrieval options:
Source-specific Spectra, Timing or Images if you want
to retrieve the Pipeline Products of the EPIC sources
Custom if you want to define a customized sub-set of PPS
products for each observation/exposure (e.g.: only the EPIC
event lists, the RGS spectra and/or the PPS cross-correlation
products). In order to specify the content of the customized set,
click on the Define Custom button. The "Custom Definition"
window (see Fig.10) allows to choose individual datasets and/or
"groups" (event lists, images, source lists, light curves, spectra
and/or finding charts)
The option MULTIPLE can be used to combine any of the
above options in a single retrieval request
Fig.10 - The "Custom Dataset Definition" panel
7.1 Direct data download
Use the direct download button besides each observation or
exposure catalogue record (see Fig.9). This button activates directly
an ftp session, downloading data products corresponding to that
observation or exposure.
Fig.9 - XSA results panel. The Retrieve button for
direct download is located besides each observation (white) or exposure
(blue) record.
For the data covered by proprietary rights, the icon display is disabled.
7.2 Retrieve data through the shopping basket
With this method you will transfer the records of the
observations/exposures/XSA source lists, whose data you want to retrieve,
into a "shopping basket" . This method is envisaged for complex,
multi-observation or multi-exposure retrieval requests. Once the
shopping basket contains all the datasets you intend to retrieve, these
can be transferred to a secure ftp area.
7.2.1 Selecting the data and moving it to the shopping basket
To select data for retrieval click on the check box to the left of
each record of the results list, and then click on
Move selected to basket at the top of the panel. The observation
will disappear from the results screen. To see what is in your shopping
basket, click the Shopping basket button. The observations,
exposures or sources that you intended to move shall now have disappeared
from the Latest Results panel and appeared in the
Shopping basket panel.
It is possible to move to the shopping basket entire observations (with
all its displayed exposure/source records), individual exposures, or
sources. Exposures pertaining to different observations can be freely
selected and combined in the shopping basket. If you want to request
ALL the observations that your query produced you can click on the
Move All to Basket button. This will move all the observations
into the Shopping Basket and it will also take you to the Shopping Basket
so that you can make the request straight away.
7.2.2 In the shopping basket
The shopping basket contains the instantaneous list of all the
observation, exposure and XSA source records, whose data are available
for you to retrieve. You can delete items from the shopping basket by
clicking on the check box to the left of them and then clicking the
Delete selected box to the top right.
When records corresponding to observation, exposure and XSA source
catalogues are simultaneously present in the shopping basket, the button
Next in the top left corner allows to select the type of
products you want to retrieve for each group (observations, exposures
and source products).
A quota on the volume of retrievable observations per user exists if
the shopping basket is used as retrieval method. This quota is
currently set to 2 GBytes per user. The quota is refreshed every three
days. If you think you need an increase of your own quota, please do
not hesitate to contact the
XMM-Newton HelpDesk, so that we can examine which the best way to
help you. No quota exists on the direct download (mechanism 7.1.
above).
When you are happy with your retrieval choice, you can retrieve the
corresponding dataset(s) by clicking on the Submit Request
button.
7.2.3. Retrieval
The last step prompts the overlay of the Request summary
panel. In this panel the number of requested observations, the size
of the dataset and the remaining quota are reported. You can go back
to the Query specification or to the Latest Results
screen if you want to change your query or shopping basket selection,
respectively. You can also choose the preferred mean of delivery,
and the compression format, and whether you prefer the PPS products
as individual files (which can be browsed in the ftp area)
or as a single tarfile (ODFs are always distributed as an individual
gzipped tarfile). Confirm your request with the button at
the bottom of the panel. A request ID is shown. This means that
the generation of the dataset has started. If ftp has been chosen,
an e-mail will be sent to the request originator as soon as the data
are available on the ftp area for retrieval.
8. Interactive generation of scientific products
8.1 Interactive production of spectra and light curves for
EPIC sources
EPIC spectra and light curves are data products and can be downloaded
like other PPS products. Spectra and light curves can also be generated
interactively from the XSA user interface (together with the appropriate
spectral response matrices) through the following steps:
perform a query, with Result Display = Exposures
click on an EPIC exposure image icon to define regions, from
which spectra and light curve can be extracted.
click on Interactive Image Analysis in the pop'd-up
display window
click Spectra/Light on the Survey Product Display
Tool window (see Fig.9). Response matrices are produced alongside
source and background spectra.
define:
Create=Spectra or Create=Light Curve
Binsize=n, where n is the light curve bin
size in seconds (for Light Curve only)
click with the left mouse button on the image (see Fig.10). A
green circle appears
modify the size of the circle by dragging one of highlighted
corners with the right mouse button
change Select source to Select background
repeat steps 6. and 7. above (the circle is now blue)
you may type
selectlib expressions in the Expression widget,
if you wish (standard data screening expression are already
there by default)
click Run
You will receive an e-mail, notifying the availability of the requested
scientific products for ftp retrieval.
Fig.11 - The XSA image viewer. Its sub-panel is activated, which
allows the interactive generation of spectra and light curves. In the
example: the green circle indicates the region, from which the source
spectrum is being extracted; the blue circle indicates the region, from
which the background spectrum is being extracted
Fig.12 - Sub-panel to set the interactive scientific products
extraction parameters
8.2 Process data "on-the-fly" to have the most updated data
products
The XSA contains data processed with different versions of the data
reduction software and calibrations. Although all the data are
periodically reprocessed with the same SAS
version and the same
calibration files, a certain degree of inhomogeneity is unavoidable,
as the XSA is the science archive of an operational mission, whose
content is being continuously upgraded.
In order to alleviate this inhomogeneity, the XSA user interface
provides a tool to process
Observation Data Files with the latest public version of
software and calibrations, with no need to install SAS locally,
through the following steps:
select the observation whose data you want to process, by
clicking on the check box besides the observation record
click the button Move Selected to Basket
move to the "shopping basket" by clicking the button
Shopping Basket at the top of the
Latest Results panel
[at this point you are requested to log in as a registered
XSA user if you have not done it yet. If you do not own an
XSA account yet, create a new one]
select Observation Products=Data Processing
select one of the following items in the menu besides each
record in the shopping basket:
OM Images will process OM image data with
omichain
ALL will process all the instruments data
click on the button Submit Request
click on the button Confirm in the new panel
(Confirmation of Request), that appears automatically.
An e-mail is sent, notifying the availability of processed data for
ftp retrieval.
8.3 Filter of bad events from an EPIC event list prior
to retrieval
This can be done following these steps
select the observation whose data you want to process, by
clicking on the check box besides the observation record
click the button Move Selected to Basket
move to the "shopping basket" by clicking the button
Shopping Basket at the top of the Latest Results
panel
[at this point you are requested to log in as a registered
XSA user if you have not done it yet. If you do not own an XSA
account yet, create a new one]
select Observation Products=Data Processing
select event list for the Observation you want
to process
in the pop'd-up window, select the file corresponding to
the event lists you wish to retrieve
in order to define the data selection expression, you may
follow either one of the following ways:
write the selection expression in the SAS Expression
widget
or:
click on Background GTI Filtering
fill the widgets, corresponding to the data filtering
criteria you intend to apply
click on Generate SAS Expression
click OK
click on the button Submit Request
click on the button Confirm in the new panel
(Confirmation of Request), that appears automatically.
An e-mail will be received, notifying the availability of the requested
scientific products for ftp retrieval.
9. Request Monitor
You can monitor your shopping basket download or data processing
requests, through a specific "Request Monitor" tool. To start it,
click on the Request Monitor button in the top row of
each user interface panel.
The status of each process is color-coded.
10. Exit
Just click on the 'File' menu at the top left of the applet and select
Exit. It is recommended to exit when not used. Note that when
you exit all selections will be lost.
