PPS Frequently Asked Questions (FAQ)

The Frequently Asked Questions (FAQ) help page has been organized into the following topics questions (Click on the topic of interest to see the answer):

Contents:

User Account and General Information
STORM- Data Search, Custom Subsets and Subscriptions- PPS Data Ordering Interface
THOR- Orbit Viewer Information
Retrieving and Reading PPS Data Products

 

PPS User Accounts and General Information

 

STORM- Data Search, Custom Subsets and Subscriptions- PPS Data Ordering Interface

 

THOR- Orbit Viewer Information

 

Retrieving and Reading PPS Data Products

 

PPS User Accounts and General Information

 

  1. How do I register to obtain TRMM, GPM and partner data from PPS?

    2. PPS no longer requires users to have a special science user account which provided a dedicated personal ftp area and STORM account. Instead, GPM and TRMM data users can acquire data and use STORM after completing a brief and simple registration process.

    All access to the online PPS ftp data archive for GPM, TRMM and partner data at::
    ftp://arthurhou.pps.eosdis.nasa.gov/
     (direct ftp and STORM access) will require a registered email address to be entered. Please note that both your user name and password is your registered email address. You can register now at:

    http://registration.pps.eosdis.nasa.gov/
     For more details:
    http://pps.gsfc.nasa.gov/register.html

    When you register, an e-mail will be sent to you. Please click on the link and update "Contact Information and Area of Interest".

    Data can also be ordered using our PPS Data Ordering Interface (STORM).
    https://storm-pps.gsfc.nasa.gov after registering

     

  • What are TRMM, GPM, PPS/TSDIS, TRMM Office, and the DAAC / DISC?

    • The Tropical Rainfall Measuring Mission (TRMM) is a satellite mission which provides the first detailed and comprehensive data set on the four dimensional distribution of rainfall and latent heating over vastly undersampled oceanic and tropical continental regimes. To get additional information, see the TRMM homepage:

    TRMM has its own unique Precipitation Processing System (PPS), formerly known as the TRMM Science Data and Information System (TSDIS) at GSFC to process the information from three satellite instruments and four ground radars. PPS analyzes the rainfall data and also provides validation from nearly a dozen ground radar sites. The PPS homepage is located at:

    GPM (Global Precipitation Measurement) is an international satellite mission launched by NASA and JAXA on Feb. 27, 2014, that will set new standards for precipitation measurements worldwide using a network of satellites united by the GPM Core Observatory. See the GPM homepage located at:

    The TRMM Office provides support for TRMM, in connection with the ground based validation of the TRMM satellite observations. The TRMM Office homepage is located at:

    The operational archiving and distribution of TRMM science data products is provided by the Goddard Distributed Active Archive Center (DAAC), now known as DISC (Data and Information Services Center). In addition to archiving and distributing the TRMM science data, the DISC also provides necessary information and support for manipulating these data files, which are provided in NCSA's Hierarchical Data Format (HDF). The Goddard DISC homepage is located at:

     

  • How do I obtain TRMM, GPM and partner data?

    All data users must complete a simple registration process as detailed above in "How do I register to obtain TRMM, GPM and partner data from PPS?".  Once registered, users can obtain data directly through our arthurhou ftp area ftp://arthurhou.pps.eosdis.nasa.gov or order data through PPS's STORM data ordering interface where you can search for data, create custom subsets and establish data subscriptions.

 

 

 

 

STORM- Data Search, Custom Subsets and Subscriptions- PPS Data Ordering Interface

 

  • Where can I find more information on how to use PPS's STORM?

    • Please go directly to the STORM web portal at: https://storm-pps.gsfc.nasa.gov   For complete instructions, user should dowload the Adobe PDF format document "PPS STORM User Guide"

     

  • Can the PPS Order Notifications be sent to more than one person and/or place?

    • The PPS ordering system is designed to send notices to a single user, presumably the registered user as established through PPS's simple registration process.. That single "user" could also be a mailing list alias defined on your computer. Using an alias is the simplest method, although a bit cumbersome since a system administrator is needed to make changes to the mailing list. You may also want to consider setting up a majordomo list or using mail filtering.   To set up the local mailing list alias using Linux, complete the following:

      -Set up a local mailing list alias in /etc/aliases:

      -Example: pps-users: user1, user2@their.computer.edu, user3

      -Run "newaliases" so your send mail will be aware of the changes.

      -Verify the alias is working.

     

  • I tried to order a large quantity of data or broad range of dates but receive a warning message that there are too many granules that meet the search criteria. What is the best way get my desired data?

    • In STORM, the size of the search results is limited by 2,000 granules.  Please use a narrower date range, select fewer product types, or select a smaller geographic area. For example, a user could split a large order with an expansive date range or an order that contains many data products into smaller multiple chronological orders for a shorter period of time or place multiple orders with fewer data types. 

     

  • Can I preview data images from the STORM web site before ordering?

    • Once you search for data products and obtain search results, certain GPM, TRMM and partner data products can be previewed directly from the "Search Results" area under the column "Download/View".  Click on the camera icon to see a preview low resolution browse image of the selected file. A variety of data products can also be analyzed and viewed with PPS's THORonline which is a web based version of the "Orbit Viewer-THOR" .  To see the selected file in THOR, please click on the "THOR" icon also in the Download/View" column.

     

  • What is the difference between a Standing Order and a Special Request?

    A Special Request is made when ordering a product that has already been produced. Special Requests are filled soon after placing your order and you will receive an E-mail data notification to your registered email address.

    A Standing Order is submitted for a product that has not yet been produced and is being requested for an extended period of time. Standing Orders are processed daily and are filled as the files are produced. Data products will be delivered to your ftp "pgs/so" directory. A data notice will be emailed to you at the provided email address.

    If you are requesting both past and future products, or only future products, use the following method:

    1. Enter Product Type, Start Date/Time, and Stop Date/Time from the Order TRMM Products Page
    2. Select the desired products by checking the boxes to the left or using the "Select All Granules" button. At any time during this step you can start anew by clicking on the "Clear All Selections"
    3. It is very IMPORTANT to note, that if you are only interested in establishing a Standing Order for future products, you will still need to select a past date/time first and the desired product type. For each data type you must at least select one granule to establish a Standing Order for TRMM future products.
    4. Once you are satisfied with your selections select "Continue Order" at the bottom of the page.
    5. You are now presented with two ordering options
    6. Enter Yes for "Submit standing order for future products with these criteria" if you want the selected products produced in the past plus all future products of the type/types indicated.
    7. You also have the additional options of selecting the only the "product" or "browse" files or both at the "Check the product types you would like included with this order" option.
    8. Select "Submit Request" to place your order

    You will be presented with the "Request Status" confirmation page which will indicate if your request was successful along with your order number "Request ID" number. At this stage you can "logout" , go back and order additional TRMM data products, or explore additional features by selecting the "Services" button.

    Additionally, from the "STORM Services" page you can select "Track Order Status" which will provide a detailed list and history of your order/orders.

THOR- Orbit Viewer Information

 

  • Does PPS provide any software to read and visualize the dta that I have downloaded?

    • Yes, PPS develops and provides  the free "Orbit Viewer THOR" software written in the IDL language which is a tool for displaying the standard data products of the Global Precipitation Measurement (GPM) mission, partner satellite data and for the Tropical Rainfall Measuring Mission (TRMM). THOR stands for the Tool for High-resolution Observation Review. THOR displays GPM and TRMM products on a map of the Earth when possible, and otherwise, as 2D plots or text output. Since 1997, the Precipitation Processing System (PPS) at NASA Goddard has developed and distributed many versions of Orbit Viewer THOR.    

 

Retrieving and Reading PPS Data Products

 

  • I've placed an order through STORM and received a data notice, but have questions on how to download the data?

    • After you place your order, you will receive a "PPS Order Notification" Email from the Helpdesk containing details on where the files can be found and downloaded from our "arthurhou" ftp server. There are also several text based attachments included; one file (ftp_url....) contains the ftp_url of all the files in that order; while another file, (ftp_get...) is a script which can be used to automate the download/s. The other attached file (RequestParameters....) documents the specific parameters for your requested files.

    The following is an example of how to use the "ftp_get_.txt." script within a MS Windows environment

    -First, copy the script file: ftp_get_xxx_xxxxxxxxxxxx.txt
    to the intended directory; for example:
    C:\users\myaccount>
    ( Note: Your user account name in place of the above \myaccount example)

    -From the Windows command prompt which can be accessed via the "Run" Windows start menu item (type 'cmd') or via the "accessories" file folder from the main Windows Start click on the Windows"Command Prompt"

    -Then run the script (ftp -n -s:name_of_scriptfile) Example:
    c:\users\myaccount>ftp -n -s:ftp_get_xxx_xxxxxxxxxxxx.txt
    (name of the scriptfile)

    -The data files in that script will be downloaded to the target directory C:\users\myaccount

    -PPS data users can also use a dedicated FTP client to access the PPS FTP location files.
    If you require any further assistance, please send Email to: helpdesk@mail.pps.eosdis.nasa.gov

     

  • I am using the Safari web browser on a Mac OSX computer and cannot login to PPS's ftp data archives such as: ftp://arthurhou.pps.eosdis.nasa.gov/ using the browser interface. Is this a known problem and is there a solution?

    • There is known compatibility issue with the Mac OSX Safari web browser and how it cannot resolve the ftp login challange on our arthurhou and other ftp servers such as PPS's GPM NRT server jsimpson.  PPS web development and system programming staff currently recommend that Mac OS users use Mozilla's Firefox web browser or the Google Chrome browser for full compatibility when visiting these ftp areas via a web browser on a Mac OS system. PPS will continue to investigate this issue.

     

  • I use a special script or other automated method to download GPM data from PPS's FTP site, but am encountering access problems.

    • Although PPS does not block our ftp users from running their own scripts, etc.for data retrieval, etc., PPS does limit the number of concurrent ftp sessions per user (registered Email address) to three. We recommend either modifying your code or make sure that you try to minimize the number of active (concurrent) ftp sessions.  For instance; if you have your web browser open when visiting the "arthurhou" ftp site and then open another session to run your code, you may be exceeding this connection limit.  We limit concurrent ftp sessions to 3 connections per registered user.   This measure is neccessary to ensure that our network maintains an acceptible throughput and avoids prolonged periods of slowness due to an excessive number of open ftp sessions.

    It is also important to note that users who employ scripts or other methods to obtain PPS data enable passive mode, “PASV” as the correct option.

     

  • How long will my ordered data be available for retrieval?

    • Once you have received your PPS Order Notification message/s, the files will be available for a period of approximately 2 weeks from the time of the Email/s. It is best to retreive your data as soon as possible once the PPS Oder Notifications are delivered to you. If you are away, or have not yet obtained the data after the preset time of the data purge, then you will have to place a new data order through STORM for the missed products.

     

  • How can I read the HDF files?

    • Orbital TRMM and GPM data products are in HDF format, which may be accessed in several ways:

    1. Algorithm developers use the PPS TKIO Science Algorithm toolkits to write the TRMM and GPM files, so the toolkit is ideal for reading the files into your own programs. The PPS Toolkit (TKIO) for GPM is available from our FTP location:

      ftp://gpmweb2.pps.eosdis.nasa.gov/pub/PPStoolkit/GPM/

    2. The PPS TRMM toolkit is callable from both C and FORTRAN. You can download this from:
    3. The PPS Orbit Viewer-THOR is a tool written in IDL to display all GPM,TRMM and some partner satellite HDF files. This is a viewing tool only, and does not allow any data manipulation. Please see the "tutorial.pdf" document for background information, instructions on how to install the software and how to use THOR. The Orbit Viewer can be downloaded from the PPS THOR FTP area. Downloading the most current version of THOR for your respective operating system is recommended:
    4. NASA's Atmospheric Science Data Center has a page devoted to HDF format:

      5. The "Public Domain and Commercial HDF Tools" section also has more information on viewing and manipilating HDF data.

       

  • What Type of File Transfer Protocals are supported by PPS?

    • Currently, PPS supports the following file transfer protocols: FTP and SFTP . Many FTP software clients already provide for secure FTP transfers (sftp, etc.) such as "Filezilla" or "WS_FTP Professional" , to name a few.

     

  • I would like to obtain TRMM data from the PPS "arthurhou" FTP server directly, but am unfamiliar with the various directories.

    Within theTRMM Production FTP Directory Structure located at:
    ftp://arthurhou.pps.eosdis.nasa.gov/trmmdata/

    The /trmmdata directory serves two purposes.  First, the trmmdata directory is the archive that the TRMM and GPM data ordering system - STORM uses to obtain TRMM data.  The most user-friendly way to obtain TRMM data files may be to use STORM by going to https://storm-pps.gsfc.nasa.gov/storm/ .  Secondly, the trmmdata directory is a place where researchers can download data files via an FTP client in their web browser or a stand-alone FTP client such as the "ftp" command in Linux.  Only some of the subdirectories of trmmdata are likely to be useful to researchers looking at TRMM data with an FTP client rather than through STORM.

    If you know which TRMM HDF4 data product you are interested in, you can obtain the granules of that data product in the ByDate subdirectory of trmmdata.  Other specialized directories that may be of interest are the 3G subdirectory, which contains the 3G family of ASCII gridded products, and the GIS subdirectory, which contains the TMPA 3-hour gridded surface precipitation product converted into a GIS data-format (TIFF+WorldFile).

    An alphabetical list of the subdirectories of the top-level /trmmdata directory is listed below.  Not all of the directories are of interest to first time TRMM researchers.

    • 3G: Contains the "3G" family of data products which contain ASCII text gridded TRMM data. Each file contains either 24 one-hour grids to summarize one day or contains one grid that summarizes an entire month.  The file format described in the following documents available on ftp://trmmopen.pps.eosdis.nasa.gov/pub/ : README.3G68, README.3G01, README.3G68.QuarterDegree, and README.3G68Land .
    • Auxiliary: Contains orbit-definition files not intended for use by typical TRMM researchers.
    • Browse: Contains one PNG browse image for each HDF4 granule of many TRMM standard data products.  Browse images are rarely downloaded directly from the FTP site and are instead viewed from a link in the Search Result table of the STORM data ordering system: https://storm-pps.gsfc.nasa.gov/storm/ .
    • ByDate: Perhaps the most popular FTP directory for obtaining TRMM standard products.  Organized by reprocessing data version, year, month, and day.  The file format of these files are documented in the V7 file specification: http://pps.gsfc.nasa.gov/Documents/filespec.TRMM.V7.pdf .
    • ByInstrument: An alternative FTP directory for obtaining TRMM standard products.  Organized the same way as the ByDate subdirectory, except that an instrument-specific subdirectory occurs in between data version and year. 
    • CSI: This directory is no longer active.  Prior to the launch of GPM, it contained subsets of TRMM standard products over pre-defined geographic regions.
    • Climatology: Contains files in the TRMM Composite Climatology developed by Wang, Adler, Hoffman, and Bolvin.  A journal article on this topic is available here: http://journals.ametsoc.org/doi/abs/10.1175/JCLI-D-13-00331.1 .
    • GIS: Contains a GIS-friendly reformatted version of the TMPA data product (aka TRMM 3B42).  TMPA is a 3-hourly average global grid of surface rainfall estimates from a collection of microwave radiometers.  The GIS-reformatted version is a 2-byte TIFF image plus an ESRI WorldFile text metadata file.  This format is documented here: ftp://trmmopen.gsfc.nasa.gov/pub/trmmdata/GIS/README.GIS.pdf
    • L0: This subdirectory contains low-level packet-data downloaded from the TRMM satellite.  Typical TRMM researchers are unlikely to be interested in this data. Furthermore, this data is not written in HDF4, so it cannot be read by PPS tools such as the THOR data viewer.
    • PF: The files of the TRMM Precipitation Feature database.  These files are a useful way to summarize rain events observed by TRMM, although they are not designated as a TRMM standard product.  Documentation of the TRMM Precipitation Feature database is available here: http://trmm.chpc.utah.edu/ .
    • SpecialLH: Contains latent heating products of interest only to latent-heating researchers.
    • shortTermAuxiliary, shortTermGV, and shortTermMOC: Does not contain files of interest to beginning TRMM researchers.

      •  

  • Is there a GPM data product equivalent to the TRMM TMPA (3B4x) Products?

    • Yes, The Integrated Multi satellitE Retrievals for GPM (IMERG) Final Run, which is the post real-time research product in the IMERG suite of products , is now available at PPS at https://storm.pps.eosdis.nasa.gov and ftp://arthurhou.pps.eosdis.nasa.gov/

    IMERG is presently available for the period mid-March 2014 to the present (delayed by about 3 months), which is the most current data given the latency of the ECMWF input data required for the production runs. The half-hourly products have the prefix “3B-HHR”and the monthly products have the prefix “3B-MO” . The complete file naming convention can be found at :  http://pps.gsfc.nasa.gov/Documents/FileNamingConventionForPrecipitationProductsForGPMMissionV1.4.pdf

    Example:

    half-hourly 10.5067/GPM/IMERG/HH/3B
    monthly 10.5067/GPM/IMERG/MONTH/3B

    Note that you will need to be a registered user to access the data. Register online at :http://registration.pps.eosdis.nasa.gov

     

  • I have a general question regarding TRMM or GPM File specifications and range values as stated in the documentation?

    • For example, if the filespec says the range is 0-3000 but the data is always within 0-50, then the range is still correct.
    If a datum was 4000, then either the data is in error or the range should be changed.

    Only if the data is outside the stated range could the range be questioned.

     

  • I am using a script or other automated method to download large volumes of GPM and/or partner data and am encountering problems with the way my username/ password address is resolved?

    • there is no ability to register an account without using an email address as the login username and password. However, certain script and automation problems can occur because of the way the "@" symbols in the Email addresses are resolved. One solution is to modify the the username/password (address) in the URL. The first two "@" symbols have to be replaced with their Unicode representation "%40". For example; if you are trying to obtain files from our near realtime "jsimpson" server in an automated fashion, you would use something similar to the example given below:

    ftp://scienceuser%40gmail.com:scienceuser%40gmail.com@jsimpson.pps.eosdis.nasa.gov/NRTPUB/imerg/gis/2015/12/3B-HHR-L.MS.MRG.3IMERG.20151203-S000000-E002959.0000.V03E.7day.tif.gz