Moving NOAA Data on the NGI

John Kinsfather
NESDIS / NGDC

Massive and rapidly growing volumes of NOAA data need to be moved around the country from points of collection to processing centers, modelers, product developers, distribution centers, archives, and numerous other end users. NOAA does not maintain a dedicated WAN for all of these purposes so must rely on shared use of various NGI linkages. One major source of new data coming to NOAA is from the NASA Earth Observing System (EOS). NOAA will become the long term archive and distribution facility for the EOS data. One EOS instrument, MODIS, on one satellite generates 250+ gigabytes of data and products per day. NGDC in Boulder has been testing and prototyping the operations involved for transferring and ingesting hundreds of gigabytes of MODIS data per day from NASA Goddard Space Flight Center (GSFC). This test utilized (1) the NGDC LAN, (2) the Boulder David Skaggs Research Center network, (3) the Boulder Research and Administrative Network (BRAN), (5) WAN connectivity vis the Abilene network and (6) GSFC networks connecting to the distribution facility. While the smallest network pipe, end to end, is 155 megabit OC3, nothing close to that rate can be achieved at present. Funding for these tests was provided by the NOAA HPCC program, NOAA CLASS program, NASA, and others sources.

Details are presented on the knowledge gained, changes made (LAN, MAN, WAN, hardware & software) to improve throughput, and requirements to support operational transfers of massive amounts of data in the future.

BIO - John Kinsfather Chief of the Information Services Division of the NESDIS National Geophysical Data Center for ten years. Educational background in Geology, Geophysics, Oceanography and Computer Science at the University of Hawaii, Scripps Institute of Oceanography and University of Colorado. Worked 4 years with the Naval Oceanographic Office followed by 25 years with NOAA collecting, archiving, accessing and distributing environmental data and information. A long time continuing involvement deploying leading edge information technologies for effective data management.

Conference Center - Paper Tuesday - 2:00 - 2:20 P.M.