Page 27 - FCW, November/December 2019

BEST IN CLASS - FEDERAL CIVILIAN
NOAA’s Big Data Project
National Oceanic and Atmospheric Administration, Department of Commerce
Although there are impressive technical elements to the National Oceanic and Atmospheric Administration’s Big Data Project, the partnerships with commercial cloud providers are what truly set the initiative apart.
NOAA collects more than 20 terabytes of data daily from sensors, satellites, weather stations and other systems, and that volume continues to expand. But limitations in storage, processing power and funding meant just a small fraction of NOAA’s data was historically made public.
In 2015, NOAA officials announced a four-year experiment to use financial incentives to encourage private-sector partners to create robust data repositories that the agency could not afford to create and maintain on its own.
Through cooperative research and development agreements with NOAA, five partners — Amazon Web Services, Google, IBM, Microsoft and the Open Commons
Consortium — were allowed to provide public access to any of NOAA’s open data for free, while charging users for value- added services and analytical tools.
The Cooperative Institute for Satellite Earth System Studies serves as a broker between NOAA and the cloud providers.
Those partners have made over 70 agency datasets available to the public, which means that NOAA’s data is not only more accessible but also primed for far more productive use. One partner, for example, funneled data based on more than 300 measurements directly into a FirstNet dashboard to help first responders know how weather could impact disaster response.
The Big Data Project’s experimental phase concluded in 2019, and NOAA is working toward an operational phase in 2020.
November/December 2019 FCW.COM 25