FAQ

Developer FAQs

Derived from a session at the ESIP Summer Meeting in 2012, for more information see Implementing USGIN, a Distributed Data Network for Geoscience Information. For additional developer questions, visit the Developers page or USGIN Lab.

Q: How was Excel chosen, over other tools such as Access or other database files, for standard input into the system? How is the data compiled or expected to be compiled on the provider’s side of things in to these Excel files? What is the process for exporting it to these Excel files? That is, what is the burden on the contributor and how do they feel about the process?

Q: Where are the standards listed for the syntax for the spreadsheet—how do you show which version of Lat/Long, etc.

Q: How did we define the symbology for the WFS? How does the WFS ingest the symbology?

Q: How are we handling multiple authors or resources in a single field on the spreadsheet?

Q: Does the validation on the metadata only review that the field is complete or are their syntax checks in place as well?


Q: What is the NGDS?

A:The National Geothermal Data System (NGDS) is a distributed data system providing access to information resources related to geothermal energy from a network of data providers. Data are contributed by academic researchers, private industry, and state and federal agencies. Built on the scalable and open U.S. Geoscience Information Network (USGIN) platform, NGDS respects data provenance while promoting shared resources. Since NGDS is built using a set of open protocols and standards, relying on the Open Geospatial Consortium (OGC) and International Organization for Standardization (ISO), members of the community can access the data using a variety of proprietary and open-source applications and software. In addition, developers can add functionality to the system by creating new applications utilizing the open protocols and standards of the NGDS.

Q: Who uses the NGDS?

A: NGDs users broadly fall into three groups: data providers, data consumers, and developers. Data providers may have environmental resource or geophysical data intended for wide availability and distribution or subscription-controlled access. Data consumers include state and federal government entities concerned with natural resources and geophysical data preservation, energy corporations accessing data for areas of potential resource (i.e., geothermal energy) development, as well as individuals conducting research or interested in local geophysical data. Due to the open standards and protocols on which the NGDS is built, developers who are interested in creating a new application, for example a data gap analysis or financial analysis, can build their application to consume data resources in the NGDS. Some applications that can consume NGDS data are listed on the Tools & Applications page.

This site is designed to invite and address the interests of each of these users. Data providers should visit the Contribute portion of the site; data consumers should visit the Map or Library search portals or Tools & Applications portion of the site; and developers should utilize the Developers portion of the site.

Q: How do I access or zero-in on NGDS data?

A: The NGDS portal application that you are using supports basic search and discovery of datasets, with some simple browse capability to view data accessible via map services in a map view, or to view tabular-structured data in a table view. In order to perform actual data analysis, data will need to be downloaded or a user application will need to connect to the web services providing the data if those are available. Data published using OGC web map services can be viewed in the NREL Geothermal Prospector using the 'View in Prospector' button if that appears in the results listing. The Tools & Applications page lists a variety of other applications that can be used with NGDS web services. If you’re a developer interested in how you can build an application for the NGDS, visit our Developer’s page, which includes a mechanism for submitting your application to the NGDS.

Q: How do I submit or contribute data?

A: If you are a U.S. DOE Geothermal Technologies Office awardee, data that is a product of DOE funding should be submitted to the DOE Geothermal Data Repository. Other data providers can easily set up your own node if you have access to a web server (see Installing GINstack). Alternatively, you can make arrangements with an existing NGDS data publisher to register your dataset, or contact an NGDS aggregator to harvest your metadata conforming to the NGDS metadata profile into the NGDS catalog. In any case, your data will need to be accessible online (go here for more data accessibility information).

Q: Will NGDS own my data if I contribute to the system?

A: No, the intention of the system is to allow data providers to maintain ownership of the authoritative version of their data. Data can be registered/contributed to the system in a variety of ways. The simplest and easiest method for contributing data is to create a metadata record for your resource and have it harvested into the NGDS catalog (see above). In this instance, you will maintain the data at your site (either in hard copy or digitally on your server/website); the metadata provides a description of your dataset and how to access it. If your dataset is deployed in an OGC-compliant web service, it is provided to the public as a read-only dataset that can be downloaded and queried in geospatial software against other related datasets. The Node-in-a-Box software stack allows you to create and publish OGC compliant web services.

Q: What are Web Services?

A: A web service is a computer program that provides functional capabilities that can be invoked via a request (message) sent using the World Wide Web. Web services are the primary means by which digital geoscience information can be shared according to NGDS specifications. Services are well suited to this task for a number of reasons: owing to standardized data requests, they can be accessed from a variety of platforms; they are read-only and preserve data ownership; they come in a number of varieties for different purposes. An important operation of a web service includes the message format used to encode information which is sent to the server in a request, and the encoded information returned by the server. For the Web Map Service (WMS) examples on this page, the message format of particular interest is the XML encoding used to transmit requested data back to the client computer. Web Feature Services (WFS) provide structured data encoded in XML. The NGDS uses the 'simple feature' WFS profile, which provides a simple XML format that can be utilized directly by programs like ESRI ArcGIS or Microsoft Excel.

For a data consumer, using a web service allows you to access a distributed network of data from many sources having disparate data types.

>Q: Who is currently providing data to the NGDS?

>A: There are currently five NGDS projects each representing a broad range of participants. A complete list of all project participants can be found at the Data Contributors portion of this site.

Q: What is United States Geoscience Information Network (USGIN)?

A: USGIN is a platform for data publication on the WWW based on a collection of protocols and standards. The platform is a distributed data-sharing network that uses open-source software and existing World Wide Web infrastructure and browsers. The central concepts are standardized metadata and catalog protocols, and documented 'Information Exchanges' to enable interoperable data sharing. For more information visit the USGIN website.


Q: How was Excel chosen, over other tools such as Access or other database files, for standard input into the system? How is the data compiled or expected to be compiled on the provider’s side of things in to these Excel files? What is the process for exporting it to these Excel files? That is, what is the burden on the contributor and how do they feel about the process?

A: It is important to distinguish between “standard input into the system” and these Excel files. From the system’s perspective, “standard input” is data provided according to a specific content model (that is, has the right fields) and accessible via WFS requests. The Excel files that we have built are simply tools to help data providers map their data, in whatever format they have it, into a system-standard content model. The Excel files do not represent the only way for data to be included into the system; they’re simply a tool to assist in data transformation.

Excel files were chosen as they are a great way to organize your data into a scheme that USGIN can use to make your data interoperable. You may search the names of the content models at Data Interchange Content Models and choose the content model that best suits your data type. These excel files allow several tabs to explicate the process, data types, specific terms, and other information valuable to the data provider. Often, we do have project participants that submit Access databases to us and we work with them to help parse that data into content models, and are happy to do so. We like to assess what data the provider has and help them to submit their data. Additionally, these content models (given on our site as Excel files) may be viewed in Access, allowing the data provider to add data from their database, then submit to us, all using Access. (Actually this is great from our standpoint, as at this point, we can view the Access database in ArcCatalog and transform the data into the desired Geodatabase Feature Class for service deployment.)

Q: Where are the standards listed for the syntax for the spreadsheet—how do you show which version of Lat/Long, etc.

A: The versions of Latitude, Longitude, and all other fields are explicitly requested in a tab within the template excel file for any given dataset. This tab is the “Field List” where a short description of the intended data and data type are given for each of the field headings in the content model. See the complete information exchanges at the USGIN Schemas site.

Q: How did we define the symbology for the WFS? How does the WFS ingest the symbology?

A: Strictly speaking, there's no such thing as symbology in a WFS. WFS conveys the raw data. We wrote SLD and ESRI Layer files, which are descriptions of how to look at the raw data and determine what a default symbolization should be; these files must be used when deploying a tier 3, standardized WMS (which is a symbolized picture of the data rather than just a data dump).

Q: How are we handling multiple authors or resources in a single field on the spreadsheet?

A: They're just pipe-delimited in a standard Excel content model spreadsheet. Advanced scripting later parses them separately.

Q: Does the validation on the metadata only review that the field is complete or are their syntax checks in place as well?

A: There is some level of "syntax" checking going on, but it isn't particularly robust. It’s more like logic tests: if you told me the author's email address then you don't have to tell me her phone number—or, your abstract ought to have more than 50 characters in it.