Use Case - Granule Search (CWIC/CMR)

Use Case - Granule Search (CWIC/CMR)#

This chapter provides a comprehensive and detailed process about how to implement a WGISS OpenSearch client, which includes how to retrieve the IDN collection ID for the collection of interest, and how to build an OpenSearch request for a CWIC data partner. Within CMR, CWIC partners are available directly from CMR. The IDN response records contain direct links to the OSDD from the CWIC partner server, where possible. This can be used to construct a granule search directly from the partner.

Retrieve Collection OSDD via IDN OpenSearch#

The steps in this section are identical to the steps explained in the section Retrieve Collections via IDN OpenSearch. To fit the end-to-end CWIC use case, the parameter values for the collection search has been selected in such a way that the granule search is eventually performed through CWIC.

Use Case Overview
Title	Clients start from the IDN OpenSearch
Description	This use case describes steps for retrieving a collection ID from the IDN OpenSearch.
Actors	OpenSearch client, IDN OpenSearch Server
Initial Status and Preconditions	Clients have the IDN OpenSearch access URL.

The following steps describe this use case.

Step 1

Obtain the IDN OpenSearch OSDD to formulate a valid IDN OpenSearch request.

The OpenSearch Description Document is accessible at the fixed location https://cmr.earthdata.nasa.gov/opensearch/collections/descriptor_document.xml?clientId=ceosOpenSearchDoc and contains the URL template to be used for collection search.

response = requests.get( url_osdd )

xmlstr = minidom.parseString(response.text).toprettyxml(indent='  ',newl='')
# md("```xml\n" + xmlstr + "\n```\n")

Find the URL template for collection search with Atom responses.

root = ElementTree.fromstring(response.text)

ns = {'os': 'http://a9.com/-/spec/opensearch/1.1/'}
collection_url_atom = root.find('os:Url[@rel="collection"][@type="application/atom+xml"]', ns)

collection_template = collection_url_atom.attrib['template']
collection_template

'https://cmr.earthdata.nasa.gov/opensearch/collections.atom?keyword={os:searchTerms?}&instrument={echo:instrument?}&satellite={eo:platform?}&boundingBox={geo:box?}&lat={geo:lat?}&lon={geo:lon?}&radius={geo:radius?}&geometry={geo:geometry?}&placeName={geo:name?}&startTime={time:start?}&endTime={time:end?}&cursor={os:startPage?}&numberOfResults={os:count?}&offset={os:startIndex?}&uid={geo:uid?}&hasGranules={echo:hasGranules?}&isCwic={echo:isCwic?}&isGeoss={echo:isGeoss?}&isCeos={echo:isCeos?}&isEosdis={echo:isEosdis?}&isFedeo={echo:isFedeo?}&provider={echo:provider?}&clientId=ceosOpenSearchDoc'

Step 2

Search collections of interest through IDN OpenSearch with proper request parameters (e.g. spatial footprint, temporal extent and keyword). A complete list of supported request parameters, extracted from the IDN OpenSearch OSDD, was included in the section Available Collection Search Criteria.

An example request can be formed as follows.

request_url = get_api_request(collection_template, {'count': '1', 'searchTerms': 'Landsat_8'})
request_url

'https://cmr.earthdata.nasa.gov/opensearch/collections.atom?keyword=Landsat_8&numberOfResults=1&clientId=ceosOpenSearchDoc'

response = requests.get( request_url )
xmlstr = minidom.parseString(response.text).toprettyxml(indent='   ', newl='')
md("```xml\n" + xmlstr + "\n```\n")

<?xml version="1.0" ?><feed xmlns:relevance="http://a9.com/-/opensearch/extensions/relevance/1.0/" xmlns:os="http://a9.com/-/spec/opensearch/1.1/" xmlns:georss="http://www.georss.org/georss" xmlns="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:echo="https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#atom" xmlns:esipdiscovery="http://commons.esipfed.org/ns/discovery/1.2/" xmlns:gml="http://www.opengis.net/gml" xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/" xmlns:eo="http://a9.com/-/opensearch/extensions/eo/1.0/" esipdiscovery:version="1.2">   
     <updated>2024-12-06T16:28:26.794Z</updated>   
     <id>https://cmr.earthdata.nasa.gov/opensearch/collections.atom</id>   
     <author>      
          <name>CMR</name>      
          <email>echodev@echo.nasa.gov</email>      
     </author>   
     <title type="text">ECHO dataset metadata</title>   
     <subtitle type="text">Search parameters: keyword =&gt; Landsat_8</subtitle>   
     <link href="https://cmr.earthdata.nasa.gov/opensearch/granules/descriptor_document.xml" hreflang="en-US" type="application/opensearchdescription+xml" rel="search"/>   
     <link href="https://cmr.earthdata.nasa.gov/opensearch/collections.atom?keyword=Landsat_8&amp;numberOfResults=1&amp;clientId=ceosOpenSearchDoc" hreflang="en-US" type="application/atom+xml" rel="self"/>   
     <link href="https://cmr.earthdata.nasa.gov/opensearch/collections.atom?keyword=Landsat_8&amp;numberOfResults=1&amp;clientId=ceosOpenSearchDoc&amp;cursor=2" hreflang="en-US" type="application/atom+xml" rel="last"/>   
     <link href="https://cmr.earthdata.nasa.gov/opensearch/collections.atom?keyword=Landsat_8&amp;numberOfResults=1&amp;clientId=ceosOpenSearchDoc&amp;cursor=1" hreflang="en-US" type="application/atom+xml" rel="first"/>   
     <link href="https://wiki.earthdata.nasa.gov/display/echo/Open+Search+API+release+information" hreflang="en-US" type="text/html" rel="describedBy" title="Release Notes"/>   
     <os:Query xmlns:geo="http://a9.com/-/opensearch/extensions/geo/1.0/" role="request" os:searchTerms="Landsat_8"/>   
     <os:totalResults>1</os:totalResults>   
     <os:itemsPerPage>1</os:itemsPerPage>   
     <os:startIndex>1</os:startIndex>   
     <entry>      
          <id>https://cmr.earthdata.nasa.gov/opensearch/collections.atom?uid=C1235542031-USGS_LTA</id>      
          <author>         
               <name>CMR</name>         
               <email>echodev@echo.nasa.gov</email>         
          </author>      
          <consortium>CEOS</consortium>      
          <consortium>CWIC</consortium>      
          <title type="text">Landsat 8</title>      
          <summary type="text">The Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) are onboard the Landsat 8 satellite, have acquired images of the Earth since  February 2013. The sensors collect images of the Earth with a 16-day repeat cycle, referenced to the Worldwide Reference System-2. The approximate scene size is 170 km north-south by 183 km east-west (106 mi by 114 mi).

Landsat 8 image data files consist of 11 spectral bands with a spatial resolution of 30 meters for bands 1-7 and bands 9-11; 15-meters for the panchromatic band 8. Delivered Landsat 8 Level-1 data typically include both OLI and TIRS data files; however, there may be OLI-only and/or TIRS-only scenes in the USGS archive.  A Quality Assurance (QA.tif) band is also included. This file provides bit information regarding conditions that may affect the accuracy and usability of a given pixel – clouds, water or snow, for example.</summary>      
          <updated/>      
          <link rel="enclosure" hreflang="en-US" href="http://glovis.usgs.gov"/>      
          <link rel="enclosure" hreflang="en-US" href="http://earthexplorer.usgs.gov"/>      
          <link rel="describedBy" hreflang="en-US" href="http://landsat.usgs.gov/landsat8.php"/>      
          <link length="0.0KB" rel="http://esipfed.org/ns/fedsearch/1.1/search#" hreflang="en-US" href="https://cmr.earthdata.nasa.gov/opensearch/granules/descriptor_document.xml?collectionConceptId=C1235542031-USGS_LTA"/>      
          <link href="https://cmr.earthdata.nasa.gov/opensearch/granules/descriptor_document.xml?collectionConceptId=C1235542031-USGS_LTA" hreflang="en-US" type="application/opensearchdescription+xml" rel="search" title="Non-CMR OpenSearch Provider Granule Open Search Descriptor Document"/>      
          <link href="https://cmr.earthdata.nasa.gov/search/concepts/C1235542031-USGS_LTA.xml" hreflang="en-US" type="application/xml" rel="via" title="Product metadata"/>      
          <dc:identifier>C1235542031-USGS_LTA</dc:identifier>      
          <dc:date>2013-02-11T00:00:00.000Z/</dc:date>      
          <echo:datasetId>Landsat 8</echo:datasetId>      
          <echo:shortName>Landsat_8</echo:shortName>      
          <echo:versionId>Not provided</echo:versionId>      
          <echo:entryId>Landsat_8</echo:entryId>      
          <echo:dataCenter>USGS_LTA</echo:dataCenter>      
          <echo:archiveCenter>DOI/USGS/EROS</echo:archiveCenter>      
          <echo:organization>DOI/USGS/EROS</echo:organization>      
          <echo:processingLevelId>Not Provided</echo:processingLevelId>      
          <echo:coordinateSystem>CARTESIAN</echo:coordinateSystem>      
          <echo:orbitParameters/>      
          <georss:box>-82.71 -180 82.74 180</georss:box>      
          <echo:hasVariables>false</echo:hasVariables>      
          <echo:hasFormats>false</echo:hasFormats>      
          <echo:hasTransforms>false</echo:hasTransforms>      
          <echo:hasCombine>false</echo:hasCombine>      
          <echo:hasSpatialSubsetting>false</echo:hasSpatialSubsetting>      
          <echo:hasTemporalSubsetting>false</echo:hasTemporalSubsetting>      
          <echo:cloudHosted>false</echo:cloudHosted>      
          <relevance:score>0.97999995</relevance:score>      
          <echo:tag>         
               <echo:tagKey>org.ceos.wgiss.cwic.granules.prod</echo:tagKey>         
               <echo:data>&quot; &quot;</echo:data>         
          </echo:tag>      
          <echo:tag>         
               <echo:tagKey>org.ceos.wgiss.cwic.granules.provider</echo:tagKey>         
               <echo:data>&quot;USGSLSI&quot;</echo:data>         
          </echo:tag>      
          <echo:tag>         
               <echo:tagKey>org.geoss.geoss_data-core</echo:tagKey>         
          </echo:tag>      
          <echo:tag>         
               <echo:tagKey>gov.nasa.eosdis</echo:tagKey>         
          </echo:tag>      
          <echo:is_geoss>true</echo:is_geoss>      
          <echo:is_eosdis>true</echo:is_eosdis>      
     </entry>   
</feed>

Step 3

From the IDN OpenSearch response, obtain the OSDD endpoint for the collection by parsing the href attribute under <link rel="search" type=”application/opensearchdescription+xml” >

root = ElementTree.fromstring(response.text)
# Extract <link> element with the OSDD for the granule search
el = root.find('{http://www.w3.org/2005/Atom}entry/{http://www.w3.org/2005/Atom}link[@rel="search"][@type="application/opensearchdescription+xml"]')
xmltxt = ElementTree.tostring(el, encoding='unicode', method='xml')
md("```xml\n" + xmltxt + "\n```\n")

<ns0:link xmlns:ns0="http://www.w3.org/2005/Atom" href="https://cmr.earthdata.nasa.gov/opensearch/granules/descriptor_document.xml?collectionConceptId=C1235542031-USGS_LTA" hreflang="en-US" type="application/opensearchdescription+xml" rel="search" title="Non-CMR OpenSearch Provider Granule Open Search Descriptor Document" />
    

This response will provide the key component of searching CMR for the desired collection using the CMR conceptID. For Landsat 8, this will be C1235542031-USGS_LTA. Now obtain from CMR the URL for the local OSDD (in this case, at USGS), with the request

url_osdd = el.attrib['href']
el.attrib['href']

'https://cmr.earthdata.nasa.gov/opensearch/granules/descriptor_document.xml?collectionConceptId=C1235542031-USGS_LTA'

Retrieve the response (i.e. the OSDD document).

response = requests.get( el.attrib['href'] )

xmlstr = minidom.parseString(response.text).toprettyxml(indent='  ',newl='')
md("```xml\n" + xmlstr + "\n```\n")

<?xml version="1.0" ?><feed xmlns="http://a9.com/-/spec/opensearch/1.1/" xmlns:geo="http://a9.com/-/opensearch/extensions/geo/1.0/" xmlns:params="http://a9.com/-/spec/opensearch/extensions/paramseters/1.0/" xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/" xmlns:eop="http://a9.com/-/opensearch/extensions/eo/1.0/">  
    <title>OpenSearch Exception</title>  
    <subtitle type="text">No native id present within the `org.ceos.wgiss.cwic.granules.native_id` tag.</subtitle>  
    <updated>2024-12-06T16:28:27Z</updated>  
    <id>https://cmr.earthdata.nasa.gov/opensearch/granules/descriptor_document.xml?collectionConceptId=C1235542031-USGS_LTA</id>  
    <link title="self" rel="self" type="application/atom+xml" href="https://cmr.earthdata.nasa.gov/opensearch/granules/descriptor_document.xml?collectionConceptId=C1235542031-USGS_LTA"/>  
    <totalResults>0</totalResults>  
</feed>

Warning

In the previous step, the OSDD could not be retrieved as the URL included in the response is not the one with short_name. See also step 5 in chapter 5 “Outline” with the workaround to fix that.

Retrieve Granules via OpenSearch#

Available Granule Search Parameters#

After retrieving the OSDD endpoint by querying through the IDN OpenSearch, OpenSearch clients will sequentially interact with the host remote server for inventory search. The following table shows the basic information about the use case of interacting with WGISS Connected Data Assets via OpenSearch.

TBD