G-LiHT#

This notebook illustrates search, directly loading DEMs, and downloading DEMs and LPCs for local processing

Dataset

Alias

Type

Start

End

Extent

Source

NASA G-LiHT

gliht

LiDAR

2011-07-28

2020-03-12

US Territories

NASA G-LiHT

 
import coincident
import geopandas as gpd
from shapely.geometry import box
import matplotlib.pyplot as plt
# %config InlineBackend.figure_format = 'retina'

/home/docs/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/src/coincident/io/download.py:25: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)
  from tqdm.autonotebook import tqdm

Matplotlib is building the font cache; this may take a moment.

Get Data#

NASA G-LiHT has many different provided gridded datasets. The following collections below are the current datasets supported by coincident.

Collection

Description

GLORTHO_001

orthorectified high-resolution aerial photography

GLCHMT_001

maximum canopy height and canopy variability information

GLDSMT_001

Digital Surface Model, Mean, Aspect, Rugosity, and Slope

GLDTMT_001

bare earth elevation, aspect and slope on the EGM96 Geoid

GLLIDARPC_001

LiDAR Point Cloud data product (LAS format)

Note

Not all G-LiHT flights will contain every single product listed. For example, a flight may have ‘dsm’ data but not ‘ortho’ data.

# Subset a particular scene
collection = "GLDTMT_001"
gf_gliht = gf.query(f'collection == "{collection}"')
gf_gliht.id

10    GLDTMT_SERC_CalTarps_31July2017_am_l9s0_DTM
11    GLDTMT_SERC_CalTarps_31July2017_am_l8s0_DTM
12    GLDTMT_SERC_CalTarps_31July2017_am_l6s0_DTM
13    GLDTMT_SERC_CalTarps_31July2017_am_l2s0_DTM
14    GLDTMT_SERC_CalTarps_31July2017_am_l7s0_DTM
15    GLDTMT_SERC_CalTarps_31July2017_am_l3s0_DTM
16    GLDTMT_SERC_CalTarps_31July2017_am_l1s0_DTM
17    GLDTMT_SERC_CalTarps_31July2017_am_l5s0_DTM
18    GLDTMT_SERC_CalTarps_31July2017_am_l4s0_DTM
19    GLDTMT_SERC_CalTarps_31July2017_am_l0s0_DTM
41        GLDTMT_SERC_ForestGEO_31July2017_am_DTM
Name: id, dtype: str

# Just use the first one
gs_item = gf_gliht.iloc[0]
c = gs_item.geometry.centroid
mini_aoi = gpd.GeoDataFrame(
    geometry=[box(c.x - 0.0045, c.y - 0.0045, c.x + 0.0045, c.y + 0.0045)],
    crs="EPSG:4326",
)

m = gf_gliht.explore(column="id")
mini_aoi.explore(m=m, color="red", style_kwds={"fill": False, "weight": 3})
Make this Notebook Trusted to load map: File -> Trust Notebook

Stream#

First we’ll show how to stream a subset of a gridded DSM directly into Xarray

Important

Unlike the G-LiHT search, you will need NASA Earthdata credentials (aka EarthData Login (EDL)) to read in and download the gridded datasets from G-LiHT. This requires creating an EDL Token and making sure you’ve set the environment variable EARTHDATA_TOKEN=xxxxx

%%time

da = coincident.io.xarray.load_gliht(
    item=gs_item,
    aoi=mini_aoi,
)
da

CPU times: user 3.65 ms, sys: 2.08 ms, total: 5.74 ms
Wall time: 624 ms

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/xarray/backends/file_manager.py:219, in CachingFileManager._acquire_with_cache_info(self, needs_lock)
    218 try:
--> 219     file = self._cache[self._key]
    220 except KeyError:

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/xarray/backends/lru_cache.py:56, in LRUCache.__getitem__(self, key)
     55 with self._lock:
---> 56     value = self._cache[key]
     57     self._cache.move_to_end(key)

KeyError: [<function open at 0x75a86e8b6820>, ('https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/GLDTMT.001/GLDTMT_SERC_CalTarps_31July2017_am_l9s0_DTM/GLDTMT_SERC_CalTarps_31July2017_am_l9s0_DTM.tif',), 'r', (('sharing', False),), '05906d13-38bd-46ef-b92d-418f8f587f58']

During handling of the above exception, another exception occurred:

CPLE_HttpResponseError                    Traceback (most recent call last)
File rasterio/_base.pyx:320, in rasterio._base.DatasetBase.__init__()
--> 320 'Could not get source, probably due dynamically evaluated source code.'

File rasterio/_base.pyx:232, in rasterio._base.open_dataset()
--> 232 'Could not get source, probably due dynamically evaluated source code.'

File rasterio/_err.pyx:359, in rasterio._err.exc_wrap_pointer()
--> 359 'Could not get source, probably due dynamically evaluated source code.'

CPLE_HttpResponseError: HTTP response code: 401 - Failure writing output to destination, passed 77 returned 0

During handling of the above exception, another exception occurred:

RasterioIOError                           Traceback (most recent call last)
Cell In[6], line 1
----> 1 get_ipython().run_cell_magic('time', '', '\nda = coincident.io.xarray.load_gliht(\n    item=gs_item,\n    aoi=mini_aoi,\n)\nda\n')

File <timed exec>:1
----> 1 'Could not get source, probably due dynamically evaluated source code.'

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/src/coincident/io/xarray.py:763, in load_gliht(item, aoi, mask, **kwargs)
    753 # Use a rasterio.Env context to handle authentication for remote NASA access
    754 with rasterio.Env(
    755     GDAL_DISABLE_READDIR_ON_OPEN="EMPTY_DIR",
    756     CPL_VSIL_CURL_USE_HEAD="NO",
   (...)    761     GDAL_HTTP_COOKIEJAR="/tmp/cookies.txt",
    762 ):
--> 763     da = rioxarray.open_rasterio(
    764         data_assets[asset_keys_to_load[0]]["href"],
    765         masked=mask,
    766         **kwargs,
    767     )
    768     da = da.rename({"band": "time"})
    769     da["time"] = item.datetime

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/rioxarray/_io.py:1140, in open_rasterio(filename, parse_coordinates, chunks, cache, lock, masked, mask_and_scale, variable, group, default_name, decode_times, decode_timedelta, band_as_variable, **open_kwargs)
   1138     else:
   1139         manager = URIManager(file_opener, filename, mode="r", kwargs=open_kwargs)
-> 1140     riods = manager.acquire()
   1141     captured_warnings = rio_warnings.copy()
   1143 # raise the NotGeoreferencedWarning if applicable

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/xarray/backends/file_manager.py:201, in CachingFileManager.acquire(self, needs_lock)
    186 def acquire(self, needs_lock: bool = True) -> T_File:
    187     """Acquire a file object from the manager.
    188 
    189     A new file is only opened if it has expired from the
   (...)    199         An open file object, as returned by ``opener(*args, **kwargs)``.
    200     """
--> 201     file, _ = self._acquire_with_cache_info(needs_lock)
    202     return file

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/xarray/backends/file_manager.py:225, in CachingFileManager._acquire_with_cache_info(self, needs_lock)
    223     kwargs = kwargs.copy()
    224     kwargs["mode"] = self._mode
--> 225 file = self._opener(*self._args, **kwargs)
    226 if self._mode == "w":
    227     # ensure file doesn't get overridden when opened again
    228     self._mode = "a"

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/rasterio/env.py:464, in ensure_env_with_credentials.<locals>.wrapper(*args, **kwds)
    461     session = DummySession()
    463 with env_ctor(session=session):
--> 464     return f(*args, **kwds)

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/rasterio/__init__.py:367, in open(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, thread_safe, opener, **kwargs)
    364     path = _parse_path(raw_dataset_path)
    366 if mode == "r":
--> 367     dataset = DatasetReader(path, driver=driver, sharing=sharing, thread_safe=thread_safe, **kwargs)
    368 elif mode == "r+":
    369     dataset = get_writer_for_path(path, driver=driver)(
    370         path, mode, driver=driver, sharing=sharing, **kwargs
    371     )

File rasterio/_base.pyx:329, in rasterio._base.DatasetBase.__init__()
--> 329 'Could not get source, probably due dynamically evaluated source code.'

RasterioIOError: HTTP response code: 401 - Failure writing output to destination, passed 77 returned 0

coincident.plot.plot_dem(da)
plt.title(gs_item.id);

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[7], line 1
----> 1 coincident.plot.plot_dem(da)
      2 plt.title(gs_item.id);

NameError: name 'da' is not defined

Download#

Download multiple STAC Item Assets#

This will put STAC metadata, extended metadata, a browse/thumbnail image, and a TIF in a specified folder

# back to a GeoDataFrame from a Series
gfi = gpd.GeoDataFrame([gs_item])

# And to a Pystac Item
items = coincident.search.to_pystac_items(gfi)
items[0]
<Item id=GLDTMT_SERC_CalTarps_31July2017_am_l9s0_DTM> 
# Download the item assets (browse, metadata, thumbnail)
local_item = await coincident.io.download.download_item(
    items[0],
    path="/tmp/gliht/",
)

---------------------------------------------------------------------------
DownloadError                             Traceback (most recent call last)
Cell In[9], line 2
      1 # Download the item assets (browse, metadata, thumbnail)
----> 2 local_item = await coincident.io.download.download_item(
      3     items[0],
      4     path="/tmp/gliht/",
      5 )

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/src/coincident/io/download.py:141, in download_item(item, path, config)
    138 config = _set_config(item, config)
    140 # NOTE: prevent in-place modification of remote hrefs with item.clone()
--> 141 return await stac_asset.download_item(item.clone(), posixpath, config=config)

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/stac_asset/_functions.py:254, in download_item(item, directory, file_name, infer_file_name, config, messages, clients, keep_non_downloaded, max_concurrent_downloads, stream)
    248 async with Downloads(
    249     config=config or Config(),
    250     clients=clients,
    251     max_concurrent_downloads=max_concurrent_downloads,
    252 ) as downloads:
    253     await downloads.add(item, Path(directory), file_name, keep_non_downloaded)
--> 254     await downloads.download(messages, stream)
    256 self_href = item.get_self_href()
    257 if self_href:

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/stac_asset/_functions.py:173, in Downloads.download(self, messages, stream)
    171             exceptions.append(result.error)
    172 if exceptions:
--> 173     raise DownloadError(exceptions)

DownloadError: ClientResponseError: 401, message='Unauthorized', url='https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/GLDTMT.001/GLDTMT_SERC_CalTarps_31July2017_am_l9s0_DTM/GLDTMT_SERC_CalTarps_31July2017_am_l9s0_DTM.tif'
ClientResponseError: 401, message='Unauthorized', url='https://cmr.earthdata.nasa.gov/search/concepts/G2793865577-LPCLOUD.xml'

Download a single file#

tif_key = [x for x in gs_item.assets.keys() if x.startswith("001")][0]
href = gs_item.assets[tif_key]["href"]
coincident.io.download.download_files([href], "/tmp/")

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
Cell In[10], line 3
      1 tif_key = [x for x in gs_item.assets.keys() if x.startswith("001")][0]
      2 href = gs_item.assets[tif_key]["href"]
----> 3 coincident.io.download.download_files([href], "/tmp/")

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/src/coincident/io/download.py:196, in download_files(files, output_dir, product, chunk_size)
    193     continue
    195 resp = requests.get(url, stream=True, headers=headers, timeout=60)
--> 196 resp.raise_for_status()
    197 total = int(resp.headers.get("content-length", 0))
    198 inner = tqdm(
    199     total=total,
    200     unit="iB",
   (...)    204     leave=False,
    205 )

File ~/checkouts/readthedocs.org/user_builds/coincident/checkouts/stable/.pixi/envs/docs/lib/python3.14/site-packages/requests/models.py:1167, in Response.raise_for_status(self)
   1162     http_error_msg = (
   1163         f"{self.status_code} Server Error: {reason} for url: {self.url}"
   1164     )
   1166 if http_error_msg:
-> 1167     raise HTTPError(http_error_msg, response=self)

HTTPError: 401 Client Error: Unauthorized for url: https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/GLDTMT.001/GLDTMT_SERC_CalTarps_31July2017_am_l9s0_DTM/GLDTMT_SERC_CalTarps_31July2017_am_l9s0_DTM.tif