Missing assets for Sentinel-2

[nafgarda]

We are having problems with some assets that appear on the ESA repository but not in the Planetary computer. Here you have an example from May 1st 2023 over Spain. Just a portion of the orbit appears on the Planetary Computer, meanwhile at ESA is full. I copy the query here:

Regards

from pystac_client import Client
import planetary_computer as pc

# Search against the Planetary Computer STAC API
catalog = Client.open(
  "https://planetarycomputer.microsoft.com/api/stac/v1"
)

# Define your area of interest
aoi = {
  "type": "Polygon",
  "coordinates": [
    [
      [-6.143189189970428, 40.97406119377399],
      [-2.3972230172855973, 40.97406119377399],
      [-2.3972230172855973, 42.63542536197471],
      [-6.143189189970428, 42.63542536197471],
      [-6.143189189970428, 40.97406119377399]
    ]
  ]
}

# Define your temporal range
daterange = {"interval": ["2023-05-01T10:25:31Z", "2023-05-01T23:59:59Z"]}

# Define your search with CQL2 syntax
search = catalog.search(filter_lang="cql2-json", filter={
  "op": "and",
  "args": [
    {"op": "s_intersects", "args": [{"property": "geometry"}, aoi]},
    {"op": "anyinteracts", "args": [{"property": "datetime"}, daterange]},
    {"op": "=", "args": [{"property": "collection"}, "sentinel-2-l2a"]}
  ]
})

# Grab the first item from the search results and sign the assets
first_item = next(search.get_items())
pc.sign_item(first_item).assets

[TomAugspurger]

Thanks for the report. I'll take a look.

[TomAugspurger]

We did have a batch of scenes missing from that date. I've kicked off a job to reingest those. They'll start appearing over the next 30 minutes or so, though it'll take a little while for all of them to show up.

I'm also going to investigate why we were missing those scenes in the first place.

[nafgarda]

Thank you for the fast response. The problem for May 1 is solved.
We identified same issue on Feb 17th on same area. Planetary Computer have no data for Sentinel-2. Can you reingest them?

Another problem we have found is that when ESA split the granule in two parts, only one of the parts appear on Planetary Computer. Lately ESA is splitting many granules in two parts. Both granules have same name except the last characters with the processing time. You may be interpreting that they are duplicated but the truth is that part of the land is in one file and part in another.

Check this two granules as an example:
https://dataspace.copernicus.eu/
S2B_MSIL2A_20230406T110619_N0509_R137_T30TVM_20230406T124210.SAFE
S2B_MSIL2A_20230406T110619_N0509_R137_T30TVM_20230406T140259.SAFE

In Planetary Computer there is only one https://planetarycomputer.microsoft.com/api/stac/v1/collections/sentinel-2-l2a/items/S2B_MSIL2A_20230406T110619_R137_T30TVM_20230406T162119

[TomAugspurger]

Thanks. I'll take at that date as well.

(We're also going to do a complete inventory of the entire history, but that will take some time to implement and run).

I'll also look into the scenes sharing the most of the datatake ID. We did have some issues with that in the past, but the item you shared comes after that fix was implemented.

[nafgarda]

Hello again, we are finding new data gaps in recent dates. Should we expect some kind of improvement or we shoud take it as it is?.

Regards

[TomAugspurger]

We intend to have the entire catalog. I'm looking into this today, but we haven't had a chance to start that historical check yet.

Another problem we have found is that when ESA split the granule in two parts, only one of the parts appear on Planetary Computer. Lately ESA is splitting many granules in two parts. Both granules have same name except the last characters with the processing time. You may be interpreting that they are duplicated but the truth is that part of the land is in one file and part in another.

xref #256

[TomAugspurger]

@nafgarda these latest scenes should all be available now. We're still planning to do an historical census to make sure everything is in sync, but let us know if you spot any missing scenes / dates and we'll address those immediately.

[nafgarda]

Thanks @TomAugspurger. We'll tell you.

[bmcandr]

The assets associated with this item appear to be missing.

[bmcandr]

Found another one.

[ghidalgo3]

Closed due to inactivity, feel free to reopen if you would like to continue this discussion.

[bmcandr]

All Assets for this Item except for preview are missing.