Error while publishing CamTrap resource

[vechocho]

While publishing this error appears:

Publishing version #1 of resource 2885622f-fe86-4ffa-b1ed-045014edc966 failed: Archive generation for resource 2885622f-fe86-4ffa-b1ed-045014edc966 failed: Problem occurred while adding metadata file to data package folder

The log says this: log.txt

[mike-podolskiy90]

Thank you for reporting the issue @vechocho
I will have a look

[vechocho]

I tried on the UAT IPT: https://ipt.gbif-uat.org/manage/resource?r=2c370b08-1338-4c5d-9e16-0c7626180e41 with the same result.

[mnhn-paul]

Hi, I seem to get the same issue:

ERROR 14-Jan-2025 07:59:50 [org.gbif.ipt.task.GenerateDataPackage] - Exception occurred trying to generate data package for resource farmland bird survey - 2024 test (farmlandbirdsurvey): Problem occurred while adding metadata file to data package folder
org.gbif.ipt.task.GeneratorException: Problem occurred while adding metadata file to data package folder

This happens to me when trying to publish a new dataset recently downloaded from Agouti.
When republishing an already existing dataset on the IPT there is no error.

tested on Integrated Publishing Toolkit (IPT) Version 3.1.0

also tested on Version 3.0.6 : same error

[vechocho]

Mine was downloaded from agouti too, maybe @peterdesmet knows something?

[peterdesmet]

Can you try to validate the package you are trying to upload?

1. First install frictionless-py

pip install frictionless

2. Then validate the archive you are trying to upload:

frictionless validate path/to/datapackage.json

This should report any validation issues with the data and metadata.

[vechocho]

Thanks @peterdesmet this error is presented:

The zip was exported from agouti:

paisajes-ct.zip

[mnhn-paul]

can confirm the same error for my .json

InvalidAnchor: '#$defs/version' is not a valid anchor, neither as a plain    │
│ name anchor nor as a JSON Pointer. You may have intended to use              │
│ '#/$defs/version', as the slash is required *before each segment* of a JSON  │
│ pointer.       

From what I can see the .json referres to this: https://raw.githubusercontent.com/tdwg/camtrap-dp/1.0/camtrap-dp-profile.json

it seem this contains a "$ref": "#$defs/version"
Which to my undestanding of the checker should be "$ref": "#/$defs/version"

[peterdesmet]

Right, this error is picked up in recent versions of frictionless-py. Can you validate with an older version:

pip install frictionless==5.15.0
frictionless validate path/to/datapackage.json

Note: this is solved in the 1.0.1 patch of Camtrap DP, but that isn't exported by Agouti yet.

[mnhn-paul]

Ok tried this, now the validation is running. I'm getting the error that there are deployments that do not have any coordinates added to them. This is a required field however.
I'll try to add some coordinates to those deployments at then publish via IPT

Tables ────────────────────────────────────
                                  deployments                                   
┏━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Row ┃ Field ┃ Type             ┃ Message                                     ┃
┡━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 10  │ 4     │ constraint-error │ The cell "" in row at position "10" and     │
│     │       │                  │ field "latitude" at position "4" does not   │
│     │       │                  │ conform to a constraint: constraint         │
│     │       │                  │ "required" is "True"                        │
│ 10  │ 5     │ constraint-error │ The cell "" in row at position "10" and     │
│     │       │                  │ field "longitude" at position "5" does not  │
│     │       │                  │ conform to a constraint: constraint         │

I fixed the missing coordinates in my csv, but that does not seem to be solving the Archive generation for resource farmland failed: Problem occurred while adding metadata file to data package folder issue.

After changing the missing coordinates the validate path/to/datapackage.json validates the files without issue

[vechocho]

I have this message:

(base) C:\Users\choch>frictionless validate C:\Users\choch\Downloads\paisajes-ct\datapackage.json

dataset

┏━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ name ┃ type ┃ path ┃ status ┃
┡━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ deployments │ table │ deployments.csv │ VALID │
│ media │ table │ media.csv │ VALID │
│ observations │ table │ observations.csv │ VALID │
└──────────────┴───────┴──────────────────┴────────┘`

It seems that everything is ok so i don't know what is happening on the IPT

[peterdesmet]

Thanks @vechocho and @mnhn-paul! We can now rule out that the issue is caused by invalid packages, since both your (corrected) packages are valid. Sees to be IPT related then. @mike-podolskiy90 can you investigate this further?