How to reference and retrieve another RO-Crate #296

stain · 2024-04-25T23:46:17Z

This PR fixes #228 #160

Generalizes the Content-negotiate-or-signposting section from not just Profile Crates.

For ZIP files this is still vague in that it says If the retrieved resource is a ZIP file (Content-Type: application/zip), then extract ro-crate-metadata.json, or, if the archive root only contains a single folder (e.g. folder1/), extract folder1/ro-crate-metadata.json
I've also added BagIt reference as this would be a second folder, e.g. folder1/data/ro-crate-metadata.json and then the checksums should be verified first as we do in https://trefx.uk/5s-crate/0.4/#check-phase

As for referencing another RO-rate from another, either the referenced RO-Crate can have its own distribution with a conformsTo:

  {
    "@id": "./",
    "@type": "Dataset",
    "identifier": "https://doi.org/10.48546/workflowhub.workflow.775.1",
    "url": "https://workflowhub.eu/workflows/775/ro_crate?version=1",
    "name": "Research Object Crate for Jupyter Notebook Molecular Structure Checking",
    "distribution": {"@id": "https://workflowhub.eu/workflows/775/ro_crate?version=1"},
    "…": ""
  },
  {
    "@id": "https://workflowhub.eu/workflows/775/ro_crate?version=1",
    "@type": "DataDownload",
    "encodingFormat": ["application/zip", {"@id": "https://www.nationalarchives.gov.uk/PRONOM/x-fmt/263"}],
    "conformsTo": { "@id": "https://w3id.org/ro/crate" }
  }

or it can have a subjectOf to a ro-crate-metadata.json:

{
  "@id": "http://example.com/another-crate/",
  "@type": "Dataset",
  "conformsTo": { "@id": "https://w3id.org/ro/crate" },
  "subjectOf": { "@id": "http://example.com/another-crate/ro-crate-metadata.json" }
},
{
  "@id": "http://example.com/another-crate/ro-crate-metadata.json",
  "@type": "CreativeWork",
  "encodingFormat": "application/ld+json"
}

As used by the 5s-crate profile: https://trefx.uk/5s-crate/0.4/#referencing-a-workflow-crate

stain · 2024-05-20T19:13:51Z

Could @dgarijo or @ptsefton have a look at this? I've used it here:
https://stain.github.io/workflow-run-crate/profiles/0.5-DRAFT/process_run_crate/ro-crate-preview.html#https%3A//www.researchobject.org/workflow-run-crate-paper/mapping/

Perhaps we should add that isPartOf pattern as well on how to mention a file within another crate? (Could get tricky to make absolute URIs..)

dgarijo · 2024-05-20T19:16:28Z

Will do when I finish the ISWC reviews that are due tomorrow :((( El lun., 20 may. 2024 9:14 p. m., Stian Soiland-Reyes < ***@***.***> escribió:

…

Could @dgarijo <https://github.com/dgarijo> or @ptsefton <https://github.com/ptsefton> have a look at this? I've used it here: https://stain.github.io/workflow-run-crate/profiles/0.5-DRAFT/process_run_crate/ro-crate-preview.html#https%3A//www.researchobject.org/workflow-run-crate-paper/mapping/ Perhaps we should add that isPartOf pattern as well on how to mention a file within another crate? (Could get tricky to make absolute URIs..) — Reply to this email directly, view it on GitHub <#296 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALTIGT5ZN6HYRP3WSEQEI3ZDJDQLAVCNFSM6AAAAABGZZMP2CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRRGA2DMNZXGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

dgarijo · 2024-05-27T09:23:37Z

Thanks @stain , I have had a look. The only thing that it is not fully clear to me is where the distribution information is supposed to be added: is it on the crate referencing the other crate, or in the referenced crate metadata?

For example, let's say crate A references crate B. Usually I would add a link in A to B. But here you recommend adding also where B is stored, correct? As opposed to adding a link to B, and hoping that when I resolve the id I get a JSON-LD with the distribution information.

The only potential issue I see is that distributions may not have persistent ids. If the link from A to B persists, but the distribution is hosted elsewhere in the meantime, B has no means to tell A about this. But I am ok with this limitation

docs/1.2-DRAFT/data-entities.md

dgarijo

Small clarification

stain · 2024-07-11T09:17:40Z

Think this is good to go in now, waiting final approval from @dgarijo or @jmfernandez

stain · 2024-07-18T08:48:25Z

I can't seem to resolve the requested changes from @dgarijo although I have addressed them in earlier commits.

dgarijo · 2024-07-18T08:50:45Z

Sorry @stain I am not available right now for review. Will do as soon as I can.

stain · 2024-08-22T23:20:49Z

After meeting 2024-08-22, revised to not require subjectOf declaration if the RO-Crate metadata file can be resolved. This meant adding a section on how the absolute URI of the RO-Crate can be determined if there is no identifier

I now feel this should be split out into a page separate from data-entities.md

dgarijo · 2024-08-23T08:14:36Z

Once again, apologies. Will review by the end of the week.

dgarijo · 2024-08-24T17:19:01Z

docs/_specification/1.2-DRAFT/data-entities.md

+
+##### Referencing RO-Crates that have a persistent identifier
+
+If the referenced RO-Crate B has a `identifier` declared as B's [Root Data Entity identifier](root-data-entity#root-data-entity-identifier), then this is a _persistent identifier_ which SHOULD be used as the URI in the `@id` of the corresponding entity in RO-Crate A.  For instance, if crate B had declared the identifier `https://pid.example.com/another-crate/` then crate A can reference B as an entity:


a identifier -> an identifier

dgarijo · 2024-08-24T17:20:45Z

docs/_specification/1.2-DRAFT/data-entities.md

+If an `identifier` is not declared in a referenced RO-Crate B, but the determined absolute URI has [Signposting] declared for a `Link:` with `rel=cite-as`, then that link MAY be considered as an equivalent permalink for B.
+
+
+##### Determening entity identifier for a referenced RO-Crate


Determening -> Determining

dgarijo · 2024-08-24T17:24:19Z

docs/_specification/1.2-DRAFT/data-entities.md

+
+##### Referencing another metadata document
+
+If a referenced RO-Crate Metadata Document is known at a given URI or path, but its corresponding RO-Crate identifier can't be determined as above (e.g. [Retrieving an RO-Crate](#retrieving-an-ro-crate) fails or requires heuristics), then an referenced metadata descriptor entity SHOULD be added. For instance, if `http://example.com/another-crate/ro-crate-metadata.json` resolves to an RO-Crate Metadata Document describing root `./`, but `http://example.com/another-crate/` always return a HTML page without [Signposting] to the metadata document, then `subjectOf` SHOULD be added to an explicit metadata descriptor entity, which has `encodingFormat` declared for JSON-LD:


then an referenced metadata descriptor -> then a referenced metadata descriptor

dgarijo

Minor editorial changes. It looks very nice. The bit on content negotiation helps the issue I opened about retrieving contents, since you may add a distribution where to retrieve the zipped contents now (issue #228)

jmfernandez · 2024-08-28T12:09:47Z

docs/_specification/1.2-DRAFT/data-entities.md

+    "@id": "https://workflowhub.eu/workflows/775/ro_crate?version=1",
+    "@type": "DataDownload",
+    "encodingFormat": ["application/zip", {"@id": "https://www.nationalarchives.gov.uk/PRONOM/x-fmt/263"}],
+    "conformsTo": { "@id": "https://w3id.org/ro/crate" }


I understand that this is needed to be correct describing downloadable RO-Crate. But, in this scenario, if the graph does not contain an ro-crate-metadata.json (or similar) node, or contains it without a proper conformsTo, shall we assume that metadata should appear at the RO-Crate fetchable from the downloadable content?

This is a cheeky triple-use of our conformsTo as we define what the profile mean on different files.. so here the implication is that if there is https://w3id.org/ro/crate as profile of a DataDownload then yes, there should be a ro-crate-metadata.json inside it's archive, which presumably would have its own (versioned) conformsTo declared. Or if it's a Dataset with the same conformsTo then the metadata file is found differently.

We also use the same profile as part of the JSON-LD content type https://www.researchobject.org/ro-crate/specification/1.2-DRAFT/appendix/jsonld.html#ro-crate-json-ld-media-type

You could argue this should be several different profile URIs.. I think it's a bit overloading, but if you squeeze your eyes a bit it's all saying "an RO-Crate" just in different ways in.

But we won't try to list the content of the zip-file in the referring crate, unless we need to (which would bring in arcp etc), so that's a nice simplification that we moved to having the references on the Dataset rather than requiring mention of the second RO-Crate Metadata Document.

https://signposting.org/adopters/#workflowhub Signposting also uses the same profile for the ZIP file. But that should also be valid to mean "There will be an ro-crate-metadata.json inside, as stated in top of section "#### Retrieving an RO-Crate"

jmfernandez

Although heuristics are properly described, I'm realizing we are going to need in some moment code snippet examples in a couple of programming languages to demonstrate how to properly fetch an RO-Crate with negotiation.

stain · 2024-08-29T10:19:59Z

Thanks everyone, will merge now after fixing latest round of typos! Then we see if we need to move it out.

stain added 7 commits April 25, 2024 23:47

Define how to reference an RO-Crate

83dc72b

clarify encodingFormat

46de12f

Add Retrieving an RO-Crate section

d465ee0

Reference generalized "How to retrieve a Profile Crate"

0969f4a

reference relocated how-to-retrieve section

1744a20

allow nested ro-crate-metadata.json

394aeb7

also BagIt!

783a4d8

stain requested a review from dgarijo April 25, 2024 23:46

stain mentioned this pull request Apr 26, 2024

Soften Profile inheritence rules #262

Merged

stain self-assigned this May 7, 2024

stain mentioned this pull request May 17, 2024

Add Profile Crates for 0.5-DRAFT ResearchObject/workflow-run-crate#73

Merged

stain mentioned this pull request May 23, 2024

Peter's draft on publishing RO-Crates #220

Closed

stain added this to the RO-Crate 1.2 milestone May 23, 2024

This was referenced May 23, 2024

Use Case: Resolve RO-Crate from persistent identifier/DOI with landing page #160

Closed

Content negotiation for RO-Crate identifier #297

Closed

dgarijo reviewed May 27, 2024

View reviewed changes

docs/1.2-DRAFT/data-entities.md Outdated Show resolved Hide resolved

dgarijo requested changes May 27, 2024

View reviewed changes

stain added 4 commits July 11, 2024 08:37

Merge branch 'main' into referencing-crates

40346c8

clarify where the metadata goes

445344b

about ZIP file

e79b073

also about @id and identifier

c08ee14

stain requested a review from dgarijo July 11, 2024 09:17

fix internal anchor links

9d9c641

stain requested a review from jmfernandez July 18, 2024 08:00

stain mentioned this pull request Jul 19, 2024

Use Case: Building combined knowledge graph of nested RO-Crates #342

Open

stain added 3 commits August 22, 2024 21:26

Update data-entities.md

0af8e56

More algorithm!

2574205

Way too many details

782eb72

#downloadable-dataset

3cc0c16

dgarijo reviewed Aug 24, 2024

View reviewed changes

dgarijo requested changes Aug 24, 2024

View reviewed changes

jmfernandez reviewed Aug 28, 2024

View reviewed changes

jmfernandez approved these changes Aug 28, 2024

View reviewed changes

stain added 3 commits August 29, 2024 10:59

typo

e51b1ea

Typo

ca3c1f5

Typo

fedb143

stain merged commit d92d9f1 into main Aug 29, 2024
1 check passed

stain deleted the referencing-crates branch August 29, 2024 10:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to reference and retrieve another RO-Crate #296

How to reference and retrieve another RO-Crate #296

stain commented Apr 25, 2024

stain commented May 20, 2024

dgarijo commented May 20, 2024 via email

dgarijo commented May 27, 2024

dgarijo left a comment

stain commented Jul 11, 2024

stain commented Jul 18, 2024

dgarijo commented Jul 18, 2024

stain commented Aug 22, 2024

dgarijo commented Aug 23, 2024

dgarijo Aug 24, 2024

dgarijo Aug 24, 2024

dgarijo Aug 24, 2024

dgarijo left a comment

jmfernandez Aug 28, 2024

stain Aug 29, 2024

stain Aug 29, 2024

jmfernandez left a comment

stain commented Aug 29, 2024


		##### Referencing RO-Crates that have a persistent identifier

		If the referenced RO-Crate B has a `identifier` declared as B's [Root Data Entity identifier](root-data-entity#root-data-entity-identifier), then this is a _persistent identifier_ which SHOULD be used as the URI in the `@id` of the corresponding entity in RO-Crate A. For instance, if crate B had declared the identifier `https://pid.example.com/another-crate/` then crate A can reference B as an entity:

		If an `identifier` is not declared in a referenced RO-Crate B, but the determined absolute URI has [Signposting] declared for a `Link:` with `rel=cite-as`, then that link MAY be considered as an equivalent permalink for B.


		##### Determening entity identifier for a referenced RO-Crate


		##### Referencing another metadata document

		If a referenced RO-Crate Metadata Document is known at a given URI or path, but its corresponding RO-Crate identifier can't be determined as above (e.g. [Retrieving an RO-Crate](#retrieving-an-ro-crate) fails or requires heuristics), then an referenced metadata descriptor entity SHOULD be added. For instance, if `http://example.com/another-crate/ro-crate-metadata.json` resolves to an RO-Crate Metadata Document describing root `./`, but `http://example.com/another-crate/` always return a HTML page without [Signposting] to the metadata document, then `subjectOf` SHOULD be added to an explicit metadata descriptor entity, which has `encodingFormat` declared for JSON-LD:

How to reference and retrieve another RO-Crate #296

How to reference and retrieve another RO-Crate #296

Conversation

stain commented Apr 25, 2024

stain commented May 20, 2024

dgarijo commented May 20, 2024 via email

dgarijo commented May 27, 2024

dgarijo left a comment

Choose a reason for hiding this comment

stain commented Jul 11, 2024

stain commented Jul 18, 2024

dgarijo commented Jul 18, 2024

stain commented Aug 22, 2024

dgarijo commented Aug 23, 2024

dgarijo Aug 24, 2024

Choose a reason for hiding this comment

dgarijo Aug 24, 2024

Choose a reason for hiding this comment

dgarijo Aug 24, 2024

Choose a reason for hiding this comment

dgarijo left a comment

Choose a reason for hiding this comment

jmfernandez Aug 28, 2024

Choose a reason for hiding this comment

stain Aug 29, 2024

Choose a reason for hiding this comment

stain Aug 29, 2024

Choose a reason for hiding this comment

jmfernandez left a comment

Choose a reason for hiding this comment

stain commented Aug 29, 2024