produce-xml-converter.xsl: XSpec tests and minor XSLT improvements #87

galtm · 2023-11-20T18:48:44Z

Committer Notes

Closes #86

XSpec tests:

XSpec tests for produce-xml-converter.xsl in isolation
Stub of tests for produce-xml-converter.xsl from the context of produce-json-converter.xsl
Helper XSLT for use in XSpec tests
Sample definition maps to provide testing contexts

XSLT improvements:

Declare data types where known
Declare context item for named templates
Start XPath path from context instead of root, for testability

I ran the XSpec tests in Oxygen 26.0 and there are no failures.

All Submissions:

Have you followed the guidelines in our Contributing document?
Have you checked to ensure there aren't other open Pull Requests for the same update/change?
Have you squashed any non-relevant commits and commit messages? [instructions]
Do all automated CI/CD checks pass?

Changes to Core Features:

Have you added an explanation of what your changes do and why you'd like us to include them?
Have you written new tests for your core changes, as applicable?
Have you included examples of how to use your new feature(s)?
Have you updated all website and readme documentation affected by the changes you made? Changes to the website can be made in the website/content directory of your branch.

galtm · 2023-11-20T18:49:42Z

src/converter-gen/testing/xml-to-json/produce-xml-converter-json-usage.xspec

+    <!-- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -->
+    <!-- produce-json-converter.xsl reaches these templates by name
+        instead of by match -->
+    <x:scenario label="Tests for name=make-template template" pending="TODO">


The TODO notations in this file are because I jotted down needs for future work. I anticipate that the work would be a separate pull request.

src/converter-gen/xml-to-json/produce-xml-converter.xsl

galtm · 2023-11-20T18:54:10Z

src/converter-gen/xml-to-json/produce-xml-converter.xsl

@@ -190,18 +199,19 @@
                <xsl:call-template name="provide-namespace"/>
                <xsl:apply-templates select="." mode="make-key-flag"/>
                <xsl:apply-templates select="*" mode="make-pull"/>
-                <xsl:variable name="keyname" select="@_key-name"/>


I moved this variable inside xsl:if because that's the only place where the variable is used

galtm · 2023-11-20T18:56:29Z

src/converter-gen/xml-to-json/produce-xml-converter.xsl

-    <xsl:template priority="10" match="*[@scope = 'global'] | group[*/@scope = 'global']" mode="make-xml-match">
+    <xsl:template priority="10" match="*[@scope = 'global'] | group[*/@scope = 'global']" mode="make-xml-match"
+        as="text()">


Below this point, it looks like I changed a lot of lines. What I actually did was add @as attributes and wrap the template output in xsl:value-of. The reason for the latter is that when I considered the output data type of the template, it seemed simpler to consolidate multiple text nodes into a single text node.

galtm · 2023-11-20T18:58:17Z

src/converter-gen/xml-to-json/produce-xml-converter.xsl


    <!-- A field without a GI is implicit in the XML; Metaschema prevents it from having flags -->
-    <xsl:template priority="2" match="field[empty(@gi)][(.|value)/@as-type='markup-multiline']" mode="make-xml-pull">
+    <!-- This template takes precedence over the one that matches field, unqualified
+        (even without priority=2 QUESTION: remove priority=2?) -->


Reviewers: Should I remove priority=2 or leave it as is?

Hm, inclined to say leave as is, but maybe there's a reason to remove I am missing?

I thought the priority wasn't necessary, but I'm fine with leaving it. I pushed a new commit that removes the parenthetical part of the comment.

galtm · 2023-11-20T18:59:34Z

src/converter-gen/testing/xml-to-json/produce-xml-converter.xspec

+            <!-- QUESTION: The context is contrived and might not be the right way
+                to exercise the code in the situation where $parent-match contains
+                multiple match expressions. Does the context need fixing? -->


Reviewers: I wasn't sure about how to set up this scenario.

galtm · 2023-11-20T18:59:49Z

src/converter-gen/testing/xml-to-json/produce-xml-converter.xspec

+                    but no team members have recursive="true", so disambiguation
+                    of 1st and 3rd items is not needed.
+                -->
+                <!-- QUESTION: Is it OK that they're not distinct? -->


Question for reviewers...

XSpec tests: - XSpec tests for produce-xml-converter.xsl in isolation - Stub of tests for produce-xml-converter.xsl from the context of produce-json-converter.xsl - Helper XSLT for use in XSpec tests - Sample definition maps to provide testing contexts XSLT improvements: - Declare data types where known - Declare context item for named templates - Start XPath path from context instead of root, for testability

github-actions · 2024-03-08T22:31:46Z

XSpec Test Results

2 files ±0 40 suites ±0 0s ⏱️ ±0s
105 tests ±0 90 ✅ ±0 15 💤 ±0 0 ❌ ±0
114 runs ±0 99 ✅ ±0 15 💤 ±0 0 ❌ ±0

Results for commit 04f5159. ± Comparison against base commit 04a4a4c.

♻️ This comment has been updated with latest results.

galtm · 2024-03-08T22:37:44Z

I rebased my commit off the develop branch and force-pushed, to refresh this pull request. The checks ran and all passed.

However, I don't think the new XSpec tests that are in this pull request are being picked up in the CI/CD testing.

wendellpiez · 2024-06-04T21:04:14Z

@galtm it is possible the configuration for XSpec runtimes here is skipping yours - we will have to look. Most likely there is a line to add in a Makefile or something similar.

What is your current sense? -- after merging the earlier two outstanding PRs I wonder if we should look at this together. The CI/CD doesn't actually have to run yet (in my view we can come back to that) but just in case there is something else. Apart from that question, how close is this to ready?

galtm · 2024-06-05T12:10:31Z

Apart from that question, how close is this to ready?

There are two comments that contain the word "reviewers" or "Reviewers" and that haven't been resolved yet.

galtm · 2024-06-05T12:13:24Z

What is your current sense? -- after merging the earlier two outstanding PRs I wonder if we should look at this together. The CI/CD doesn't actually have to run yet (in my view we can come back to that) but just in case there is something else.

Sure, we can look at it together next time we talk. Or, if you are inclined to merge this even before the CI/CD runs the new tests, I'm OK with that. If the CI/CD somehow picks up the new tests after the PR is merged, great; if not, one of us can create a GitHub issue so we won't forget to come back to this later.

wendellpiez · 2024-06-05T13:09:39Z

Another option maybe we should consider is merging this PR into develop, but into a common branch where we can test further, prior to the develop merge.

Indeed, such a staging branch might be a good place also to

Consolidate JSON Schema generation improvements #108 prior to merging it into develop
Sort out CI/CD issues and house rules on XSpec deployment

And of course can work out remaining code/test issues there also.

As for xsl:template/@priority, I don't mind leaving them around when they make things more explicit. Of course comments make things more explicit as well.

Thanks Amanda!

galtm · 2024-06-05T14:27:18Z

I have no opinions about branching approaches, so whatever you want to do is fine with me.

wendellpiez · 2024-06-06T22:20:50Z

So ... I made branch merge-June2024 as a place to pull things together.

Let's try pointing this PR there and seeing what trouble we get into. The merge branch is the same as develop right now so I expect no issues.

Then on that branch I can address #117 before merging into develop.

wendellpiez · 2024-06-07T19:12:47Z

I'm merging this into a merge branch where the work can be reviewed before pushing up to develop.

Also will work on #117 in that branch.

* produce-xml-converter.xsl: XSpec tests and minor XSLT improvements (#87) * XSpec tests and minor XSLT improvements * JSON Schema generation improvements (#108) Refactoring submodules; XProc-based testing; XSpec testing of JSON Schema generation; improvements to JSON Schema generation - addressing #105 also #2007. * June 2024 Integration (#118) Integrating branches * June 2024 in-progress merge (#120) Includes updates to unit testing, Inspector XSLT, JSON Schema (#105) and others * More testing (#122) Mostly testing support - * Adding Makefile in compose phase to run XSpec - presently showing errors! * Extending and reorganizing testing, including removing temporary artifacts from git * Adding subpipeline mistakenly omitted from the last PR * New Makefile for testing/tinydata * Mending XSpecs testing Metaschema composition (regression protection); also a new tinydata sample. * Update src/testing/tinydata/tiny-data/terms-of-art.xml (AJ Stein) Co-authored-by: Wendell Piez <[email protected]> Co-authored-by: Amanda Galtman <[email protected]> Co-authored-by: A.J. Stein <[email protected]>

galtm commented Nov 20, 2023

View reviewed changes

galtm changed the title ~~XSpec tests and minor XSLT improvements~~ produce-xml-converter.xsl: XSpec tests and minor XSLT improvements Nov 20, 2023

galtm force-pushed the testing-produce-xml-converter branch from c2c751b to 9744dc8 Compare March 8, 2024 22:30

Merge branch 'develop' into testing-produce-xml-converter

42a437b

Update a comment

04f5159

galtm mentioned this pull request Jun 5, 2024

Make CI/CD include newer XSpec tests #117

Open

wendellpiez mentioned this pull request Jun 6, 2024

JSON Schema generation improvements #108

Merged

8 tasks

wendellpiez changed the base branch from develop to merge-June2024 June 7, 2024 19:10

wendellpiez marked this pull request as ready for review June 7, 2024 19:10

wendellpiez requested a review from a team as a code owner June 7, 2024 19:10

wendellpiez merged commit a0816ac into usnistgov:merge-June2024 Jun 7, 2024
3 checks passed

galtm mentioned this pull request Jun 10, 2024

Testing and repairs including JSON Schema generation enhancements #119

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

produce-xml-converter.xsl: XSpec tests and minor XSLT improvements #87

produce-xml-converter.xsl: XSpec tests and minor XSLT improvements #87

galtm commented Nov 20, 2023 •

edited

Loading

galtm Nov 20, 2023

galtm Nov 20, 2023

galtm Nov 20, 2023

galtm Nov 20, 2023

wendellpiez Jun 4, 2024

galtm Jun 5, 2024 •

edited

Loading

galtm Nov 20, 2023

galtm Nov 20, 2023

github-actions bot commented Mar 8, 2024 •

edited

Loading

galtm commented Mar 8, 2024

wendellpiez commented Jun 4, 2024 •

edited

Loading

galtm commented Jun 5, 2024

galtm commented Jun 5, 2024

wendellpiez commented Jun 5, 2024 •

edited

Loading

galtm commented Jun 5, 2024

wendellpiez commented Jun 6, 2024

wendellpiez commented Jun 7, 2024 •

edited

Loading

produce-xml-converter.xsl: XSpec tests and minor XSLT improvements #87

produce-xml-converter.xsl: XSpec tests and minor XSLT improvements #87

Conversation

galtm commented Nov 20, 2023 • edited Loading

Committer Notes

All Submissions:

Changes to Core Features:

galtm Nov 20, 2023

Choose a reason for hiding this comment

galtm Nov 20, 2023

Choose a reason for hiding this comment

galtm Nov 20, 2023

Choose a reason for hiding this comment

galtm Nov 20, 2023

Choose a reason for hiding this comment

wendellpiez Jun 4, 2024

Choose a reason for hiding this comment

galtm Jun 5, 2024 • edited Loading

Choose a reason for hiding this comment

galtm Nov 20, 2023

Choose a reason for hiding this comment

galtm Nov 20, 2023

Choose a reason for hiding this comment

github-actions bot commented Mar 8, 2024 • edited Loading

XSpec Test Results

galtm commented Mar 8, 2024

wendellpiez commented Jun 4, 2024 • edited Loading

galtm commented Jun 5, 2024

galtm commented Jun 5, 2024

wendellpiez commented Jun 5, 2024 • edited Loading

galtm commented Jun 5, 2024

wendellpiez commented Jun 6, 2024

wendellpiez commented Jun 7, 2024 • edited Loading

galtm commented Nov 20, 2023 •

edited

Loading

galtm Jun 5, 2024 •

edited

Loading

github-actions bot commented Mar 8, 2024 •

edited

Loading

wendellpiez commented Jun 4, 2024 •

edited

Loading

wendellpiez commented Jun 5, 2024 •

edited

Loading

wendellpiez commented Jun 7, 2024 •

edited

Loading