Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Testing and repairs including JSON Schema generation enhancements #119

Merged
merged 5 commits into from
Aug 30, 2024
Merged

Testing and repairs including JSON Schema generation enhancements #119

merged 5 commits into from
Aug 30, 2024

Conversation

wendellpiez
Copy link
Collaborator

This PR, when ready, will contain:

  • XSpecs unit testing XML converter generator XSLT
  • XSpec and XProc testing for schema generation and other processes
  • InspectorXSLT developments
  • Refactoring submodules

And other improvements.

Apologies for the size of this PR!

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Have you squashed any non-relevant commits and commit messages? [instructions]
  • Do all automated CI/CD checks pass?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you included examples of how to use your new feature(s)?
  • Have you updated all website and readme documentation affected by the changes you made? Changes to the website can be made in the website/content directory of your branch.

@galtm @wendellpiez
produce-xml-converter.xsl: XSpec tests and minor XSLT improvements (#87)
a0816ac 
* XSpec tests and minor XSLT improvements

Co-authored-by: Wendell Piez <[email protected]>
@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 7, 2024 •
edited
Loading

XSpec Test Results

  5 files  +  3   93 suites  +53   0s ⏱️ ±0s
232 tests +127  215 ✅ +125  17 💤 +2  0 ❌ ±0 
243 runs  +129  225 ✅ +126  18 💤 +3  0 ❌ ±0 

Results for commit 4e31191. ± Comparison against base commit 04a4a4c.

♻️ This comment has been updated with latest results.

@wendellpiez
JSON Schema generation improvements (#108)
24e5007 
Refactoring submodules; XProc-based testing; XSpec testing of JSON Schema generation; improvements to JSON Schema generation - addressing #105 also #2007.
@wendellpiez
June 2024 Integration (#118)
74f3a11 
Integrating branches
Co-authored-by: A.J. Stein <[email protected]>
@wendellpiez wendellpiez requested a review from aj-stein-nist June 7, 2024 21:03
@wendellpiez
Copy link
Collaborator Author

#117 is addressed in the file src/converter-gen/Makefile.

In bash, invoke the XSpecs under testing in this folder with make spec-tests or less directly by make test.

It is also invoked under CI/CD, as shown in results.

Planned next is cleanup, build-out and documentation of all testing. Making this easier for devs can be on this punchlist.

@galtm
Copy link
Collaborator

galtm commented Jun 10, 2024

I drilled down into the XSpec Test Results (https://github.com/usnistgov/metaschema-xslt/pull/119/checks?check_run_id=25960758809) by clicking Raw output under There are 229 tests, see "Raw output" for the full list of tests. I wanted to see if the tests ran that #87 (comment) says didn't run in that earlier PR. I searched the raw results for a few of the x:expect labels and they were there. 👍

(The raw results appear to alphabetize the labels, btw.)

@wendellpiez
Copy link
Collaborator Author

Excellent, thanks @galtm. Before this branch is merged, more tests will be placed under the make logic, is the current plan....

wendellpiez and others added 2 commits June 11, 2024 12:24
@wendellpiez @aj-stein-nist
June 2024 in-progress merge (#120)
52c9f6d 
Includes updates to unit testing, Inspector XSLT, JSON Schema (#105) and others

Co-authored-by: A.J. Stein <[email protected]>
@wendellpiez @aj-stein-nist
More testing (#122)
4e31191 
Mostly testing support -
* Adding Makefile in compose phase to run XSpec - presently showing errors!
* Extending and reorganizing testing, including removing temporary artifacts from git
* Adding subpipeline mistakenly omitted from the last PR
* New Makefile for testing/tinydata
* Mending XSpecs testing Metaschema composition (regression protection); also a new tinydata sample.
* Update src/testing/tinydata/tiny-data/terms-of-art.xml (AJ Stein)
Co-authored-by: A.J. Stein <[email protected]>

---------

Co-authored-by: A.J. Stein <[email protected]>
@wendellpiez
Copy link
Collaborator Author

wendellpiez commented Aug 14, 2024 •
edited
Loading

This is ready to shake down. It includes lots of new tests, and refactors submodules.

@wendellpiez wendellpiez marked this pull request as ready for review August 14, 2024 14:17
@wendellpiez wendellpiez requested a review from a team as a code owner August 14, 2024 14:17
aj-stein-nist
aj-stein-nist approved these changes Aug 20, 2024
View reviewed changes
Copy link
Contributor

@aj-stein-nist aj-stein-nist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There some really good improvements here. I did some quick integration testing with OSCAL models and it adds some nice cosmetic improvements. I think there are some other changes that can be made for JSON Schema generation, but that can wait for later issues and PRs filed.

Thanks for this big pile of work.

src/compose/testing/1_collected/working_metaschema-COLLECTED.xml
@@ -75,15 +75,15 @@
</define-flag>
<METASCHEMA abstract="yes"
module="oscal-test-m1"
_base-uri="file:/C:/Users/wap1/Documents/usnistgov/metaschema/toolchains/xslt-M4/compose/testing/module_metaschema.xml">
_base-uri="...">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: thanks for cleaning these things up, I appreciate it.

src/converter-gen/testing/definition-maps/profile-reduce-map-output.xml
@@ -0,0 +1,7009 @@
<?xml version="1.0" encoding="UTF-8"?>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: as always, I will recommend deleting intermediate outputs if possible to make the noise on changes as tests evolve cleaner and not confuse novice devs in the repo, but I defer to you as the maintainer on this project. 😄

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @aj-stein-nist . If I'm interpreting this PR correctly, I introduced this file in #87. The purpose of the file is to provide markup that XSpec can use for unit testing of pieces of this pipeline. Can your concern about novice devs be addressed by code comments in the file or a README in the directory?

@wendellpiez wendellpiez merged commit b63b373 into develop Aug 30, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@galtm galtm galtm left review comments

@aj-stein-nist aj-stein-nist aj-stein-nist approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@wendellpiez @galtm @aj-stein-nist