From cea91605ed2066e92c938a992fbebe080a853e9f Mon Sep 17 00:00:00 2001 From: Wendell Piez Date: Wed, 7 Aug 2024 09:44:43 -0400 Subject: [PATCH] Misc adjustments and updates --- tutorial/TUTORIAL-PAGE.xpl | 103 +++++++++--------- .../oscal-convert/oscal-convert_101_src.html | 24 +++- tutorial/source/setup/setup_101_src.html | 4 +- tutorial/source/setup/setup_102_src.html | 6 +- tutorial/source/unpack/unpack_102_src.html | 59 +++++----- tutorial/source/unpack/unpack_599_src.html | 22 ++-- tutorial/src/tutorial-authoring.css | 11 ++ 7 files changed, 137 insertions(+), 92 deletions(-) diff --git a/tutorial/TUTORIAL-PAGE.xpl b/tutorial/TUTORIAL-PAGE.xpl index db276d04..6af93b67 100644 --- a/tutorial/TUTORIAL-PAGE.xpl +++ b/tutorial/TUTORIAL-PAGE.xpl @@ -57,57 +57,60 @@ + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- - -
- - -
- - - - - - - - - + + + + + + + + +
+ + +
+ + +
+ + +
+ + + + + + + + diff --git a/tutorial/source/oscal-convert/oscal-convert_101_src.html b/tutorial/source/oscal-convert/oscal-convert_101_src.html index 883544ea..86c92ff5 100644 --- a/tutorial/source/oscal-convert/oscal-convert_101_src.html +++ b/tutorial/source/oscal-convert/oscal-convert_101_src.html @@ -26,8 +26,9 @@

Resources

Step one: convert some OSCAL XML into OSCAL JSON

-

Acquisition pipeline in the project folder collects some OSCAL onto the local system, where it can be - managed, easily inspected, controlled, and edited if necessary.

+

An acquisition pipeline in the project folder + collects some OSCAL onto the local system, where it can be managed, easily inspected, controlled, and edited + if necessary.

But sources and results from a conversion process do not always have to work with local resources. XProc sees the Internet itself - whatever protocols are supported by the processor (the file and http protocols are required for conformance) work as they do on the Worldwide Web. Of @@ -48,6 +49,25 @@

Step one: convert some OSCAL XML into OSCAL JSON

Step two: return trip

Two ways: separate pipeline; and single pipeline; also a 'switcher' pipeline?

+
+

What is this XSLT?

+

If your criticism of XProc so far is that it makes it look easy when it isn't, you have a point. Conversion + from XML to JSON isn't free, assuming it works at all. In this case, the heavy lifting is done by the XSLT + component - the Saxon engine invoked by the p:xslt step, applying logic defined in an XSLT + stylesheet (aka transformation) stored elsewhere. It happens that a converter for OSCAL data is available in + XSLT, so rather than having to confront this considerable problem ourselves, we drop that in.

+

In later units we will see how using the XProc steps described, rudimentary data manipulations can be done + using XProc by itself, without entailing the use of either XSLT or XQuery (another capability invoked with a + different step). At the same time, while pipelines are based on the idea of passing data through a series of + processes, there are many cases where logic is sufficiently complex that it becomes essential to maintain – + and test – that logic externally from the XProc. At what point it becomes more efficient to encapsulate + logic separately (whether by XSLT, XQuery or other means), depends very much on the case.

+

The p:xslt pipeline step in particular is so important for real-world uses of XProc that it is + introduced early, to show such black-box application. XProc also makes a fine environment for testing XSLT + developed or acquired to handle specific tasks, a topic covered in more depth later. Indeed XSLT and XQuery + being, like XProc itself, declarative languages, it makes sense to factor them out while maintaining easy + access and transparency for analysis and auditing purposes.

+

What could possibly go wrong?

When coping with errors, syntax errors are relatively easy. But anomalous inputs, especially invalid inputs, diff --git a/tutorial/source/setup/setup_101_src.html b/tutorial/source/setup/setup_101_src.html index 96d385ee..cee5c349 100644 --- a/tutorial/source/setup/setup_101_src.html +++ b/tutorial/source/setup/setup_101_src.html @@ -15,8 +15,8 @@

Goals

Prerequisites

-

You have Java installed with a JVM (Java Virtual Machine) available on the command line, version 8.0 and 11 - (and later).

+

You have Java installed with a JVM (Java Virtual Machine) available on the command line (a JRE or JDK), + version 8 (and later).

You have a live Internet connection and the capability to download and save resources (binaries and code libraries) for local use.

Tip: check your Java version from the console using java --version.

diff --git a/tutorial/source/setup/setup_102_src.html b/tutorial/source/setup/setup_102_src.html index be4b699d..47d9dfb8 100644 --- a/tutorial/source/setup/setup_102_src.html +++ b/tutorial/source/setup/setup_102_src.html @@ -9,7 +9,8 @@

102: Examining the setup

Goals

  • Look at some pipeline organization and syntax on the inside
    • -
  • Success and failure invoking XProc pipelines: an early chance to learn to die, fast and well
    • +
  • Success and failure invoking XProc pipelines: an early chance to “learn to die” gracefully (to use the + gamers' idiom).

Resources

@@ -18,7 +19,8 @@

102: Examining the setup

Prerequisites

Please complete the repository setup and smoke tests as described in the 101 lesson. In this lesson, we will run these pipelines with adjustments, or similar pipelines.

This discussion assumes basic knowledge of coding, the Internet (including retrieving resources via file and http protocols, and web-based technologies including HTML.

-

XML knowledge is also assumed, while at the same time we are interested to know where this gap needs to be filled.

+

XML knowledge is also assumed, while at the same time we are interested to know where this gap needs to be + filled.

You will also need a programmer's plain text editor, XML/XSLT editor or IDE (integrated development environment) for more interactive testing of the code.

diff --git a/tutorial/source/unpack/unpack_102_src.html b/tutorial/source/unpack/unpack_102_src.html index b5865aa5..738e73d0 100644 --- a/tutorial/source/unpack/unpack_102_src.html +++ b/tutorial/source/unpack/unpack_102_src.html @@ -13,11 +13,12 @@

102: XProc fundamentals

Resources

-

The same pipelines you ran in setup: Setup 101.

-

This tutorial's handmadeXProc dashboard with links

+

Take a quick look now:

+

This tutorial's handmade XProc dashboard with links

Also, the official XProc.org dashboard page

Also, check out XProc index materials produced in this repository: XProc docs

+

And the same pipelines you ran in setup: Setup 101.

Prerequisites

@@ -208,30 +209,33 @@

XProc embedded documentation

XProc step prologue and body

Keep in mind that every XProc pipeline is also, potentially and actually, a step. There are two things we need to know about steps - how to define them, and how to use them.

-

We begin with how to recognize and use steps, but we can't avoid how to define them - because an XProc +

We begin with how to recognize and use steps, but we can't avoid how to define them: because an XProc pipeline is also an XProc step, we can't use steps without ending up with a pipeline. We have only to look at the working pipeline we make with our steps, to see how a step is made.

As described in the XProc 3.0 specification, XProc step declarations can be divided into an initial set of elements for setup and configuration, followed by what the specification calls a subpipeline, which is typically a - sequence of steps to be executed – any steps available, which could be anything.

-

Fortunately, the list of elements that come before the steps is short: p:import, + sequence of steps to be executed – any steps available, which could be anything. Think of the subpipeline + as the working parts of the pipeline, while the rest is all about how it is set up.

+

The list of elements that come before the steps is short, which helps: p:import, p:import-functions, p:input, p:output, p:option or p:declare-step. Everything coming after is a step.

-

Within this set of elements XProc further distinguishes between the imports for steps and - functions, appearing first (elements p:import and p:import-functions), to be - followed by elements configuring the step: p:input, p:output, - p:option – elements together called the Within this set of elements (all preceding, none following the subpipeline) XProc further distinguishes + between the imports for steps and functions, appearing first (elements p:import and + p:import-functions), to be followed by elements configuring the step: + p:input, p:output, p:option – elements together called the prologue.

The prologue is used to define ports and options for the pipeline - the points of control for its interfaces. (Technically: runtime bindings, and parameter or option settings.) If only a single input is - needed, a single input port (named source) will be assumed.

+ needed, a single input port (named source) can be assumed, so prologues can be empty (and + invisible, or not there).

Following the prologue, a step may also have local step definitions (p:declare-step).

-

After imports, prologue and (optional) step declarations, the step sequence that follows is called the subpipeline. One other complication: for - purposes of exposition, we are pretending that p:variable (a variable declaration) is a - step, which is not how the XProc recommendation describes it, although it appears in the subpipeline with - and among its steps.

+

After imports, prologue and (optional) step declarations, the step sequence that follows comprises the subpipeline.

+

One other complication: for purposes of exposition, we have pretended that p:variable (a + variable declaration) and p:documentation (for out-of-band documentation) are steps, which + is not how the XProc recommendation describes them, although they appears in the subpipeline with and + among its steps.

In summary: any XProc pipeline, viewed as a step declaration, can have the following --

  • Pipeline name and type assignment (if needed), given as attributes at the top
    • @@ -250,7 +254,8 @@

    XProc step prologue and body

  • Variable declarations - p:variable
  • Finally, as noted above, p:documentation can appear anywhere in a pipeline, but it will - be ignored except when appearing inside p:inline – a detail to be covered later.
    • + be ignored except when appearing inside p:inline. What to do with these is a topic to be + covered later.

    NB: the pipelines run so far have XML comments demarcating the prologue from the steps

    @@ -271,7 +276,7 @@

    Atomic and compound steps

  • p:for-each - - produce subpipeline results for each member of a (provided or defined) sequence of inputs
    • + - produce subpipeline results for each member of a sequence of inputs (documents or nodes)
  • p:if @@ -279,11 +284,11 @@

    Atomic and compound steps

  • p:choose - - execute subpipeline based on a series of tests
    • + - execute a subpipeline conditionally (switch/case operator)
  • p:group - - group a subpipeline (step sequence) into a single step
    • + - group a subpipeline (step sequence) into a single logical step
  • p:viewport @@ -295,15 +300,16 @@

    Atomic and compound steps

    - execute a subpipeline, and deliver its results, or if it fails, a fallback subpipeline given in a p:catch
    • -

    Additionally to these elements, XProc subpipelines may contain variable declarations, to be covered - RSN.

    +

    Additionally to these elements, XProc subpipelines may contain variable declarations and documentation, + as noted below.

    Namespaces and extension steps

    We recognize steps because we either recognize them by name - for standard steps in the p: (XProc) namespace such as p:filter and p:add-attribute - or because we do not. Extension steps in XProc take the form of elements in an extension namespace. Generally speaking, that - is, any element not prefixed with p: is subject to evaluation as an extension.

    + is, any element not prefixed with p: is treated as out of scope for XProc and to be ignored, + while subject to evaluation as an extension.

    In an XProc pipeline (library or step declaration) one may also see a namespace c:. TODO - come back to

    @@ -375,13 +381,16 @@

    Exercise: Knowing what to look for

    section p:output) this pattern would be used only for very special purposes, if at all. Defining an output port, p:output, when given, shows not what will or should happen with process results (outputs), but rather exactly what kinds - of outputs are available, i.e. a pipelines defined outputs with the names and configurations including - which results they capture. + of outputs are available, with the names and configurations including which results they capture. If such + a port is not connected to a processing result but to a static resource, the static resource is what will + appear there.
  • Reverse this logic and you can see that a pipeline with no p:output must somewhere among its steps have one or more p:store steps, since these are the only ways results are made available externally to the pipeline runtime.
  • For security analysts: yes, this last point is consequential for purposes of auditing and assessing - vulnerabilities in and with XProc.
    • + vulnerabilities in and with XProc. A pipeline with no p:store has no effects on a file + system where it runs; a pipeline with no output ports exposes no results (for a calling process to + receive) - so to have neither is effectively to have no effects anywhere.
  • Both p:load and p:store are commonly provided with @message attributes, which are used to produce console messages (in a tool like Morgana) when steps in a subpipeline are executed.
    • diff --git a/tutorial/source/unpack/unpack_599_src.html b/tutorial/source/unpack/unpack_599_src.html index 051e45b3..ddc7bea3 100644 --- a/tutorial/source/unpack/unpack_599_src.html +++ b/tutorial/source/unpack/unpack_599_src.html @@ -30,18 +30,18 @@

    Prerequisites

    XProc schema

    +

    A schema for the XProc language, considered as core steps (compound and atomic) plus optional + community-defined steps, is referenced from the XProc Specification. This RNG schema is + very useful.

    +

    It may often be considered gratuitous to validate XProc files against a schema, when the application (for + us, Morgana)must in any case take responsibility for conformance issues, as it sees fit. The reference + schema becomes useful if we find or suspect bugs in Morgana, but until then it need not have any direct role + in any runtime operation.

    +

    Nevertheless the RNG schema still serves as a reference and an object for querying -- queries whose results + tell us about XProc. A pipeline for acquiring both RNG schema + and its RNC (compact syntax) variant is provided for interest and possible later use.

    -

    A schema for the XProc language, considered as core steps (compound and atomic) plus optional community-defined - steps, is referenced from the XProc - Specification. This RNG schema is very - useful.

    -

    It may often be considered gratuitous to validate XProc files against a schema, when the application (for us, - Morgana)must in any case take responsibility for conformance issues, as it sees fit. The reference schema - becomes useful if we find or suspect bugs in Morgana, but until then it need not have any direct role in any - runtime operation.

    -

    Nevertheless the RNG schema still serves as a reference and an object for querying -- queries whose results - tell us about XProc. A pipeline for acquiring both RNG schema and - its RNC (compact syntax) variant is provided for interest and possible later use.

    XPath

    Like other XDM-based technologies, XProc embeds and incorporates XPath, an expression language for XML. diff --git a/tutorial/src/tutorial-authoring.css b/tutorial/src/tutorial-authoring.css index 0e9bdf14..0f2a18bc 100644 --- a/tutorial/src/tutorial-authoring.css +++ b/tutorial/src/tutorial-authoring.css @@ -13,6 +13,17 @@ h4 { color: #00cc44 } h5 { color: #00ff55 } h6 { color: #33ff77 } +h1:before, +h2:before, +h3:before, +h4:before, +h5:before, +h6:before { + float: left; clear: both; font-family: sans-serif; outline: thin solid blue; + color: steelblue; margin-right: 0.2rem; padding: 0.3em; font-size: 40%; + vertical-align: text-top; +content: oxy_name() } + a:after { content: oxy_urlChooser(