Better specification for how nested lookups (GSUB type 5) are to be processed

[davelab6]

In #1 (comment) @PeterConstable wrote,

[OFF has an] incomplete specification (multiple details) for how nested lookups (e.g., GSUB type 5) are to be processed—all details that the spec should make clear in order to ensure interoperability.

I believe this is discussed in MicrosoftDocs/typography-issues#407

I hope this issue can track not only this being fixed in OFF spec text, but examples of un-interoperable nested lookups (GSUB type 5) already implemented being identified and fixed.

[behdad]

There's two aspects to nested lookups that need clarification:

1. The LookupFlags used. Is clear now. There was a MicrosoftDocs issue about it. The nested-lookup's lookup flags shall prevail,
2. The range of the buffer area available to the nested-to lookups. I don't believe any limitation should be encoded. There's one or two MicrosoftDocs issues about this already.

[dscorbett]

For reference, Behdad’s first point’s issue was MicrosoftDocs/typography-issues#407 and the second’s are MicrosoftDocs/typography-issues#105 and MicrosoftDocs/typography-issues#184.