YAML document separator in indented triple-quote blocks gets misinterpreted

[jpetazzo]

Hi!

Here is what I do:

I put a YAML document separator (---) within an indented triple-quote:

- Example YAML with multiple docs in it:
  ```yaml
  key1: value1
  ---
  key2 value2
  ```

Here is what I'm expecting:

something looking like this

• Example YAML with multiple docs in it:

  key1: value1
  ---
  key2: value2

Here is what I see instead:

The --- disappears, the triple quote doesn't seem to be interpreted, and there is an horizontal line in the middle.

it looks like this

• Same thing without the extra #:
  ```yaml key1: value1

---

key2: value2 ```

Extra information:

Just in case GitHub formatting isn't super clear, here is a jsfiddle exhibiting the issue:

https://jsfiddle.net/Lhwucmkb/3/

I haven't dived in the code yet so no idea how gnarly this is going to be 😅

[tripu]

Here's a dirty hack: insert a zero-width space somewhere inside (or around) that --- to prevent the parser from interpreting it as a horizontal rule.

You can copy the char from here: https://qwerty.dev/whitespace/

In your fiddle that does seem to do the trick, and you no longer need to append #.

[jpetazzo]

Unfortunately, that doesn't work for my use-case; here is why:

My goal is to include YAML snippets in presentations (specifically, training materials) so that people can copy-paste them. If I put a zero-width space, it will be copy-pasted as well, and YAML parsers will fail on it, with very weird error messages. (Typically, they'll say that they couldn't find an expected colon, because the --- followed by a zero-width space then becomes a key instead of a document separator.)

I'll keep looking! :)

[tripu]

Unfortunately, that doesn't work for my use-case; here is why:
My goal is to include YAML snippets in presentations (specifically, training materials) so that people can copy-paste them. If I put a zero-width space, it will be copy-pasted as well, and YAML parsers will fail on it, with very weird error messages. (Typically, they'll say that they couldn't find an expected colon, because the --- followed by a zero-width space then becomes a key instead of a document separator.)
I'll keep looking! :)

Ah, that makes sense.

Sorry that I don't have the time now to help with this.

[lukaszpiotrluczak]

Hi @tripu

Would you mind if I try to prepare some solution for that problem?

[tripu]

Hi @tripu
Would you mind if I try to prepare some solution for that problem?

@lukaszpiotrluczak:

I'm just a contributor here. Of course you're welcome to submit a PR! 👍

Anyway, note that I'm working on #680, which involves upgrading marked. And that change alone seems to fix quite a number of oddities in MD processing in Remark.

I did a quick test here pasting the MD code in your JSFiddle, and it looks fine. So, perhaps you prefer just to wait until I finish with my changes…

[lukaszpiotrluczak]

@tripu

Ok, it looks like upgrading marked should fix this and maybe even add something I miss.

I will wait for you to complete and if the problem persist I will take a look on it.

Thanks!

[lukaszpiotrluczak]

Hi @tripu any news? :) How it is going with update?

[tripu]

@lukaszpiotrluczak, since you asked… I could use some help 😇

I have made several upgrades and changes on my own fork; see the diff here:

develop...tripu:remark:develop

Among others, I upgraded marked.

Everything seems to work fine after my changes, except for the few tests about highlighting. Specifically:

• The two tests about code line highlighting
• The four tests about code block span highlighting

I haven't understood yet what's the root of the issue.

Care to review?

/cc @gnab, in case you have some minutes to spare!