text() method merges text parts

[kvetoslavnovak]

text() method in JQuery keeps whitespaces between text parts. Cheerio unfortunatelly merges text parts from tags thogether givign you quite gibberish words.

See exampel https://api.jquery.com/text/ I tested it by myself and JQuery text() method keeps whitespaces between the text parts of the tags. This is very useful.

Cheerio on the other hand messes text parts alltogther without no spaces which is a pain.

E.g. from this

TITLE>
<TI>
<P>PART ONE</P>
</TI>
<STI>
<P>
<HT TYPE="BOLD">GENERAL PROVISIONS</HT>
</P>
</STI>
</TITLE>

JQuery text() gives you expected text PART ONE GENERAL PROVISIONS but Cheerio gives you PART ONEGENERAL PROVISIONS

Also standard HTML innerText property works the same.

[kvetoslavnovak]

OK. This seemt to do the trick:

cheerio(elem).html().replaceAll(/<\/?[a-zA-Z0-9=" ]*>/g, ' ').replace(/\s\s+/g, ' ').trim())

But better would if cheerio keep the whitespace which can ve trime manually as user needs to.

[apsquared]

Facing the same issue and hitting it via langchain CheerioWebLoad so I cant use the approach above.

[Properko]

I believe this is related #2841 and really unfortunate 😬
That regex can get heavy for huge htmls