Add implementation of MultipartFormData streaming encoder

[andreiltd]

I did a quick comparison of encoded FormData between SM and Chrome using following JS code:

Simple FormData

async function readStream(stream) {
 const reader = stream.getReader();
 const chunks = [];

 while (true) {
   const { done, value } = await reader.read();
   if (done) {
     break;
   }
   chunks.push(value);
 }

 let totalLen = 0;
 for (const chunk of chunks) {
   totalLen += chunk.length;
 }

 const result = new Uint8Array(totalLen);

 let offset = 0;
 for (const chunk of chunks) {
   result.set(chunk, offset);
   offset += chunk.length;
 }

 return result;
}

async function main() {
 const form = new FormData();
 form.append('field1', 'value1');
 form.append('field2', 'value2');

 const file = new File(['Hello World!'], 'dummy.txt', { type: 'foo' });
 form.append('file1', file);

 const req = new Request('https:example.com', {
   method: 'POST',
   body: form,
 });

 const type = req.headers.get('Content-Type');
 const boundary = type.split('boundary=')[1];

 // assert(type.startsWith('multipart/form-data; boundary='), "Content-Type is multipart/form-data");
 const data = await readStream(req.body);
 const dec = new TextDecoder();
 const bodyStr = dec.decode(data);

 console.log(bodyStr);
}

main();

The results are:

• Chrome:

------WebKitFormBoundaryhpShnP1JqrBTVTnC
Content-Disposition: form-data; name="field1"

value1
------WebKitFormBoundaryhpShnP1JqrBTVTnC
Content-Disposition: form-data; name="field2"

value2
------WebKitFormBoundaryhpShnP1JqrBTVTnC
Content-Disposition: form-data; name="file1"; filename="dummy.txt"
Content-Type: foo

Hello World!
------WebKitFormBoundaryhpShnP1JqrBTVTnC--

• SM:

----BoundaryjXo5N4HEAXWcKrw7
Content-Disposition: form-data; name="field1"

value1
----BoundaryjXo5N4HEAXWcKrw7
Content-Disposition: form-data; name="field2"

value2
----BoundaryjXo5N4HEAXWcKrw7
Content-Disposition: form-data; name="file1"; filename="dummy.txt"
Content-Type: foo

Hello World!
----BoundaryjXo5N4HEAXWcKrw7--

I also did some manual fuzzing to test the streaming logic by varying the buffer sizes the encoder writes into. Specifically, I tested these buffer sizes in bytes: 1, 2, 4, 8, 32, 1024, and 8192 by changing the size in the implementation. Though, having some unit test framework would be nice.

Relevant RFC: https://www.rfc-editor.org/rfc/rfc2046#section-5.1.1

[andreiltd]

It looks like the CI failure is unrelated to this PR (I've observed the same failure on main branch). This is a problematic test:

    const response = await fetch("https://http-me.glitch.me/meow?header=cat:é");
    strictEqual(response.headers.get('cat'), "é");

[tschneidereit]

@andreiltd, is this ready to review, or are you still expecting to make changes to it?

[andreiltd]

Yes, this is ready to review :)

[tschneidereit]

Either I'm missing something in looking at the diff, or there is some code missing—see the inline comment on form-data-encoder.h.

One thing that I can't fully evaluate, because it'll presumably be part of the implementation of encode_stream: IIUC, field names, will always be first have their newlines normalized to CRLF, and then have those escaped. It'd be good to fold those into one operation, if possible.

Otherwise, I left a few comments and suggestions. I'm a bit concerned about allocation and general failure handling, so it'd be good to go over those aspects in some detail.

[tschneidereit]

Suggested change

	if (i + 1 < src.size() && src[i + 1] == LF) {
	len += newline_len;
	i++;
	} else {
	len += newline_len;
	}
	len += newline_len;
	if (i + 1 < src.size() && src[i + 1] == LF) {
	i++;
	}

[tschneidereit]

it seems like this is only ever called with CRLF as the input for newline. Given that, does it make sense to remove the newline parameter and hardcode use of CRLF instead? Chances are LLVM emits the same code anyway, but it'd be better not to have to rely on that.

[tschneidereit]

Since this isn't just for newlines, it'd be good to give it a different name. Perhaps escape_name, given that this applies escaping of a set of characters in names?

[tschneidereit]

The implementation of these seems to be missing? (And in the case of query_length I also can't find any uses.)

[tschneidereit]

I think this needs a null check?

[tschneidereit]

The filename check needs to happen right after the call to escape_newlines: if that operation errors, we don't want to run any more fallible code.

[tschneidereit]

Does this need a null check?

[tschneidereit]

IIUC, this method is deliberately infallible? If so, could you document that fact?

[tschneidereit]

The return value isn't ever used, it seems. Maybe remove it?

[tschneidereit]

Is this an infallible operation? It seems like it should need to allocate memory, so I'm not sure I understand how that works. And ideally, if it does allocate, we should make that fallible—or at least assert that it didn't fail and abort execution.