[Feature] JPEG XL HDR support

[Qhilm]

The feature

I export my developed (read: tuned/modified) DSLR pictures in JPEG XL HDR P3 from Adobe Lightroom and they appear very dark in Immich, both when using Firefox on macOS and Chrome on macOS. I have attached two samples:

• JPEG_XL_HDR.jxl <= this appears very dark
• JPEG_XL_SDR.jxl <= this appears fine.

Github does not support jxl files, hence I put both in a zip file.

Disclaimer 1: I know "does google photos have it?" is a criteria for Immich Feature Requests and JPEG XL support (let alone JPEG XL HDR P3) is not yet available on google photos: Google has a heated and complicated relation to JPEG XL, but it is being adopted by many big players such as Apple, Adobe, etc., hence I am confident it will be added in time to google photos as well.

Disclaimer 2: this is not a very "backup" centric feature and I understand Immich main purpose is backing up mobile photos. I do wish Apple and Google default camera apps would start to use JPEG XL HDR tomorrow, as it's awesome and it's long overdue to have large support for HDR in photos (not tone mapping) – I never understood why video had it for so long while photos where lagging. I guess there's less money to be made with photos ;)

JPEG XL HDR Test.zip

Platform

• Server
• Web
• Mobile

[mertalev]

Thanks for providing example images! HDR images appearing darker is interesting. I can take a look at it later to see if there's anything we can do there.

As for general JPEG XL support, the question isn't so much about whether Google Photos has it or if it's backup-related (immich is more than a photo backup solution at this point), but client support. Chrome (infamously) doesn't support it and I don't think the Flutter mobile app does either, although the latter can be remedied.

I've been laying the groundwork for allowing generated images to be in other formats besides JPEG and WebP. The primary motivation has been for HDR AVIF, but it might be viable to add JPEG XL as a target too if we can get it working on the mobile app.

[Starbix]

I'm not sure if this should be in issue instead, if so, I'll post it there as well.
I also use Adobe Lightroom to develop (in my case convert from negative) my pictures and then export them as AVIF HDR (P3). I use AVIF because my MacBook doesn't support the newest macOS and thus doesn't support JPEG XL natively (I really hope I can switch soon though).

These AVIFs look great natively, in Safari, and in the Photos app. After uploading them to Immich, the conversion to thumbnails renders the images very flat and with a wrong black point. Exporting the AVIF to JPG using macOS's Preview looks good.

I've attached a few files to showcase the differences. The AVIF file is the original file as exported by Lightroom, the preview file is the AVIF exported to JPG by Preview, and immich-thumbnail is directly downloaded from Immich. Note that the original color profile is Display P3; SMPTE ST 2084 PQ, while the the color profile for the thumbnail is sP3C (which I've never heard of).

Using ffmpeg to convert the AVIF to JPG without any options yields a very similar result to Immich's thumbnail. Using the options -vf "zscale=transfer=linear,tonemap=hable,zscale=transfer=bt709" yields a better result. Not as good as the export done by Preview though (definitely partly because of the preservation of the color space).
avif.zip

A correct color conversion for HDR pictures in AVIF (or JPEG XL) is crucial as they aren't simple SDR pictures with gain maps.

[AngelaDMerkel]

A small update, but Google and Adobe seem to have plans to unify their gain map strategy per this talk by Eric Chan who is the lead developer on all of Adobe's HDR features. Apple is currently working with this ISO for their HDR, but it seems in the future they might try to homogenise their format with Adobe and Google.

EDIT: apparently Adobe software can (or will be able to) produce AVIFs/JXL with gain maps, but it's still experimental and not exposed by default:

In the talk they discuss how some unique box code referred to as hrgm could be added to JXL, but that doesn't seem to be the standard way that the devs expected HDR to be included in JXL images. Devs commenting on an imagemagick issue have said they plan to include support for AVIF and HEIC in libultrahdr so there is honestly probably nothing that needs to be done on the part of vips for this to work considering magick will get it virtually out of the box. There is an open PR to include the new support.

I asked one of the devs for JXL Jon Sneyers if he can provide some additional context and if he responds I'll update.

[AngelaDMerkel]

Update: it appears that JXL uses this draft spec to encode HDR base images without the use of any gain map. JXL also supports gain maps, but my understanding is that HDR is handled in JXL primarily by encoding this way.

[jcupitt]

Ah nice. That's a much more sensible standard than those dumb gain maps!

[AngelaDMerkel]

I agree, and my experience is that rendering is much better using this technique than gain maps.

[Starbix]

As mentioned here, Photomator is able to convert AVIF (with only NCLX) to AVIF (with ICC) so that libvips can create nice looking preview JPGs. Not really practical for my workflow (especially since Photomator isn't free), but probably quite useful for testing.

[Qhilm]

FYI: it appears that iPhones will be able to shoot in JPEG XL with iOS 18 (although it's just a rumor at this stage and it probably will not be the default).

I really hope this tips the scale and we finally move toward a "new jpeg" (royality free, omnipresent, omnicompatible, backward compatible with jpeg, HDR, transparency, and much more.). I really hope the support in iPhones will get google to add jpeg xl support in Chrome, then I would say the odds would be good for jpeg xl to become the "new jpeg".

Long story short: I really think this feature would make sense in Immich =)

[Qhilm]

let's see if cameras will actually save the jpeg xl in progressive encoding format or not (it's optional). If not, there's no storage benefit, Immich would have to reencode the jpeg xl file in progressive for the thumbnail.

[AngelaDMerkel]

@Qhilm I can't say what others will do, but for now Adobe is not producing progressive encodes when exporting from Lightroom or Camera RAW. jxlinfo doesn't report progressive and actually isn't even certain it's lossless, reporting (possibly) lossless

[Starbix]

The first feature Immich (well rather libvips with some FUTO money hopefully) should look into is that HDR JPEG XL (from Adobe or possibly iPhone) is displayed correctly in the generated previews. This is definitely more relevant, assuming iPhone will shoot in JPEG XL, as the Immich library will look very dull and display wrong colors.

[AngelaDMerkel]

@Starbix vips has some grant money now and looks like they are going to invest that in better JXL HDR support. Having looked at their issues, there will probably be some moves to add support for converting JXL to Ultra HDR which chrome android etc display "perfectly well" and this is probably good enough for now

[Starbix]

Oh I didn't notice they got some EU grant money. That's great news! I hope that work also translates well to other HDR formats.

[Qhilm]

I think upgrading to libjxl-dev and libjxl to 0.9 has partially fixed the issue.

The JXL HDR pictures now do not appear dark in the browser, but instead they are quite dull, compared to a JPEG HDR picture.

My previews are set to webp, both pictures were generated with Lightroom Classic and I'm on Immich v1.116.2.

Since webp is 8bit, I don't think the issue is related to HDR rendition in a particular browser. I tested with Safari as well to be on the safe side, which suports JPEG XL HDR (tested here).

I'm interested to hear if others are able to replicate this issue.

[Edit] Same effect in the iOS app: jxl hdr image appears significantly duller than the jpeg hdr one.

[mertalev]

It might be a similar case as this issue in that the gain map of the source image isn't taken into account.

[AngelaDMerkel]

@mertalev Vips is currently in the processes of fulfilling a grant which includes this support. As far as I know, it's not a bug, but just hasn't been implemented. The next release ought to include this support and maybe sharp down the line

[jcupitt]

It looks like it'll be really easy -- libjxl has added some new API for profile loading that'll make an approximate ICC profile for you from any colour metadata on the JXL image. Maybe we just have to use this new call.

[Starbix]

@jcupitt Would it be possible to also abuse this API for approximate ICC profiles for AVIF (with the whole nclx shenanigans)?

[jcupitt]

@Starbix unfortunately it's not exposed, and (I think?) the way jxl encodes its colour metadata isn't compatible with the nclx craziness (I think? it's very unclear).

My understanding right now is that we'd need some more work on the AVIF loader to get HDR there, but I could well be wrong.

[jcupitt]

Hello everyone, I've been experimenting with the sample HDR image linked above:

libvips/libvips#4174 (comment)

With this command:

$ vips icc_import JPEG_XL_HDR.jxl x.jpg --intent perceptual

I get:

Does that look about right? I'm not sure I have the white point in the right place.

I also don't see any specular highlights over 255. Is it bright enough?

[AngelaDMerkel]

I don't see anything over 255 after doing as thorough an inspection as I can and my results look very similar to yours. I don't know if that is what you are targeting?

[Qhilm]

It looks more yellowish than the jxl sdr for me, but not subdued or darker:

This might have a completely unrelated reason.

[jcupitt]

Thanks! Yes, it looks like my whitepoint isn't quite correct (maybe a D65 / D50 mixup?) and I need to find a sample HDR JXL with larger dynamic range for testing. I'll keep tweaking.

[AngelaDMerkel]

I will send link some of HDR JXL samples with wider range. Do you require sRGB or other colour spaces?

[jcupitt]

The libjxl devs suggested:

https://sneyers.info/hdrtest/

So I think I'm OK. Thanks!

[Qhilm]

Did something change in 117.0? I can barely see the difference between JXL HDR and JPEG HDR now. The white point seems a little different somehow, but that's it.

[jcupitt]

The new libvips JXL load/save operations currently need v0.11+, fwiw.

[Starbix]

I can prepare a PR to build libjxl 0.11.0 for the base image.
EDIT: immich-app/base-images#125, will need some testing.

[jcupitt]

I've run out of time now, I'll look at this again in November.

[Qhilm]

Hmm, ok, my mistake, my picture just didn't have enough dynamic range in it. JXL HDR files are still quite subdued compared to how they look on my computer natively.

[Qhilm]

btw, ICC support for JXL files in libvips requires v8.16 it appears (to be released in a couple of days).

[Austinzveare]

What's the current status of implementing JXL HDR support in Immich? I was waiting for normal JXL and HDR video to be supported before jumping in. At the moment I have my whole DSLR and iPhone RAW/ProRAW library exported as SDR P3 JXL's but would really love HDR support considering the native photos app in iOS 18 now fully supports these HDR photo formats.

[mertalev]

It requires using (and probably writing) a different photo library that can display HDR. There are no updates on that front, but rest assured that we're aware of the issue and will work on it.