Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible fp16 overflow issues using OmniSR model #3071

Open
pokepress opened this issue Jan 4, 2025 · 1 comment
Open

Possible fp16 overflow issues using OmniSR model #3071

pokepress opened this issue Jan 4, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@pokepress
Copy link

Information:

  • Chainner version: 0.24.1
  • OS: Windows 10

Description
I'm experiencing intermittent black frames when upscaling using an OmniSR model in fp16 mode. I first presented this to the neosr project:

neosr-project/neosr#96

Basically, every so often a frame in the output will be pure black. It seems to be specific to the content of the picture, and the frames that turn out black vary depending on the iteration of the model. After some testing, we were able to determine that it appears to be an fp16 precision issue since using fp32 causes the same frame to upscale correctly. Furthermore, the person behind neosr tried converting to an onnx fp16 model and the image upscaled there without issue, so we think this may be something specific to chaiNNer. You can test things out with these files:

https://www.mediafire.com/file/xxsv5yi6unr01bd/black_frame_omnisr.zip/file

Frame 1262 upscales correctly. 1263 produces a black image.

Logs
I looked at the logs, but it doesn't seem like anything in particular shows up there when this happens. It would be nice to log a warning for each black frame if possible, as the issue is intermittent enough that it's generally faster to re-upscale affected segments using fp32 instead of running the whole video through.

@pokepress pokepress added the bug Something isn't working label Jan 4, 2025
@FNsi
Copy link
Contributor

FNsi commented Jan 4, 2025

And I guess, using PyTorch backend is working normal....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants