Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Lambda Docker Image - GPU Crashing #330

Open
melonie-miller opened this issue Jan 10, 2025 · 5 comments
Open

AWS Lambda Docker Image - GPU Crashing #330

melonie-miller opened this issue Jan 10, 2025 · 5 comments

Comments

@melonie-miller
Copy link

Node.js: 18 (public.ecr.aws/lambda/nodejs:18)
Puppeteer-Core: ^24.0.0
@Sparticuz/chromium: ^131.0.1

Due to deployment requirements, I have to create this as a Docker Image (i.e., cannot utilize lambda layers). To avoid issues with npm install in Docker hanging, I am locally running npm install into node_modules folder locally before build and copying over to the build image via Dockerfile. I am assigning the entire lambda folder chmod 777 permissions to avoid execution issues. The lambda function is behind a VPC and I am not too confident they are going to allow phoning out via a gateway for any reason.

I have tried manually installing glibc-locale-source and nss to address specific issues, but I am unable to resolve this.

Are there any steps that I am missing in terms of dependencies that should somehow be included? Or even better, a smoother process to get this configured appropriately under the current constraints?

image

image

@Sparticuz
Copy link
Owner

Sparticuz commented Jan 10, 2025

I've also seen that even with --disable-gpu, you still need the swiftshader files.

If you are using your own docker image, I would just install chromium in your docker compose.

@melonie-miller
Copy link
Author

In this case, assuming the docker build step/dockerfile? My main issue is compatibility. I have tried within this image to install chromium via yum and it does not find the package. Spent about an hour on that today... Settled for using package.json and npm install, then copy inside of the dockerfile. Is there a reason the swiftshader files are not part of the packaging?

I am considering running a separate image within the dockerfile to install chromium and dependencies and copying over to the main image if that would be cleaner.

@Sparticuz
Copy link
Owner

It probably depends on the base image you are using. I believe the amazon linux ones use dnf, not yum. But you should be able to find out with the distro docs for your base image. I would pretty much always recommend using a distro compiled chromium over this package.

If you are pulling from the releases it will have all the files included. You might want to try to use the built in args in this package. A lot of the serverless environments are very picky about the correct args.

@melonie-miller
Copy link
Author

I am on an AWS Lambda Node.js distro, which does not come with any chromium compilation. Best I can do is try to get the chromium from a linux distro and copy over, but not having very much luck with that option, either. Would it be an option to copy over into the tmp folder, similarly to a lambda layer, during the docker build process? And instead recover any missing files this way? Might be a long shot but running low on time to troubleshoot

@melonie-miller
Copy link
Author

PS - swiftshader.tar.br is included in the node_modules package, so I do not think this is the issue... Is there any way that puppeteer/chromium are trying to phone out for anything (even synchronizing versioning) which would cause it to fail?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants