Unable to run docker containers on Nx Agents

[jvandemo]

Scenario

Sometimes jobs require docker containers.

For example: integration tests may require a mock API, a mock database, etc.

Current behaviour

Docker does not work on Nx Agents.

Desired behaviour

Make it possible to run docker containers on Nx Agents.

Learnings so far

The docker command is not available out-of-the-box on Nx Agents.

When installing docker via a custom launch template:

launch-templates:
  custom-linux-extra-large-js:
    resource-class: "docker_linux_amd64/extra_large"
    image: 'ubuntu22.04-node20.11-v5'
    init-steps:
      - name: Checkout
        uses: 'nrwl/nx-cloud-workflows/v3.5/workflow-steps/checkout/main.yaml'
      - name: Restore Node Modules Cache
        uses: 'nrwl/nx-cloud-workflows/v3.5/workflow-steps/cache/main.yaml'
        env:
          KEY: 'package-lock.json|yarn.lock|pnpm-lock.yaml'
          PATHS: 'node_modules'
          BASE_BRANCH: 'main'
      - name: Restore Browser Binary Cache
        uses: 'nrwl/nx-cloud-workflows/v3.5/workflow-steps/cache/main.yaml'
        env:
          KEY: 'package-lock.json|yarn.lock|pnpm-lock.yaml|"browsers"'
          PATHS: |
            '../.cache/Cypress'
            '../.cache/ms-playwright'
          BASE_BRANCH: 'main'
      - name: Install Node Modules
        uses: 'nrwl/nx-cloud-workflows/v3.5/workflow-steps/install-node-modules/main.yaml'
      - name: Install Browsers (if needed)
        uses: 'nrwl/nx-cloud-workflows/v3.5/workflow-steps/install-browsers/main.yaml'
      - name: Configure Node.js
        script: |
          export NODE_OPTIONS=--max_old_space_size=12288
          echo "NODE_OPTIONS: $NODE_OPTIONS"
      - name: Set up docker apt repository
        script: |
          # Add Docker's official GPG key:
          sudo apt-get update
          sudo apt-get install ca-certificates curl
          sudo install -m 0755 -d /etc/apt/keyrings
          sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
          sudo chmod a+r /etc/apt/keyrings/docker.asc

          # Add the repository to Apt sources:
          echo \
          "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
          $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
          sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
          sudo apt-get update
      - name: Install docker
        script: |
          sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
          sudo usermod -a -G docker workflows
      - name: Start docker
          sudo service docker start || true
          sudo service docker status || true
      - name: Verify docker
        script: |
          sudo docker info || true
          #
          #
          # The following error appears:
          # docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
          # 186  See 'docker run --help'.
          #
          #
      - name: Start MongoDB
        script: |
          #
          # Start MongoDB in a docker container
          chmod +x ./.nx/agent-scripts/start-mongodb.sh
          sudo ./.nx/agent-scripts/start-mongodb.sh
          #
          # Fails because docker is not running

docker complains about not being able to access /var/run/docker.sock:

****************************************
docker info
****************************************
Client: Docker Engine - Community
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Version:    26.1.2
errors pretty printing info
Context:    default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version:  v0.14.0
Path:     /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version:  v2.27.0
Path:     /usr/libexec/docker/cli-plugins/docker-compose

Is it possible to run docker containers on Nx Agents?

Thanks in advance! 🙏 🙇

[sdacunha]

Trying to build a docker image as one of the steps in my workflow and running into the same issue - seems to be the only blocker I have to adopting NX Agents

[rarmatei]

Hi everyone! Docker support on Nx Agents is only available with Nx Enterprise for now. It's definitely something we don't want to restrict forever, and we're actually investigating opening up support for it for everyone, but I can't provide a clear timeline at the moment of when/how that will happen.

[sdacunha]

Hi everyone! Docker support on Nx Agents is only available with Nx Enterprise for now. It's definitely something we don't want to restrict forever, and we're actually investigating opening up support for it for everyone, but I can't provide a clear timeline at the moment of when/how that will happen.

Appreciate the update - thanks! Have our configuration ready to go once this is resolved

[jvandemo]

Thank you so much for your follow-up, @rarmatei, much appreciated! 🙏 🙇

Can't wait until Docker is officially supported. 🤞🍀 🎉

[getlarge]

Thanks, @jvandemo, for creating this issue. I am glad I read it before trying to do the same :)

[roldengarm]

It would also help with installing docker, see #67 @rarmatei

[DannyMardini]

Hi everyone! Docker support on Nx Agents is only available with Nx Enterprise for now. It's definitely something we don't want to restrict forever, and we're actually investigating opening up support for it for everyone, but I can't provide a clear timeline at the moment of when/how that will happen.

Thanks for the reply @rarmatei, to be honest this is quite disappointing as using containers for testing is not an uncommon use case. This blocks me from being able to use Nx for a project and I'd really like to use Nx.

[cofenster-tiago]

Would kaniko work?