as_file option (e.g. for saving SSH keys) for lookup module

[parkerfath]

Has anyone considered adding an as_file parameter, similar to the one in the CyberArk Conjur plugin ?

I have a use case to pull per-host SSH keys from a HashiCorp Vault, and the only way I can tell to pass an SSH key to Ansible is via a filename. So basically, what I'd like to do is:
ansible_ssh_private_key_file: "{​​{​​ lookup('hashi_vault', 'secret/data/system/abcd123:ssh_priv', as_file=True) }​​}​​"

I would set this as a host variable, so that I could pull per-host SSH private keys from HashiCorp Vault.

Of course, it would be easier if I could use one private key across all our hosts, but our corporate policy/pen testers have required that we use separate keys per host to limit the blast radius of losing any single SSH key.

EDIT: for what it's worth, here's the PR for the change in the Conjur plugin: cyberark/ansible-conjur-collection#51

[briantist]

Hi @parkerfath , thanks for opening this!

I don't think I'd be interested in adding this option directly to the lookup plugins, for a few reasons:

• Returning the file name fundamentally changes the output of the plugin when the option is given, and that makes it harder to reason about
• Generally speaking, we don't want to be writing state or modifying anything in lookups; there are exceptions but it should be avoided
• Ansible works in parallel, so it's possible for a task to be executing simultaneously for multiple hosts at the same time, all trying to write to the same file, but potentially with different values
• Templates are processed lazily, so every instance of the lookup will be re-run, re-executing it (this is already a concern sometimes) (see also https://docs.ansible.com/ansible/latest/collections/community/hashi_vault/docsite/lookup_guide.html)
• For the hashi_vault lookup specifically, we're not removing it, but we are encouraging the use of alternatives, so we wouldn't add any new functionality to that plugin, see also https://docs.ansible.com/ansible/latest/collections/community/hashi_vault/docsite/migration_hashi_vault_lookup.html
• The primary reason though, is that we can achieve this without adding new functionality.

---

It might be helpful to see some examples:

All hosts use the same key

- name: Same key for each host
  hosts: all
  vars:
    ssh_key_lookup: "{{ lookup('community.hashi_vault.vault_kv2_get', 'system/abcd123') }}"
    # this could also go in inventory
    ansible_ssh_private_key_file: /root/key_path
  pre_tasks:
    - name: Write SSH key
      run_once: true
      become: true
      delegate_to: localhost
      ansible.builtin.copy:
        dest: '{{ ansible_ssh_private_key_file }}'
        content: '{{ ssh_key_lookup.secret.ssh_priv }}'
        mode: 0600
  tasks:
    - name: Regular task
      namespace.collection.module:

Each host uses a unique key

- name: Different key per host 
  hosts: all
  vars:
    ssh_key_vault_path: 'system/{{ inventory_hostname }}/abcd123'
    ssh_key_lookup: "{{ lookup('community.hashi_vault.vault_kv2_get', ssh_key_vault_path) }}"
    # this could also go in inventory
    ansible_ssh_private_key_file: '/root/key_file_{{ inventory_hostname }}'
  pre_tasks:
    - name: Write SSH key
      become: true
      delegate_to: localhost
      ansible.builtin.copy:
        dest: '{{ ansible_ssh_private_key_file }}'
        content: '{{ ssh_key_lookup.secret.ssh_priv }}'
        mode: 0600
  tasks:
    - name: Regular task
      namespace.collection.module:

In either case you can also replace the lookup with an additional task before the one that writes the file:

- name: Different key per host 
  hosts: all
  vars:
    ssh_key_vault_path: 'system/{{ inventory_hostname }}/abcd123'
    # this could also go in inventory
    ansible_ssh_private_key_file: '/root/key_file_{{ inventory_hostname }}'
  pre_tasks:
    - name: Lookup SSH key
      register: ssh_key_lookup
      delegate_to: localhost
      community.hashi_vault.vault_kv2_get:
        path: '{{ ssh_key_vault_path }}'

    - name: Write SSH key
      become: true
      delegate_to: localhost
      ansible.builtin.copy:
        dest: '{{ ansible_ssh_private_key_file }}'
        content: '{{ ssh_key_lookup.secret.ssh_priv }}'
        mode: 0600
  tasks:
    - name: Regular task
      namespace.collection.module:

---

The above examples (and your original example) don't show how auth is handled, which can always add some issues.

If you're willing to setup and configure a Vault agent on your ansible controller, you have some other interesting avenues. The agent can use templating to write out a secret to a file on your system and keep it refreshed, in the background, independent of ansible, and then you need only refer to the file it's keeping up to date.

Another option is to use the agent as your Vault host. With the agent using auto-auth, it's able to take HTTP requests that contain no authentication, and proxy them to your Vault server with the token it keeps alive in the background.

To do this, you can use auth_method=none in this collection's plugins and modules, and use the local address of the agent as your Vault URL.

[parkerfath]

Thanks for such a thorough and helpful answer! I think I have your example working for the case with different keys per host. I had to make a few adjustments for my use case.

1. I'm using Ansible AWX, so I can't sudo on the runner. I found that the "Write SSH Key" task still works OK with become: false, though.
2. I had to set gather_facts: no and do an explicit setup task later, since otherwise Ansible will try to connect to the host and gather facts before I have the key ready.
3. I also had to add a \n at the end of the content in the copy module and surround with double-quotes - content: "{{ ssh_key_lookup.secret.sshpriv }}\n"

I'm still working on adapting this into my existing playbooks, but it seems like we might have a winner!

[briantist]

That's great to hear @parkerfath ! Thanks for sharing the changes you made, they could be helpful for someone else. I don't use AWX so I sometimes have a blind spot for how things work differently than using Ansible alone.

I think I also just realized that run_once: true should be removed when using a different key per host 😬 I'll edit those.

I've changed the discussion type to Q&A, so you can mark an answer if you'd like to.

---

You could wrap the process in a role, and then call it either in pre_tasks:, or as the first role in roles:, or the first task in tasks: (depending on your playbook(s)), which might help to clean up the top of your playbooks and allow you to make changes in one place.

[briantist]

Also if you want to avoid having to turn off fact gathering for your main play, you can do this stuff in a play before the main one, and turn off gathering there only:

- name: Different key per host 
  hosts: all
  gather_facts: false
  vars:
    ssh_key_vault_path: 'system/{{ inventory_hostname }}/abcd123'
    # this could also go in inventory
    ansible_ssh_private_key_file: '/root/key_file_{{ inventory_hostname }}'
  pre_tasks:
    - name: Lookup SSH key
      register: ssh_key_lookup
      delegate_to: localhost
      community.hashi_vault.vault_kv2_get:
        path: '{{ ssh_key_vault_path }}'

    - name: Write SSH key
      become: true
      delegate_to: localhost
      ansible.builtin.copy:
        dest: '{{ ansible_ssh_private_key_file }}'
        content: '{{ ssh_key_lookup.secret.ssh_priv }}'
        mode: 0600

- name: Main play
  hosts: all
  tasks:
    - name: Regular task
      namespace.collection.module: