Ansible: Reduce copy time when copying within localhost

[tambakhk]

copy module performing recursive copy from localhost to localhost is taking lot of time(more than 3 minutes). How can I reduce this time ?

  - name: copy folders
    copy:
      dest: "{{ tmp_repo_directory }}"
      src: "{{ item }}"
      force: true
    loop:
      - "{{ playbook_dir }}/outputs"
      - "{{ playbook_dir }}/inventory"

[bonnevil · Accepted solution]

So, I assume that this task is running in a play that only has localhost in its hosts specification.    I'm also assuming that tmp_repo_directory and playbook_directory each hold the name of one directory.

The documentation for the copy module mentions that it scales poorly once you're dealing with a large number of files.  I don't recall why that is, but I know I had a discussion with upstream developers about that once.  It might be that copy is shoving all the file content through the Ansible connection, which if true would add overhead, but I don't know whether or not that is true.  There might be ways to play with that (especially if the copy is going over an SSH connection instead of using local).

You could try to use the synchronize (ansible.posix.synchronize) module, which is a wrapper for rsync.  Read the Notes in the documentation for that module to understand how it works in an Ansible context, the details on how to specify src and dest, and do some careful testing first.  If I remember correctly (I might not), by default the source is on the control node and the dest is on the current host in the play, but you can adjust the source with delegate_to.  The remote_src option for the module also looks interesting.

[tambakhk]

Yes, synchorise option seems to reduce time by a good margin. Since, I have used hosts as the localhost, I have not used delegate option. 

- name: copy folders

  synchronize:

    dest: "{{ tmp_repo_directory }}"

    src: "{{ item }}"

  loop:

  - "{{ playbook_dir }}/outputs"

  - "{{ playbook_dir }}/inventory"