Config-management tools tested on 10 Docker targets and 5,000 tiny files.
Practical benchmark of Ansible, pyinfra, Salt-SSH, and Salt master/minion under clean runs, idempotent reruns, 80ms ±20ms latency + 5% packet loss, stopped containers, Docker network disconnects, stale IPs, mid-run kills, and recovery.
Best default
Most beginner-friendly UX: readable YAML, huge ecosystem, clear failure/idempotence recap.
Fastest feedback loop
14.062s cold vs Ansible 41.351s, Salt/minion 156.658s, Salt-SSH 344.505s.
Expected outcomes
Every success/failure/recovery scenario matched its expectation.
Workload
10 Docker targets, 500 tiny files each, 5,000 managed files per cold run.
Recommendation
Use Ansible as the safe beginner/default choice. pyinfra won speed decisively and is attractive if Python-native infrastructure code is acceptable. Ansible was slower, but it remains the readable, popular, reliable default with the clearest failure and idempotence reporting. Salt-SSH was slow and fussier in this local agentless test; Salt master/minion is a different persistent-agent architecture, but was still slower here and especially sensitive to the netem setup.
Speed: same tiny-file workload
Bars are scaled per column; shorter is better. This fresh run compares all four modes on the same 10-target, 500-files-per-target workload.
| Tool | Cold 10×500 | Warm 10×500 | 80ms + 5% loss | Recovery after faults |
|---|---|---|---|---|
| pyinfra | 14.062s1.0× fastest | 10.882s1.0× fastest | 14.719s1.0× fastest | 14.665s1.0× fastest |
| Ansible | 41.351s2.9× fastest | 30.609s2.8× fastest | 32.452s2.2× fastest | 44.645s3.0× fastest |
| Salt master/minion | 156.658s11.1× fastest | 92.099s8.5× fastest | 361.215s24.5× fastest | 163.989s11.2× fastest |
| Salt-SSH | 344.505s24.5× fastest | 153.161s14.1× fastest | 147.990s10.1× fastest | 266.788s18.2× fastest |
Fault and recovery behavior
Expected-failure rows are good when they fail loudly instead of silently succeeding. Stale/refreshed IP is only applicable to agentless SSH modes.
| Tool | Stopped targets | Network disconnect | Stale IP | Refreshed IP | Mid-run kill |
|---|---|---|---|---|---|
| pyinfra | 18.081s✓ failed as expected | 20.651s✓ failed as expected | 13.694s✓ failed as expected | 7.172s✓ survived | 31.735s✓ failed as expected |
| Ansible | 29.214s✓ failed as expected | 37.386s✓ failed as expected | 13.862s✓ failed as expected | 20.908s✓ survived | 51.364s✓ failed as expected |
| Salt master/minion | 86.289s✓ failed as expected | 180.664s✓ failed as expected | n/a | n/a | 139.014s✓ failed as expected |
| Salt-SSH | 102.773s✓ failed as expected | 252.390s✓ failed as expected | 101.933s✓ failed as expected | 59.475s✓ survived | 221.186s✓ failed as expected |
Beginner friendliness
Ansible wins for docs/ecosystem and readable recaps. pyinfra is compact if you like Python. Salt SLS is readable, but Salt-SSH and master/minion ops/debugging were more involved.
Reliability signal
The harness injected stopped containers, network disconnects, stale inventories, packet loss, and mid-run kills. All 34 scenario outcomes matched expectation in the fresh run.
Caveat
This is local Docker/SSH, not production WAN scale. Treat times as practical direction, especially for ergonomics, fault reporting, and rough speed.
Evidence & artifacts
Raw logs stay on the host; this public page is the safe shareable overview.
Versions
Tool versions captured by the harness for this run.
- docker:
Docker version 29.1.3, build 29.1.3-0ubuntu3~24.04.2 - compose:
Docker Compose version 2.37.1+ds1-0ubuntu2~24.04.1 - ansible:
ansible [core 2.19.9] - pyinfra:
pyinfra: v3.8.0 - salt_ssh_host:
salt-ssh 3007.14 (Chlorine) - salt_image:
salt-master 3007.14 (Chlorine)