This addresses both :

• some random failures we sometimes have when installing python-language-server fail
• caddy no longer being installed here since !821 (merged)