Pourquoi le transfert d'agent ssh ne fonctionne pas ?

Sur mon propre ordinateur, qui fonctionne sous MacOSX, j'ai ceci dans ~/.ssh/config

Host *
ForwardAgent yes
Host b1
ForwardAgent yes

b1 est une machine virtuelle exécutant Ubuntu 12.04. Je m'y connecte comme ceci :

ssh pupeno@b1

et je me connecte sans qu'on me demande un mot de passe parce que j'ai déjà copié ma clé publique. En raison de la redirection, je devrais pouvoir me connecter à pupeno@b1 depuis b1 et cela devrait fonctionner, sans me demander de mot de passe, mais ce n'est pas le cas. Il me demande un mot de passe.

Qu'est-ce que je rate ?

Voici la sortie verbeuse du second ssh :

pupeno@b1:~$ ssh -v pupeno@b1
OpenSSH_5.9p1 Debian-5ubuntu1, OpenSSL 1.0.1 14 Mar 2012
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to b1 [127.0.1.1] port 22.
debug1: Connection established.
debug1: identity file /home/pupeno/.ssh/id_rsa type -1
debug1: identity file /home/pupeno/.ssh/id_rsa-cert type -1
debug1: identity file /home/pupeno/.ssh/id_dsa type -1
debug1: identity file /home/pupeno/.ssh/id_dsa-cert type -1
debug1: identity file /home/pupeno/.ssh/id_ecdsa type -1
debug1: identity file /home/pupeno/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.9p1 Debian-5ubuntu1
debug1: match: OpenSSH_5.9p1 Debian-5ubuntu1 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA 35:c0:7f:24:43:06:df:a0:bc:a7:34:4b:da:ff:66:eb
debug1: Host 'b1' is known and matches the ECDSA host key.
debug1: Found key in /home/pupeno/.ssh/known_hosts:1
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/pupeno/.ssh/id_rsa
debug1: Trying private key: /home/pupeno/.ssh/id_dsa
debug1: Trying private key: /home/pupeno/.ssh/id_ecdsa
debug1: Next authentication method: password
pupeno@b1's password: