Je peux télécharger des petits fichiers assez bien en utilisant les mêmes commandes, cependant chaque fois que j'essaie de télécharger un fichier de 5 Go sur S3 alors le script s3 échoue :
[backup]$ s3put --bucket john-murdoch-bucket --reduced --debug 2 --callback 100 data.tgz
/usr/bin/s3put:45: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
err.message[len('No module named '):] + \
send: 'GET /?&max-keys=0 HTTP/1.1\r\nHost: john-murdoch-bucket.s3.amazonaws.com\r\nAccept-Encoding: identity\r\nDate: Tue, 11 Jun 2013 07:00:45 GMT\r\nContent-Length: 0\r\nAuthorization: AWS =\r\nUser-Agent: Boto/2.9.4 (linux2)\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: x-amz-id-2:
header: x-amz-request-id: 1E11AD8B3C93E6B5
header: Date: Tue, 11 Jun 2013 07:00:46 GMT
header: Content-Type: application/xml
header: Transfer-Encoding: chunked
header: Server: AmazonS3
Copying /backup/data.tgz to john-murdoch-bucket/backup/data.tgz
send: 'PUT /backup/data.tgz HTTP/1.1\r\nHost: john-murdoch-bucket.s3.amazonaws.com\r\nAccept-Encoding: identity\r\nContent-Length: 5522978068\r\nExpect: 100-Continue\r\nx-amz-storage-class: REDUCED_REDUNDANCY\r\nDate: Tue, 11 Jun 2013 07:02:15 GMT\r\nContent-MD5: idp0nQ+9y5kshw0AjL9zMw==\r\nContent-Type: application/x-tar\r\nAuthorization: AWS =\r\nUser-Agent: Boto/2.9.4 (linux2)\r\n \r\n'
0 bytes transferred / 5522978068 bytes total
0 bytes transferred / 5522978068 bytes total
0 bytes transferred / 5522978068 bytes total
0 bytes transferred / 5522978068 bytes total
0 bytes transferred / 5522978068 bytes total
0 bytes transferred / 5522978068 bytes total
0 bytes transferred / 5522978068 bytes total
Traceback (most recent call last):
File "/usr/bin/s3put", line 374, in
main()
File "/usr/bin/s3put", line 371, in main
headers=headers)
File "/usr/bin/s3put", line 216, in singlepart_upload
k.set_contents_from_filename(fullpath, *kargs, **kwargs)
File "/usr/lib/python2.6/site-packages/boto/s3/key.py", line 1241, in set_contents_from_filename
encrypt_key=encrypt_key)
File "/usr/lib/python2.6/site-packages/boto/s3/key.py", line 1172, in set_contents_from_file
chunked_transfer=chunked_transfer, size=size)
File "/usr/lib/python2.6/site-packages/boto/s3/key.py", line 710, in send_file
chunked_transfer=chunked_transfer, size=size)
File "/usr/lib/python2.6/site-packages/boto/s3/key.py", line 882, in _send_file_internal
query_args=query_args)
File "/usr/lib/python2.6/site-packages/boto/s3/connection.py", line 544, in make_request
override_num_retries=override_num_retries)
File "/usr/lib/python2.6/site-packages/boto/connection.py", line 939, in make_request
return self._mexe(http_request, sender, override_num_retries)
File "/usr/lib/python2.6/site-packages/boto/connection.py", line 901, in _mexe
raise e
socket.error: [Errno 104] Connection reset by peer
Est-ce que je manque d'autres outils ou options en ligne de commande pour s3put qui me permettraient de faire plusieurs tentatives, ou de télécharger en plusieurs morceaux, ou ainsi de suite?
P.S. Je viens de faire "sudo yum upgrade python-boto" mais cela n'a rien changé.
Mise à jour - essayé s3cmd 1.0.0 :
[ec2-user@ip-10-212-234-60 backup]$ s3cmd put data.tgz s3://john-murdoch-bucket/data.tgz
data.tgz -> s3://john-murdoch-bucket/data.tgz [1 of 1]
36864 of 5522978068 0% in 1s 20.31 kB/s failed
WARNING: Upload failed: /data.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.00)
WARNING: Waiting 3 sec...
data.tgz -> s3://john-murdoch-bucket/data.tgz [1 of 1]
233472 of 5522978068 0% in 1s 144.89 kB/s failed
WARNING: Upload failed: /data.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.01)
WARNING: Waiting 6 sec...
data.tgz -> s3://john-murdoch-bucket/data.tgz [1 of 1]
45056 of 5522978068 0% in 1s 27.88 kB/s failed
WARNING: Upload failed: /data.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.05)
WARNING: Waiting 9 sec...
data.tgz -> s3://john-murdoch-bucket/data.tgz [1 of 1]
36864 of 5522978068 0% in 0s 45.52 kB/s failed
WARNING: Upload failed: /data.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=0.25)
WARNING: Waiting 12 sec...
data.tgz -> s3://john-murdoch-bucket/data.tgz [1 of 1]
20480 of 5522978068 0% in 1s 19.90 kB/s failed
WARNING: Upload failed: /data.tgz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=1.25)
WARNING: Waiting 15 sec...
data.tgz -> s3://john-murdoch-bucket/data.tgz [1 of 1]
12288 of 5522978068 0% in 2s 4.63 kB/s failed
ERROR: Upload of 'data.tgz' failed too many times. Skipping that file.