Elasticsearch Curator ne parvient pas à se connecter au service AWS ElasticSearch

À partir d'un serveur sur AWS, j'envoie des données dans le service AWS ES. Ces données s'affichent correctement sur Kibana. Je suis également en mesure d'effectuer un nettoyage via des commandes cURL à partir de cette machine. Cependant, aucune de mes machines n'est en mesure d'exécuter ElasticSearch Curator avec succès et de se connecter à AWS ES.

...jects/elasticsearch-curator curator --host HOSTNAME.us-east-1.es.amazonaws.com delete indices --older-than 12 --time-unit days --timestring %Y.%m.%d
2016-12-18 16:54:08,631 INFO      Job starting: delete indices
2016-12-18 16:58:10,143 ERROR     Connection failure.

Ma politique d'accès m'autorise à accéder à cette machine :

~ curl HOSTNAME.us-east-1.es.amazonaws.com
{
  "name" : "Exodus",
  "cluster_name" : "########",
  "version" : {
    "number" : "2.3.2",
    "build_hash" : "##########",
    "build_timestamp" : "2016-11-14T15:59:50Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

et exécuter des commandes :

~ curl -XDELETE 'HOSTNAME.us-east-1.es.amazonaws.com/2016-11*'
{"acknowledged":true}%

J'utilise la version de Le conservateur de l'ES a déclaré travailler avec l'AWS ES :

...jects/elasticsearch-curator pip list | grep curator
elasticsearch-curator (3.5.1)

Conformément à la réponse (1), j'ai ajouté un paramètre de port : "--port 80" la connexion échoue toujours, mais beaucoup plus rapidement :

curator --host HOSTNAME --port 80 delete indices --older-than 12 --time-unit days --timestring %Y.%m.%d
2016-12-19 08:28:18,309 INFO      Job starting: delete indices
2016-12-19 08:28:18,324 ERROR     Connection failure.

et voici la sortie en mode débogage :

2016-12-19 11:25:59,247 DEBUG         curator.api.filter         get_date_regex:157  regex = \l\o\g\s\t\a\s\h\-\d{4}\.\d{2}\.\d{2}
2016-12-19 11:25:59,247 DEBUG          curator.cli.utils        filter_callback:195  REGEX = (?P<date>\l\o\g\s\t\a\s\h\-\d{4}\.\d{2}\.\d{2})
2016-12-19 11:25:59,247 DEBUG          curator.cli.utils        filter_callback:198  Added filter: {'pattern': '(?P<date>\\l\\o\\g\\s\\t\\a\\s\\h\\-\\d{4}\\.\\d{2}\\.\\d{2})', 'value': 12, 'groupname': 'date', 'time_unit': 'days', 'timestring': u'logstash-%Y.%m.%d', 'method': 'older_than'}
2016-12-19 11:25:59,247 DEBUG          curator.cli.utils        filter_callback:199  New list of filters: [{'pattern': '(?P<date>\\l\\o\\g\\s\\t\\a\\s\\h\\-\\d{4}\\.\\d{2}\\.\\d{2})', 'value': 12, 'groupname': 'date', 'time_unit': 'days', 'timestring': u'logstash-%Y.%m.%d', 'method': 'older_than'}]
2016-12-19 11:25:59,248 INFO      curator.cli.index_selection                indices:62   Job starting: delete indices
2016-12-19 11:25:59,248 DEBUG     curator.cli.index_selection                indices:65   Params: {'url_prefix': u'', 'http_auth': None, 'dry_run': False, 'certificate': None, 'loglevel': u'INFO', 'logformat': u'default', 'client_cert': None, 'host': u'HOSTNAME', 'quiet': False, 'timeout': 30, 'debug': True, 'use_ssl': False, 'logfile': None, 'master_only': False, 'port': 80, 'ssl_no_validate': False, 'client_key': None}
2016-12-19 11:25:59,248 DEBUG          curator.cli.utils             get_client:114  kwargs = {'url_prefix': u'', 'http_auth': None, 'dry_run': False, 'certificate': None, 'loglevel': u'INFO', 'host': u'HOSTNAME', 'quiet': False, 'port': 80, 'logformat': u'default', 'timeout': 30, 'debug': True, 'use_ssl': False, 'logfile': None, 'master_only': False, 'client_cert': None, 'ssl_no_validate': False, 'client_key': None}
2016-12-19 11:25:59,248 DEBUG         urllib3.util.retry               from_int:191  Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)
2016-12-19 11:25:59,248 DEBUG     urllib3.connectionpool              _new_conn:212  Starting new HTTP connection (1): HOSTNAME
2016-12-19 11:25:59,584 DEBUG     urllib3.connectionpool          _make_request:400  http://HOSTNAME:80 "GET / HTTP/1.1" 200 None
2016-12-19 11:25:59,586 INFO               elasticsearch    log_request_success:66   GET http://HOSTNAME:80/ [status:200 request:0.338s]
2016-12-19 11:25:59,586 DEBUG              elasticsearch    log_request_success:68   > None
2016-12-19 11:25:59,587 DEBUG              elasticsearch    log_request_success:69   <
<!DOCTYPE html>
<html lang="en" charset="utf-8">
<head>
<title>Sign In</title>
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
<style>
body {
         font-family: "Helvetica Neue",Helvetica,Arial,sans-serif;
font-size: 14px;
line-height: 1.42857143;
color: #333;
background: #f0f0f0;
}
.signin {
    display:block;
margin:20px auto;
max-width:400px;
background: #fff;
border:1px solid #ccc;
border-radius: 10px;
padding: 20px;
}
.center {
    text-align:center;
}
.btn {
    color: #fff;
        background-color: #428bca;
border: 1px solid #357ebd;
            -webkit-border-radius: 4;
-moz-border-radius: 4;
border-radius: 4px;
font-size: 14px;
padding: 6px 12px;
text-decoration: none;
cursor: pointer;
}

.btn:hover {
    background-color: #3071a9;
        border-color: #285e8e;
ext-decoration: none;
}
label {
    display: inline-block;
max-width: 100%;
margin-bottom: 5px;
font-weight: 700;
}
input {
    display: block;
width: 100%;
height: 34px;
padding: 6px 12px;
font-size: 14px;
line-height: 1.42857143;
color: #555;
background-color: #fff;
background-image: none;
border: 1px solid #ccc;
border-radius: 4px;
-webkit-box-shadow: inset 0 1px 1px rgba(0,0,0,.075);
box-shadow: inset 0 1px 1px rgba(0,0,0,.075);
-webkit-transition: border-color ease-in-out .15s,-webkit-box-shadow ease-in-out .15s;
-o-transition: border-color ease-in-out .15s,box-shadow ease-in-out .15s;
transition: border-color ease-in-out .15s,box-shadow ease-in-out .15s;
margin:0;
box-sizing: border-box;
}
footer {
    display:block;
font-size:10px;
color:#aaa;
text-align:center;
margin-bottom:10px;
}
footer a {
    display:inline-block;
height:25px;
line-height:25px;
color:#aaa;
text-decoration:underline;
}
footer a:hover {
    color:#aaa;
}
</style>
</head>
<body>
<div class="signin center">
<form method="GET" action="/oauth2/start">
<input type="hidden" name="rd" value="/">

<p>Authenticate using @crosswise.com</p>

<button type="submit" class="btn">Sign in with a Google Account</button><br/>
</form>
</div>

<div class="signin">
<form method="POST" action="/oauth2/sign_in">
<input type="hidden" name="rd" value="/">
<label for="username">Username:</label><input type="text" name="username" id="username" size="10"><br/>
<label for="password">Password:</label><input type="password" name="password" id="password" size="10"><br/>
<button type="submit" class="btn">Sign In</button>
</form>
</div>

<script>
if (window.location.hash) {
(function() {
var inputs = document.getElementsByName('rd');
for (var i = 0; i < inputs.length; i++) {
inputs[i].value += window.location.hash;
}
})();
}
</script>
<footer>

Secured with <a href="https://github.com/bitly/oauth2_proxy#oauth2_proxy">OAuth2 Proxy</a> version 2.2.0-alpha

</footer>
</body>
</html>

2016-12-19 11:25:59,587 ERROR          curator.cli.utils             get_client:140  Connection failure.

J'ai donc ajouté quelques lignes de diagnostic à l'endroit approprié dans utils.py :

try:
    client = elasticsearch.Elasticsearch(**kwargs)
    check_version(client)
    # Verify the version is acceptable.
    logger.debug('version is acceptable')
    check_master(client, master_only=master_only)
    # Verify "master_only" status, if applicable
    logger.debug('Verify "master_only" status')
    return client
except Exception as e:
    import traceback
    logger.error('Connection failure.')
    logger.error(e)
    traceback.print_exc()
    sy.exit(1)

et nous obtenons :

2016-12-19 11:42:37,345 ERROR          curator.cli.utils             get_client:143  Connection failure.
2016-12-19 11:42:37,345 ERROR          curator.cli.utils             get_client:144  Unknown mimetype, unable to deserialize: text/html
Traceback (most recent call last):
  File "/Users/gilzellner/.virtualenvs/temp/lib/python2.7/site-packages/curator/cli/utils.py", line 134, in get_client
    check_version(client)
  File "/Users/gilzellner/.virtualenvs/temp/lib/python2.7/site-packages/curator/cli/utils.py", line 89, in check_version
    version_number = get_version(client)
  File "/Users/gilzellner/.virtualenvs/temp/lib/python2.7/site-packages/curator/api/utils.py", line 202, in get_version
    version = client.info()['version']['number']
  File "/Users/gilzellner/.virtualenvs/temp/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/Users/gilzellner/.virtualenvs/temp/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 220, in info
    return self.transport.perform_request('GET', '/', params=params)
  File "/Users/gilzellner/.virtualenvs/temp/lib/python2.7/site-packages/elasticsearch/transport.py", line 357, in perform_request
    data = self.deserializer.loads(data, headers.get('content-type'))
  File "/Users/gilzellner/.virtualenvs/temp/lib/python2.7/site-packages/elasticsearch/serializer.py", line 74, in loads
    raise SerializationError('Unknown mimetype, unable to deserialize: %s' % mimetype)
SerializationError: Unknown mimetype, unable to deserialize: text/html