python elasticsearch.exceptions.AuthenticationException error
Problem
nmap xml data not indexing(saved) as elasticsearch
nmap port scan result -> VulntoES.py -> elasticsearch indexing
VulntoES.py ??
Vulnerability Data in ES
This small python script will ingest several different types of vulnerability/port scanners and import that data into Elasticsearch. Right now it supports:
- Nessus
- Nikto
- Nmap
- Openvas
The script creates a python dict from a vulnerability scanner output. It converts that to JSON and inserts that document into Elasticsearch.
python code download : https://github.com/ChrisRimondi/VulntoES
Error
Point : “AuthenticationException(401, u’security_exception’, u’missing authentication credentials for REST request”
status_code, error_message, additional_info
elasticsearch.exceptions.AuthenticationException: AuthenticationException(401, u'security_exception', u'missing authentication credentials for REST request [/ivre-vuln-2022/vuln]')
python /root/.../nmapelk/VulntoES/VulntoES.py -i /root/.../ivre/scans-2022-02/CLOUD/up/211/34/148/123.xml -e 1x.xx.1x7.x2 -p 9200 -r nmap -I ivre-vuln-2022
Sending Nmap data to Elasticsearch
Traceback (most recent call last):
File "/root/.../nmapelk/VulntoES/VulntoES.py", line 450, in <module>
main()
File "/root/.../nmapelk/VulntoES/VulntoES.py", line 441, in main
np.toES()
File "/root/.../nmapelk/VulntoES/VulntoES.py", line 241, in toES
self.es.index(index=self.index_name,doc_type="vuln", body=json.dumps(dict_item))
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 84, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/__init__.py", line 370, in index
"POST", _make_path(index, doc_type, id), params=params, body=body
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 358, in perform_request
timeout=timeout,
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 257, in perform_request
self._raise_error(response.status, raw_data)
File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 182, in _raise_error
status_code, error_message, additional_info
elasticsearch.exceptions.AuthenticationException: AuthenticationException(401, u'security_exception', u'missing authentication credentials for REST request [/ivre-vuln-2022/vuln]')Cause
Elasticsearch connection authentication 401 error.
Solution
Elasticsearch is using authentication. You need to set the http_auth parameter set to (‘USERNAME’, ‘PASSWORD’) when configuring the connection to Elasticsearch
OS Ubuntu
python code fix (http_urllib3.py)
/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.pyline 85 -> Modify Elasticsearch x-pack authentication credentials
http_auth=None, -> http_auth=('username', 'password'), def __init__(
self,
host="localhost",
port=9200,
http_auth=('username', 'password'),
use_ssl=False,
verify_certs=VERIFY_CERTS_DEFAULT,
ssl_show_warn=True,
ca_certs=None,
client_cert=None,
client_key=None,
ssl_version=None,
ssl_assert_hostname=None,
ssl_assert_fingerprint=None,
maxsize=10,
headers=None,
ssl_context=None,
http_compress=False,
cloud_id=None,
api_key=None,
**kwargs
):Result
code fix -> nmap port scan result -> VulntoES.py -> elasticsearch indexing
python /root/.../nmapelk/VulntoES/VulntoES.py -i /root/.../ivre/scans-2022-02/VIRTUAL/up/1/xx2/114/x0.xml -e x.xxx.xx.32 -p 9200 -r nmap -I ivre-vuln-2022
Sending Nmap data to Elasticsearch
python /root/.../nmapelk/VulntoES/VulntoES.py -i /root/.../ivre/scans-2022-02/VIRTUAL/up/1/xx2/114/x2.xml -e x.xxx.xx.32 -p 9200 -r nmap -I ivre-vuln-2022
Sending Nmap data to Elasticsearch
python /root/.../nmapelk/VulntoES/VulntoES.py -i /root/.../ivre/scans-2022-02/VIRTUAL/up/1/xx2/114/x3.xml -e x.xxx.xx.32 -p 9200 -r nmap -I ivre-vuln-2022
Sending Nmap data to Elasticsearch
python /root/.../nmapelk/VulntoES/VulntoES.py -i /root/.../ivre/scans-2022-02/VIRTUAL/up/1/xx2/114/x4.xml -e x.xxx.xx.32 -p 9200 -r nmap -I ivre-vuln-2022
Sending Nmap data to Elasticsearch
python /root/.../nmapelk/VulntoES/VulntoES.py -i /root/.../ivre/scans-2022-02/VIRTUAL/up/1/xx2/114/x5.xml -e x.xxx.xx.32 -p 9200 -r nmap -I ivre-vuln-2022
Sending Nmap data to Elasticsearch
python /root/.../nmapelk/VulntoES/VulntoES.py -i /root/.../ivre/scans-2022-02/VIRTUAL/up/1/xx2/114/x7.xml -e x.xxx.xx.32 -p 9200 -r nmap -I ivre-vuln-2022
Sending Nmap data to Elasticsearch
python /root/.../nmapelk/VulntoES/VulntoES.py -i /root/.../ivre/scans-2022-02/VIRTUAL/up/1/xx2/114/x8.xml -e x.xxx.xx.32 -p 9200 -r nmap -I ivre-vuln-2022
Sending Nmap data to Elasticsearch
python /root/.../nmapelk/VulntoES/VulntoES.py -i /root/.../ivre/scans-2022-02/VIRTUAL/up/1/xx2/114/x9.xml -e x.xxx.xx.32 -p 9200 -r nmap -I ivre-vuln-2022
Sending Nmap data to ElasticsearchElasticsearch Kibana index information after code modification
Indexing (save) successful!!
