User manual

GUI

We can use the EnrichmentEnginge application from a graphical user interface. Using the ip address example, we need to complete the following forms:

Ip - ip address you are looking for
Service - service name, e.g. spamhaus, tor, censys
Parsed - setting the value to false will display the raw output

cURL

We can also query the API using the cURL command. Analogous to the GUI, the command should be completed with specific fields. Example below:

curl -XGET "http://127.0.0.1:8000/domains?domain=test&service=spamhaus&parsed=false" -H "accept: application/json"

Output:

{"Spamhaus":{"Domain":"test","Dataset":{"DBL":{"Found":true,"response":{"status":200,"resp":[2002]},"info_2002":{"response":{"URL":"https://www.spamhaus.org/dbl/","dataset":"DBL","explanation":"The resource is or belongs to a domain name with poor reputation."}}},"ZRD":{"Found":true,"response":{"status":200,"resp":[3002]},"info_3002":{"response":{"URL":"https://www.spamhaus.org/zrd/","dataset":"ZRD","explanation":" This domain was first observed between 0 and 2 hours ago."}}}}}}

Logstash

We can enrich the data in real time using logstash. We can use the http plugin in the filter section for this.

filter {
  if [ip] {
  http {
    url => "http://localhost:8000/ip_addresses"
    query => { "ip" => "%{ip}" "parsed" => "true" }
  }
  }
}

In order for a document to be enriched with new data, it must include an IP field in this case. Below is the entire sample filter file:

filter {
  if [ip] {
  http {
    url => "http://localhost:8000/ip_addresses"
    query => { "ip" => "%{ip}" "parsed" => "true" }
  }
  }

  if [domain] {
  http {
    url => "http://localhost:8000/domains"
    query => { "domain" => "%{domain}" "parsed" => "true" }
  }
  }

  if [hash] {
  http {
    url => "http://localhost:8000/files"
    query => { "hash" => "%{hash}" "parsed" => "true" }
  }
  }

  if [url] {
  http {
    url => "http://localhost:8000/urls"
    verb => "POST"
    query => { "parsed" => "true" }
    body => { "url" => "%{url}" }
    body_format => "json"
  }
  }
}