
If you are reading this, you are probably already familiar with ELK stack, a set of tools (Elasticsearch, Logstash and Kibana) for searching, monitoring, and analyzing machine-generated big data via a Web-style interface (Elastic.co has a few great videos that explain ELK to the right level of detail, but you’ll need to register to access).
Logstash collects, processes and forwards events and log messages to Elasticsearch for indexing. Kibana provides a professional dashboard to search and navigate that data, including the cool dataviz which we have come to take as a standard feature of professional products.
Users can configure Logstash to read and write event streams in a wide range of formats, including all kinds of enrichments and transformation of their data. As you can imagine, a lot of the support for that transformation is an integral part of Logstash, while you can find additional support for more advanced use cases by installing Logstash filter plugins, available online.
Purely for sake of example, let’s assume that your events contain an IP address field. You may want to augment your data with a Country field that Kibana or some other tool will use to generate a choropleth map down the road.

Enter WURFL Device Detection
In the same spirit as the IP address example above, data professionals may want to augment their data streams with detailed information about the characteristics of devices accessing a web service. You can now use the WURFL device detection filter plugin to enrich event streams with device information. This Logstash plugin relies on any instance of WURFL Microservice (including the versions you can obtain from the AWS, Azure and GCP marketplaces) to provide WURFL device intelligence.
In this post, we will show how to integrate the WURFL device detection filter plugin with Logstash to add device information to your events. More specifically, we will configure Logstash to receive HTTP request information that the filter will augment with device capabilities obtained through the WURFL Microservice Client API. In the example, we will also configure Logstash to use its native output plugin to write to the console output.
Prerequisites
- Logstash 7.6+, Logstash 8.0
- WURFL Microservice Logstash plugin (available on RubyGems).
- A running instance of WURFL Microservice.
If you want to build the plugin from the sources, you can find the code and build instructions on GitHub repository.
The instructions that follow should be straightforward for anyone familiar with Logstash and ELK.
Installing the WURFL Filter Plugin on Logstash
From the logstash installation bin directory, run:
./logstash-plugin install --local <plugin_project_home>/logstash-filter-logstash_filter_wurfl_device_detection-x.y.z.gem
Configuring WURFL Microservice device detection plugin

Logstash uses JSON files to configure its plugins. Here’s our example: wurfl_filter_config_with_http_input.conf
input { http { host => "0.0.0.0" port => "19080" } } filter { logstash_filter_wurfl_device_detection { source => "headers" cache_size => 300000 inject_wurfl_id => true inject_wurfl_info => false inject_wurfl_api_version => false scheme => "http" host => "localhost" port => "8080" } } input { file { type => "json" path => "<path to>/logstash-filter-wurfl/sample_input/kafka_events_mini.json" start_position => "beginning" } } filter { json { source => "message" } } filter { wurfl_device_detection { source => "message" cache_size => 300000 inject_wurfl_id => true inject_wurfl_info => false inject_wurfl_api_version => false # Pick capabilities by listing them here. If no capability is specified, all available # capabilities will be made available. #static_capabilities => ["model_name", "brand_name"] #virtual_capabilities => ["form_factor"] scheme => "http" host => "localhost" port => "8080" } } output { # you may choose whatever output you want stdout { codec => rubydebug } }
This JSON snippet configures input, output, JSON and WURFL Microservice filter plugins.
The input plugin reads one or more JSON files configured in the `path` field. The JSON filter plugin parses the JSON data into a Java object that can be either a Map or an ArrayList
depending on the file structure.
The output plugin instructs Logstash to redirect the outcome of the device detection to the standard output.
The wurfl_device_detection
filter JSON snippet contains all the configurations that the WURFL Microservice plugin requires to work.
For a description of the plugin’s configuration parameters you can refer to the documentation. The host and port config parameters are mandatory (or Logstash wouldn’t be able to communicate to the WURFL Microservice server). To reiterate: the server can be running in your infrastructure (through Docker) or can be obtained from the AWS, Azure or GCP marketplaces respectively. The product performs the magic of keeping device data updated without you noticing.
You can find a complete sample configuration to use an HTTP input plugin here.
Starting Logstash
From logstash bin directory, start Logstash with -f
option:
./logstash -f <path_to_configuration>.conf>
Once started, Logstash will start ingesting data from the path configured in the input plugin, for example this simple JSON file (pretty printed for easy of reading):
{ "message": [ { "headers": { "Accept": "*/*", "Accept-Encoding": "gzip, deflate, compress", "User-Agent": "Mozilla/5.0 (Linux; Android 9; SAMSUNG SM-A105FN) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/11.1 Chrome/75.0.3770.143 Mobile Safari/537.36", "X-Forwarded-For": "1.22.33.44" }, "date": 1600760569, "node": "ip-1-2-0-3", "source": "mywebapp", "client_ip": "1.2.5.22" }, { "headers": { "Accept": "*/*", "Accept-Encoding": "gzip, deflate", "User-Agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 12_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/81.0.4044.124 Mobile/15E148 Safari/604.1", "X-Forwarded-For": "4.5.6.7" }, "date": 1600760569, "node": "ip-1-2-3-124", "source": "yourWebApp", "client_ip": "1.2.3.4" } ] }
The file contains two minified JSON records from a Kafka server with HTTP requests.
WURFL Microservice will perform the device detection and send the output to the console which consists in the HTTP request informations enriched with WURFL data, for example:
"@version" => "1", "message" => [ [0] { "date" => 1600760569, "client_ip" => "1.2.5.22", "source" => "mywebapp", "headers" => { "User-Agent" => "Mozilla/5.0 (Linux; Android 9; SAMSUNG SM-A105FN) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/11.1 Chrome/75.0.3770.143 Mobile Safari/537.36", "Accept-Encoding" => "gzip, deflate, compress", "Accept" => "*/*", "X-Forwarded-For" => "1.22.33.44" } }, [1] { "date" => 1600760569, "client_ip" => "1.2.3.4", "source" => "yourWebApp", "headers" => { "User-Agent" => "Mozilla/5.0 (iPhone; CPU iPhone OS 12_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/81.0.4044.124 Mobile/15E148 Safari/604.1", "Accept-Encoding" => "gzip, deflate", "Accept" => "*/*", "X-Forwarded-For" => "4.5.6.7" } } ], "wurfl" => [ [0] { "model_name" => "SM-A105FN", "marketing_name" => "Galaxy A10", "is_touchscreen" => "true", "is_full_desktop" => "false", "is_mobile" => "true", "is_robot" => "false", "form_factor" => "Smartphone", "is_smartphone" => "true", "is_phone" => "true", "wurfl_id" => "samsung_sm_a105f_ver1_subuan", "complete_device_name" => "Samsung SM-A105FN (Galaxy A10)", "is_tablet" => "false", "brand_name" => "Samsung", "is_smarttv" => "false" }, [1] { "model_name" => "iPhone", "marketing_name" => "", "is_touchscreen" => "true", "is_full_desktop" => "false", "is_mobile" => "true", "is_robot" => "false", "form_factor" => "Smartphone", "is_smartphone" => "true", "is_phone" => "true", "wurfl_id" => "apple_iphone_ver12_4", "complete_device_name" => "Apple iPhone", "is_tablet" => "false", "brand_name" => "Apple", "is_smarttv" => "false" } ], "path" => "/home/vercingetorige/dev/git_repo/logstash-filter-wurfl/sample_input/kafka_events_mini.json", "host" => "my-laptop", "type" => "json",
WURFL capability values are available in the wurfl
JSON object. The actual number of WURFL capabilities you have access to depends on which WURFL Microservice plan (Basic, Standard or Pro) that you have acquired from the Azure, AWS, or GCP marketplace. The Basic plan offers 13 device capabilities, the Standard plan offers 26 capabilities, and the Pro plan offers 41 capabilities. The Docker version allows you to select any of WURFL’s 500 device capabilities to address your use-case and license WURFL directly from ScientiaMobile.
Conclusion
Logstash and the ELK stack offer a powerful way to tame the streams of events and data that your service generates behind the scenes. We have shown how to augment HTTP data that goes through Logstash with device information from WURFL.
Adding device information will provide new insights into how your users are adopting your service. Thanks to WURFL and WURFL Microservice, Logstash users can now do that in a few easy steps.