“Throughout my career I swam for form.
Speed came as a result of it”
When Steve and I announced the advent of ScientiaMobile last summer, we knew that a mountain of work was awaiting us. One of the many challenges was that, over time, each WURFL API had taken a life of its own. If device detection was what ScientiaMobile sold, we’d better make that device detection consistent across all platforms and languages.
To explain the challenge to the uninitiated, let me take one step back. I could easily argue that matching user-agent strings precisely is still today more of an art than science. Obviously, there is not much wiggle room when it comes to detecting an iPhone or a Galaxy Tab, but what importance should be given to correct matching of desktop web browsers? how should one match apps that spoof the User-Agent string? is the device make and model a better “approximate match” than the correct OS version alone?
There is more than one good answer to each of these questions. Good, yet different from one another. After all, every mobile project is one-off. Which is why the different APIs had, over time, taken their own decisions based on the subtly different communities that employed them and the respective maintainers.
Last summer the challenge for ScientiaMobile was clear: how do we harmonize different APIs?
The first step was to track which requirements caused different design decisions in the first place. The second step was to agree on a common way to address those requirements.
Here are the main points that resulted from that process.
Giving Desktop Web Browsers the importance they deserve
WURFL was originally created to detect mobile devices. Back in 2005, the web browser patch introduced the ability to recognize Desktop browsers, but, let’s face it, that was always an afterthought. Version 1.4 fixes this. Recognizing Desktop Web Browsers is now a crucial new feature of API 1.4 in all platforms. The fact that the web browser patch has been merged into the big WURFL is the obvious consequence.
WURFL has now become a framework to detect everything HTTP, or, at the very least, allow its users to detect everything HTTP (there are a lot of funny UA animals in the HTTP jungle).
Two use-cases: mobile only traffic vs. mixed web-mobile traffic
If you place WURFL in front of your http://m.site.com server, 98% of your traffic will come from mobile devices. A random hit by a desktop browser won’t matter all that much.
On the other hand, if you place WURFL in front of http://www.site.com, chances are that 90% or more of your traffic comes from desktop web browser. That’s the case when you may not want the whole enchilada of WURFL matchers to take time and precious memory to track thousands of UAs that essentially just map to a (rather uninteresting) desktop web browser:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 2.0.50727)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; WinuE v6; WinuE v6)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; ShopperReports 3.0.243.0; SRS_IT_E8790072B176595B3FAD98; Hotbar 11.0.78.0)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.7413)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 2.0.50727)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 1.1.4322)
:
In essence, there are two use cases for the usage of WURFL:
- handle mobile traffic
- handle mixed desktop-mobile traffic.
We decided that the WURFL API had to support these two use cases. We handled it by introducing a new configuration called High-Performance vs. High-Accuracy.
High-Performance (HP), the default, will handle UAs that clearly come from desktops in the quickest and most economical way possible. The trade-off is that programmers will need to be happy with ‘generic_web_browser’ as a match. Don’t expect WURFL to go deeper than that. Just assume that it can bring that far fast and with no memory consumption
If you still want to know whether your Desktop web browser is MSIE, FireFox, Desktop Safari or Chrome, that’s still totally possible with the High-Accuracy (HA) mode. HA will run the matchers also for Desktop browsers. If mobile-traffic is 95% of what your site sees, HA is most probably OK.
Android: the OS vs. Device Dichotomy
Another important aspect in the new API regards Android and devices that can be upgraded from a given OS version (say 2.1) to the next (say 2.2). What should WURFL do when a device make and model is recognized, but a profile with the correct OS version is not in WURFL? There are two options, return a generic OS profile (with the right OS version of course), or return a device profile with the right make and model, but wrong OS version. We decided that the first option was better. This would eliminate the false sense of security that WURFL was correctly detecting devices when it really wasn’t. To add to that, ScientiaMobile has a process in place to constantly add unrecognized devices to the WURFL repository. “Unrecognized” devices get added to WURFL in a matter of days.
iPhonites and real iPhones that escaped from Alcatraz
Many sites look for the “iPhone” string in the User-Agent to detect iPhones and serve a better UX to them. Some browser and device manufacturers are trying to fool sites by adding the ‘iPhone’ substring in the User-Agent (‘TiPhone’ made my day). Well, not so fast, kid. The presence of the ‘iPhone’ substring is no longer enough to fool WURFL.
But there is more. There seem to be a decent amount of jailbroken iPhones out there. In a lot of cases, the UA string is modified in ways that kept WURFL from detecting the OS version. With API 1.4, your server can now correctly detect the right OS version of many jailbroken devices.
Nokia. Connecting people.
The Nokia platform has had two important evolutions. The support of Microsoft Windows Phone 7.5 and the introduction of the S40 Ovi browser on Series 40 devices. Extra logic was added to 1.4 to match these browsers and devices optimally (when matched to a WURFL device) or, at least, provide sensible defaults (you can still say a lot about a device just because you know it’s Windows Phone OS 7.5).
Of course, these are just the major improvements. There is plenty of extra logic that has found its way into version 1.4 of the WURFL API.
As far as performance is concerned, the Java API can detect 24,000+ unique devices per second in batch mode on Sri’s Vaio laptop.Moving to production, one of our enterprise customer is pushing almost 800 device detections (including 30+ capabilities) per second on a single instance of our DB API.
Finally, talking about performance. We have developed a C++ version of the WURFL API 1.4, including an Apache Module. We won’t be releasing that version publicly. Interested parties can contact us for licensing.
Try Yourself!
We launched the new APIs over the weekend. http://wurfl.sourceforge.net/apis.php
The best that you can do is download API 1.4, feed it a few of your logs and see the improvements for yourself.
Luca Passani
CTO @ScientiaMobile
