We've created a new site for Telecom Analytics news and analysis called All In The Data where you can also find my latest blog postings.

The Three Vectors of Mobile Analytics

Today’s announcement of JDSU’s acquisition of Arieso, a UK based, location analytics company, demonstrates an industry movement to obtain new sources of intelligence about the performance and usage of mobile networks.    The migration from voice to data services has left mobile operators blind to how networks, applications and services are being used by consumers – a gap that the vendor community is anxious to fill.

To fill this gap, it turns out that there are three main sources of data within mobile networks from which analytics and intelligence can be built through network and device monitoring – each have their own benefits and considerable challenges.    

Will Firefox OS win where webOS didn't?

Two separate but interconnecting news stories this week: The announcement that mobile operators including Telefonica, Telenor, America Movil and Deutche Telecom all plan to bring Firefox OS to market in new smartphones; and HP finally sold webOS to LG for use in smart TVs.

The connection?   webOS was based on the premise that beyond basic phone features, all content, games, media and services could and should be delivered through the web (as web apps), not with native applications installed on the phone.   Firefox OS as a new player to the smartphone arena also thinks HTML 5 based web apps will unlock the walled gardens built by Google and Apple around Android and iOS, because you don’t need to go to an app store to use a web app.

The logic is certainly sound – for developers in the HTML 5/JavaScript world, it doesn’t matter if you are using a PC, a Mac an iPad or an Android phone, with some minor adjustments for screen size, you only need to develop for one platform.   Lamentably, it didn’t work out for Palm/HP so what is different now and why have a litany of mobile operators signed-up?  

This story is ultimately about control and ownership of services that mobile operators feel are being ceded to Google, Apple and OTT (over-the-top) content players.  For mobile operators, having a customizable smartphone environment they can call their own, together with their powerful distribution channels places their brands back in the driving seat of service innovation.

Because of the openness and use of HTML 5 & JavaScript across all aspects of the device – even the dialer –mobile operators should be able to integrate Firefox OS tightly with the network to take advantage of investments being made in technologies such as VoLTE and RCS (rich communication services), essentially showcasing their own services over OTT players.  For consumers, the operator’s own web app store will be the first (but not only) port of call for new applications and services, providing additional revenue sources.  We can see why this begins to look like a “get well plan” for ARPU growth.

So will developers jump on the HTML 5 development bandwagon, or stay hitched to iOS and Android?   It turns out that many of the apps available today are coded as web apps in HTML 5, cunningly disguised as native apps and packaged for iOS and Android.   For the developer community, this makes for easy portability between platforms.  It’s also a strategy adopted by Blackberry for fast ramp-up of their app portfolio in Blackberry 10 OS.  The challenge has been performance and integration – web apps run more slowly than native apps and historically, you couldn't access all the cool smartphone features such as multitouch.   This has largely been addressed, but with the compromise that developers may need to modify code differently to access the “smart” features of different operating systems.

There have also been discussions about the low price point of Firefox OS devices allegedly making them ideal for South American and Asian markets.    However, given shared components and specifications, it seems unlikely that Android smartphones would be any more expensive than Firefox OS smartphones (short of operator subsidy) meaning that the two operating systems will likely duke it out, market by market.

So the difference between webOS and Firefox OS and critically, the potential for Firefox OS success is largely about timing and messaging.     Mozilla and Firefox OS are seen by mobile operators as a required strategic counter to Android and Apple hegemony.  Further, Mozilla’s ethos is attractive in a way that HP never could be, primarily because mobile operators don’t believe that Mozilla will ever compete with them for brand status or control of the eco-system.

SDN distracting from IPv6 migration needs

Another year and another heralding of IPv6 – 2013, the year where IPv4 addresses finally ran out and IPv6 becomes mainstream - perhaps.     Despite the skepticism, there are genuine reasons why this year could see your new shiny smartphone on an IPv6 network and for mobile operators at least, for the next few years migration from IPv4 is more pressing than deploying SDN.

It comes down to a twist on the original reason we were supposed to move to IPv6 – the lack of new IPv4 addresses to handout to newly minted smartphones.

By the time mobile operators came around to enabling IP networking for mobile devices, most public IPv4 addresses had been gobbled-up by the fixed-line world.    But instead of jumping into the brave new world of IPv6, mobile operators punted - instead issuing “private” IPv4 addresses to customers.   That is, addresses that can only be used on internal networks and are not valid on the global Internet.     This worked because mobile operators then purchased Carrier-grade network address translation (NAT) gateways from the likes of Cisco and Juniper, which mapped many individual private IPv4 addresses to one public IPv4 address, just like the Wi-Fi/ADSL router in your home, but for millions of devices.    In this way, the scarce public IPv4 address space has been used efficiently and the industry has been saved from making the IPv6 leap.

Sweetening the deal, an unintended security benefit came to pass.   By deploying NAT in the carrier network, it’s as if your smartphone doesn’t really exist on the public Internet unless the smartphone itself makes a connection.  So unlike your fixed line connection with a public IPv4 address, where anyone at anytime can reach your device and attempt to break-in, your smartphone is inherently secure from unsolicited traffic.   Don’t get me wrong, you might still catch viruses from email, install rouge applications and the like, but your phone is shielded from the daily exposure of having a device on the internet.

So did procrastinating on IPv6 work or is there a sting in the tail?  

It turns out there are limits to the number of private IP addresses that can exist on the same private network.   That number is shy of 22 million addresses (which includes an additional 4 million, added for carrier-grade-NAT applications in 2012).   There is a growing club of mobile operators who have way more than 22 million “always on” IP subscribers and they face some pretty stark choices:  They can break-up the network into areas, reusing the same address space by city or region; they can unofficially “borrow” IPv4 addresses not currently being deployed in the real world, run them on their private network and hope they never show-up on the public internet (at least one major US operator has been doing this); or they can migrate to IPv6.   Pick your poison.

The “break-up your network into regions and reuse the same address space” solution has been gaining the most traction, with operators pooling users into groups of 4 million.   So, in a network of 40 million customers, this means that 10 customers will share the exact same IP address at the same time.  This creates headaches for wiretap requirements, back-end analytics systems and cross network traffic routing between subscribers (how many NAT hops is too many?).   Further, as smartphone users today have more active applications running on their smartphone (talk+map+messaging+game) and enable them as hotspots, this places extra processing burdens (read financial costs) on scaling NAT.

Cynically perhaps, the end game for IPv4 was only ever going to come when we simply ran out of runway, when staying with IPv4 became too painful, making IPv6 deployment the only way forward.    Writing one more check for yet more NAT gateway equipment has become like buying another “final” bottle of Jack for your deadbeat Uncle – we all know that an intervention is the only way forward but are waiting for something dramatic to go wrong first.

It is not as if the industry hasn’t been working behind the scenes to get ready – most RFPs, most equipment specifications have required IPv6 native support for years now.    However, it was expected that our devices would run both IPv4 and IPv6 in “dual-stack” mode that would ease the path to an IPv6-only world.     The Mac on which I type this is busy searching (in vain) for an IPv6 network, as it communicates in IPv4.    

The implications for hitting the NAT wall are startling though – you really can’t hand out another IPv4 address, private or public.  The switch to IPv6, at least for new customers and new devices must be absolute.   No IPv4 address for your new smartphone.   For many operators reaching these IPv4 scaling limits, the transition point to IPv6 is likely to be alongside LTE and/or VoLTE deployment, as these technologies represent a natural transition and technology shift in the network.   This will also ensure plenty of time for testing.

So can you deploy an IPv6-only mobile network and will users notice?   Well yes and no.   The challenge is to make the network transparent to applications and services and there are two critical locations in the network where this will play out.  First, those Carrier grade NAT gateways now have to translate between the IPv6 world of your network to the public IPv4 world – you’ve solved the scaling challenge but the rest of the world needs to catch-up and until it does, this forms the connection to any public website or service not operating in IPv6.

Second, and something of a larger challenge is that the handset/smartphone itself needs to provide the appearance of having IPv4 when it doesn't or alternatively force all applications on the handset to be IPv4/IPv6 agnostic.   Operators sometimes forget that while they can mandate handset manufacturers and equipment vendors to comply with their specifications, application vendors live outside this realm. A popular app that breaks on an IPv6 only network will cause support complications and user frustration.   

A way around this is to bring NAT inside the phone itself and for any application that needs to think it is on an IPv4 network to provide a private address that is translated to IPv6 in the phone.   Such a method exists – at least in draft in draft-mawatari-v6ops-464xlat.  Example code has already been written for Android and inclusion in new smartphones would go a long way in smoothing the IPv6 transition.    Notwithstanding, application vendors, with a little prodding from the likes of Apple and Google must be encouraged not to hardcode IP addresses into their software but to use domain names (so www.kirinsolutions.com, not 207.155.253.5).   As a consequence, the first operators that make the IPv6 only transition are likely to go through a few application challenges in the first 3-6 months of operations as consumers.

Finally, let’s go back to the security benefit of all the NAT currently going on in the IPv4 world.   When you smartphone shows up with a public IPv6 address, it becomes accessible to the outside world, just as your ADSL gateway is at home on IPv4.   Your ADSL gateway likely has a firewall and it’s own NAT functionality to protect your PCs and wireless LAN connected devices from the big bad Internet.    Handset manufacturers and operators are going to have to think carefully about what level of exposure this represents.   Operators may well need to provide a firewall service in the network to afford the same level of protection, now delivered through NAT.

For an example of 464xlat on a phone see:  https://sites.google.com/site/tmoipv6/464xlat

The Hotspot that came in from the cold

 It is amazing the steps we take to put our smartphones on to Wi-Fi rather than burn through precious data plans.  Short of forcing consumers into a game of Twister, there are few things more convoluted (or frustrating) than using a smartphone to type usernames, passwords or credit card numbers, while sitting in an airport lounge or coffee shop juggling a hot drink.   Still, this learnt behavior can be seen anywhere a smartphone is out in public. 

This complexity was meant to have been solved years ago with a protocol called WISPr – a first attempt at standardizing and automating connectivity to Wi-Fi such that, as consumers, we would never have to deal with pesky web login forms and endless service menus.   Unfortunately, this protocol never made the headway expected and as a result, only with isolated effort (think AT&T Wi-Fi access in the iPhone) has automatic Wi-Fi connectivity become available to consumers out-of-the-box.

In the absence of a concerted mobile operator, equipment and smartphone vendor push, independent Wi-Fi operators and app developers stepped into the fray providing proprietary “islands” of automated connectivity.    Consumers typically download an app to their smartphone, sign-up for a plan and then hope that where they go, service is available.    Some of the networks are big – Boingo have 600,000 hotspots, Fon have over 5 million through crowd sourcing, and Devicescape claim 12 million “curated” hotspots – each company with a slightly different business model.   Operators such as T-Mobile, Orange and PCCW have all built-out their own large-scale Wi-Fi networks, often available independent from a mobile contract, always with accompanying iPhone and Android applications.

This is all set to change with the somewhat low-key arrival of new standards and products wrapped-up in what is known variously as Hotspot 2.0, 802.11u and Passpoint.  All are, confusingly, part of the same initiative – an industry accepted method for automatically authenticating and connecting Wi-Fi users using the SIM cards of their smartphones (or cameras, toys or cars for that matter).    There are a number of additional compelling features too.   Encryption to the hotspot comes as standard and there is a hotspot advertising feature which allows venues or locations such as stadiums or shopping malls to provide location specific Wi-Fi that might connect you to a specific web site through an icon that appears on your smartphone.

What’s different this time around and why we should see traction is that the industry, through the Wi-Fi Alliance, has been busy certifying infrastructure products from companies such as Aruba, Ruckus, Cisco and BellAir, along with chipset and handset vendors such as Intel, Qualcomm, Marvell, LG and Samsung.

Of course, just because products are now available doesn’t mean we can get connected tomorrow.  Niels Jonker, Chief Scientist at Boingo, speaking at CES last month claimed that roll-out would likely take from now through 2015.    Importantly though, having the entire ecosystem participate should mean that when you purchase a smartphone with an operator contract, any Wi-Fi hotspot which has a roaming agreement with your operator should (over this time period) allow automatic and seamless connectivity.

This brings us to an interesting position for mobile operators and their focus on Wi-Fi.  The assumption has been that consumers with smartphones will pay for both a voice and a data plan.   Increasingly, this is turning out not to be the case, especially in countries such as India, China and Indonesia where smartphones are becoming every bit as popular as the US and Europe.   Consumers are buying the same pre-paid voice plans as before, but exclusively leveraging Wi-Fi for their data.   

This is creating a “dammed if you do / dammed if you don’t” moment for operators.  Remember – the master industry plan involves delivering LTE to everyone everywhere and Wi-Fi expenditure by operators today is a fraction of the tens of billions being poured into 3G & 4G network upgrades.   From the operator’s perspective, the best consumer buys service but doesn’t use it – a good reason operators have encouraged Wi-Fi off-load but a strategy that might be working a little too well.    If consumers have unlimited Wi-Fi and limited 3G/4G data – more automated and ubiquitous Wi-Fi will lead to less revenue for the operators.

For this reason, we are likely to see changes in how mobile operators bill for Wi-Fi as the lines between 3G/4G and Wi-Fi blur.   To put this another way – Wi-Fi has been a method for keeping users off precious 3G/4G connections but going forward, it will be integrated into seamless service where the consumer won’t know or care if they are on LTE or Wi-Fi.    Further, as voice services migrate to VoLTE – which by the way works just fine on quality WiFi networks – Wi-Fi and/or LTE deployment will be a cost choice decision for the operators based on cell size, density and available frequency bandwidth. Operator Wi-Fi will finally come in from the cold.

This leaves Wi-Fi only players in a different position too.   They will need to leverage their global alliance networks to become full-service players and deliver voice too.   Last month’s FCC announcement by Julius Genachowski to free-up unlicensed spectrum for Gigabit Wi-Fi will surely aid in this progression and lower costs for entry, at least here in the US.

In many parts of the world, we will likely see LTE and Wi-Fi go head-to-head, especially in high bandwidth and small cell deployments where cost is a primary consideration.   Indeed, I am reminded of the battle fought in the 90s over desktop connectivity.   Numerous vendors bet against Ethernet, with Token Ring, Fiber Channel and even ATM.   They lost not because they had inferior technology but because Ethernet price points and the ubiquity of Ethernet devices ruled the day.    In the “Internet of things” the volume of connected devices will be many times more than the volume of smartphones and these “things”, for cost and availability reasons will likely have Wi-Fi, not LTE.   Putting the next billion Wi-Fi devices on service contracts is the next open space.

For reference, below is a link to a list of Passpoint products certified by the Wi-Fi Alliance:

Wi-Fi Certified Passpoint Products

Will VoLTE be too late to save Mobile Operator voice?

 2012 will likely go down as the year where VoIP apps reached critical mass in mobile networks.   The roll-out of LTE networks in the US and key cities in Asia, along with the near ubiquity of Wi-Fi have been a key enabler.  Smartphone apps such as Viber, Fring, Line2 and established fixed line alternatives including Skype and Vonage are slowly migrating voice minutes from mobile operator’s traditional services to VoIP.   Perhaps most notable of all is Viber, who between May and June of 2012 added 20 million users to their subscriber pool, ending the year at 140 million subscribers.  Their subscribers are not the silent type either - collectively they sent each other two billion texts and rack-up 1.5 billion minutes of talk time per month.

This trend has surely not gone unnoticed in Mobile Operator HQs.  According to Yankee Group, global mobile voice and messaging revenue are set to decline by $1B per month through 2013 and operators have been thinking through strategies to mitigate this loss.    The answer, at least for Verizon and AT&T has been to charge more for data and device connectivity, bundling unlimited voice and text. This may seem crazy given how much revenue is generated directly from voice today but clearly demonstrates the threat that OTT players represent when operators don’t bundle services.      The genius here is that it disincentivizes the use of over-the-top VoIP applications and creates a puzzle for competitors such as Sprint and T-Mobile who, to-date have benefited from their all-you-can-eat data plans, growing their average-revenue-per-user (ARPU)  in the process.

South Korean mobile operators have seen firsthand the impact of all-you-can-eat data tied with fixed minute calling plans.   Back in April 2011, a local company, KAKAO launched a messaging app called KakaoTalk – a homegrown Korean version of the popular SMS alternative WhatsApp.  Last year, 38 million local subscribers were happily texting each other for free (the population of South Korea is close to 50 million people).  In June of 2012 KAKAO enhanced their offering to deliver a full-blown voice service between their subscribers – for free.   The outcome was extraordinary.  Taking advantage of the cheap, unlimited data plans of the local operators, within three days of service launch, subscribers were generating 20 million calls per day. 

Seeing a massive spike in data volume and a precipitous drop in voice revenue, the South Korean mobile operators moved to action within days and began throttling and then blocking VoIP traffic over their network, obtaining the blessing of the Korean Communications Commission to block or charge more for VoIP services.   It will be fascinating to see if KAKAO and others move to hide or tunnel VoIP traffic through encryption, VPNs and port shifting, creating a cat-and-mouse game between network operator and application vendor.

Perhaps not surprisingly, Korean operators are not the only ones with a block-and-tackle approach.   Vodafone has also moved to block VoIP traffic to subscribers who aren’t on their higher-rate post-pay plans and we can expect other operators to take the same approach, where regulation permits.

With large volumes of over-the-top voice traffic now running across mobile networks and with the fixed-line world as a case-study, the long-term outcome of Telco voice is certainly unclear.  What is certain is that Operators will take advantage of LTE technologies to offer superior voice and video services of their own.    The question then, is can they deploy these services fast enough, (before users migrate to the app world) and will consumers really care?     

It must be said there is a natural protection mechanism available for most mobile operators which slows-down migration, and creates a challenge for any VoIP start-up – it is called The Community-of-Interest Problem.    Your cell phone can reach anybody else on the planet with a phone – Viber, at least today – cannot – that is, not without having you pay to reach the legacy telephone world per call or in pre-purchased minutes.   The reason KAKAO in Korea could be successful so quickly was that they already had most of the population of Korea on their texting service.    In the US, perhaps only Apple (with FaceTime) has this community-of-interest leverage.  On this note, it is also important to understand the role of the handset manufacturer in this equation.    When you consider voice as just another app on the smartphone, the voice app that ships with the device is the one that is going to be used by the majority of consumers.   If it works, is free (or included in the tariff plan), allows consumers to communicate with their friends and family, has fall-back to legacy technology and allows you to reach emergency services, then inertia dictates that these consumers won’t look further afield.

What is fascinating then is that for Mobile Operators, the driver for VoLTE adoption is not to compete with the OTT players per se.  Rather, the move to VoIP services frees up valuable spectrum currently dedicated to voice in the 2G and 3G bands.    This spectrum can then be assigned to data services to feed consumer’s insatiable demand for bandwidth.    As an added benefit, the operators eventually get to switch-off the expensive legacy voice systems and run a single, unified IP network.

From a consumer perspective, VoLTE will be touted as a higher quality service and voice will be just one of a number of components including video, text, presence and location.  Mobile operators will push for these RCS (rich communication services) to be integrated with smart phone functionality, so for example, your phonebook will show who is available to talk or if a voice call can be changed to a video call.    While their original plan may well have been to charge individually for each service, the required bundling of SMS with basic consumer plans may well be resetting expectations.     As an example, MetroPCS announced back in October an RCS service called joyn – available on specific LTE handsets in their network.  Pulling-in application developers, AT&T announced earlier this month an API, allowing developers to integrate their apps with AT&T’s IMS architecture, paving the way for joint services and revenue sharing.

So in 2013 we will continue to see OTT players and handset manufacturers (but not likely many operators) drive VoIP services and consumer adoption of these services will likely dictate outcomes.    Any player with a large enough community-of-interest, bundled with a handset partnership could make significant traction – Facebook anyone?      Operators will begin to roll-out VoLTE services as both the LTE networks and handsets with VoLTE calling capabilities reach consumers.  The race for subscribers for both OTT services and VoLTE through 2015 will likely define our industry for the next 10-15 years.

Reinventing the network dashboard around applications and subscribers

Over the last few years, mobile operators and vendors have turned their attention to improving the quality and reliability of the performance of networks, of handsets and of services.    Unfortunately, in that time, many existing tools and products have simply been rebranded with a “Customer Experience Management” and “Big Data” brush, aligning them, at least in marketing terms, towards helping reduce the industry’s perennial problem : customer attrition or churn which in most markets runs at an alarming 2% - 4% of subscribers per month.

In order to address customer satisfaction (which aside from pricing is often the biggest factor in churn), mobile operators are tasked with improving the quality of their network, and the quality of support provided when consumers call with a problem.   To do this, operators need information – information on the performance of services, of devices, of networks.    Now you would think that operators have a large of amount of data which could be analyzed and understood to improve services but here lies a paradox – the more complex and heterogeneous the networks become, the more management data is generated, the more difficult it is to get the right information at the right time and make it actionable.   To put it another way, operators now have way too much information, often of the wrong type, to make business decisions and improve customer satisfaction.    This is perpetuated by traditional Telco vendors who are focused on delivering yet more information from more systems, backhauling terabytes of management traffic from across the network under the illusion that if you throw enough processing power at the data, you can yield those illusive answers to subscriber woes.

Perhaps this isn't surprising when you consider where we've come from.   In the good old days of network management, you’d display a big map of your network, usually overlaid on a map of the country (or the globe) and any problems would blink, flash or alarm red.   Fix the red flashing light and life was good.    Then networks and services became more complex – think 2G,3G,4G, Wi-Fi, smartphone, machine-to-machine all with overlaid services.  To deliver a single service such as email or a mapping app or a web search requires the cooperation of many different systems, networks and companies.   The flashing red light, as a key performance indicator went away because we lost an end-to-end understanding of our services.   To date, as an industry we really haven’t really implemented a good alternative.

As it turns out then, finding the new “red light” means looking at the network from a user experience perspective – how each application is performing for each subscriber at key points across the network.   Aggregate this information in real-time and you recreate the red light map with a new paradigm – performance by application, by market, by service, by subscriber type, by revenue generated.  

Thinking through this problem, Vistapointe, a start-up focused on network insights, have figured out how to build this performance dashboard without having to backhaul and then analyze huge amounts of traffic from each node or location in the network.    Their method relies on placing Vistapointe Intelligence Packet Engines (VIPEs) at strategic points in the network where, based on location, they analyze and process only the critical information necessary to extract performance data, relaying not raw packet captures, but summarized and compressed information to the cloud where it is aggregated with information from other nodes to build a complete picture of each application and service in near real-time.     By performing much of the data analysis deep in the network, they've made the solution naturally scalable as servers in the cloud are concerned only with aggregation and not with packet processing.

The outcome of reinventing the mobile operator’s dashboard is that, for the first time, mobile operators can understand how each and every application on their network is performing – both those they own and monetize and those of partners and content providers - and when it doesn't perform, understand what went wrong and the impact to subscribers, to services and to revenue.