This article appeared first on the APNIC website.
At the recent SANOG meeting held in my homeland, Pakistan, I wanted to provide the local community with some insights into the importance of Internet exchanges (IXs), specifically the need to host content locally.
Knowing that data is king among network operators, I set up a virtual machine as soon as I arrived to collect information on several key metrics, including latency and the hosting location of .pk domains. Needless to say, the results were surprising.
How long does it take to connect to public Domain Name System (DNS) services?
First, I tested for latency, specifically the time it takes to PING three of the most popular public DNS services: Cloudflare DNS (1.1.1.1), Google Public DNS (8.8.8.8), and Quad9 (9.9.9.9). PING is not the best way to test DNS but this is for reachability purpose only.
Before leaving my home in Sydney, Australia, I did the same to offer a comparison. As you can see from the results in Figure 1, all were below 1ms.
The results for Pakistan were less consistent.
Cloudflare was the best of the bunch with an average of 3ms, thanks largely to the three data centres they opened in Karachi, Lahore, and Islamabad in late 2018.
Google was second and even though it has several caches hosted in Pakistan, its latency hovered around 25ms.
Quad9, which has no local points-of-presence, came in at a very slow 203ms.
What this experiment shows is the importance of hosting services locally. We could expect to see a further reduction in latency if such services were also hosted at a local IX, as it would reduce the number of hops for those that peer with it to one – the benefits of which would be reduced transit costs, with the savings being handed down to customers.
Currently, Pakistan has one IX, PKIX, which has two exchange points – one in Islamabad and one in Karachi – with 17 members peering with it.
Where are “local” websites hosted?
As we’ve seen so far, the best way to reduce latency for DNS services is to make them local, so packets don’t have to travel extreme distances. The same can be said for web content too, whereby websites hosted locally should, in theory, be faster to connect to than those hosted internationally.
So, for my second experiment, I looked at where websites with .pk country code top-level domains (ccTLDs) were being hosted.
The time of my test was short so I could only get public data, which was around 15,000 .pk domains. However, like the latency test, it provided a very clear picture of the current situation.
Of the 14,927 .pk domains, only 1,541 are being hosted by Internet service providers (ISPs) or entities registered in Pakistan, and another 1,304 are being hosted by Cloudflare. That’s less than 20% of sites that, you would think, would be targeting local users. Instead, more than 80% of “local content” is being hosted in servers outside of Pakistan, and, as such, is subject to international transit costs and increased latency.
Digging deeper, I also looked it up for gov.pk, who you would think are targeting the people in Pakistan, right? Out of 431 gov.pk domains I found in the 15,000 domain addresses, only 72 are registered or advertised by Pakistani entities, 28 are with Cloudflare, and the remaining 331 are outside of Pakistan.
Needless to say, I was quite surprised that so many local council websites are hosted outside of Pakistan, most likely because they were cheap.
Finally, I looked at edu.pk. For a university to host something outside of Pakistan and not within the university is blasphemy for me. Unfortunately, this is the case for the majority of edu.pk sites with fewer than 20% hosted locally.
How many websites have Quad A records?
The IPv6 Task Force was established in Pakistan around 2006/07 during which time it built a test network between several ISPs. Unfortunately, IPv6 never caught the limelight beyond testing and enabling in core networks, and capability remains less than 1%.
I was hoping that because many websites are hosted outside Pakistan, that content might have been accessible via IPv6. Unfortunately, this wasn’t the case either, with the exception of Cloudflare.
One thing that’s not clear from Figure 5 is that of the 1,304 websites hosted by Cloudflare, why were only 1,087 responding to AAAA requests? Have some turned off IPv6? If so, why? I plan on digging deeper into this using the virtual machine generously provided by Rapid Compute (a cloud services provider in Pakistan).
Finally, it’s worth pointing out that all of the above would be a lot worse if it weren’t for Cloudflare setting up its Content Delivery Networks (CDNs) in Pakistan in 2018. However, if more CDNs are set up in Pakistan, there needs to be more local content available for them to host. This surely cannot be too difficult for an economy with more than 200 million people.
Internet Exchange Points are vital to bringing faster and more affordable Internet to people. Learn how you can support them!