WiFi Testing with Ixia IoT
As previously discussed, RF testing has always been a major unknown to some extent because environmental factors make it extremely difficult to tell exactly what is going on with the system. I don’t think it really needs to be said but previous reviews and any controversy regarding the quality of RF has always lead to a ring of confusion and back and forth with no clear-cut answers, at least in the public domain. The Transformer Prime and Pixel C reception issues have all been cases where I’ve seen a lot of confusion over whether a problem really existed in the hardware, software, or with the end user.
Most people really don’t have any understanding of how wireless transmission works, probably because it’s not really something you can see. As far as I know, no one is capable of seeing radio waves, even at high frequencies like 60 GHz. Of course, the problem is that for quite some time our testing was also not really ideal for seeing the quality of an RF implementation. While iPerf does provide some useful data, free space testing means that we’re dealing with channel conditions that inherently cannot be controlled. As a result, the only sensible test we could do with iPerf was focus on maximum throughput in the best conditions we could provide. The only thing that this can highlight is the upper bound of efficiency for WiFi due to the carrier sense multiple access scheme in most cases, and rarely detects a whole class of problems that affect user experience on WiFi.
In order to test these things we’ve moved to using a proper testing system that is actually used by at least a few OEMs today, namely Ixia IoT. While we discussed the possibilities for testing, at this time due to the RF isolation chamber used we are limited to AP simulation only, so we can’t properly simulate clients in the channel without restricting ourselves to a single spatial stream for both the AP and client. This wouldn’t be a very useful test if set up in this manner as most devices today that we’re testing have support for two spatial streams, and many routers have three or even four spatial streams at this point.
The first set of results we can talk about that will be of interest is rate vs range. This is a fairly simple test at a conceptual level, as it simply tries to see how well a device can maintain its performance in the face of reducing signal to noise ratio for a given modulation and coding scheme. This is a good high level test of how well a device can maintain a connection as reception degrades. In this test the HTC 10 had an initial RSSI of -28 dBm while the GS7 was at -21 dBm and the iPhone 6s at -22 dBm, which allows us to calculate the path loss and determine the RSSI as a function of the transmit power.
The results of this test are interesting to say the least. Off the bat, every device had different RSSIs measured, so this meant that everything had different levels of path loss. The HTC 10 seemed to have the most path loss, while the Galaxy S7 and iPhone 6s were functionally identical. However it looks like RSSI is really an insufficient metric here because while the iPhone 6s was able to reach maximum throughput using NSS 2 MCS 8, the HTC 10 and Galaxy S7 did its best at NSS 2 MCS 4 or 5. I suspect this may be just due to placement as device positioning strongly affects MIMO as receive-side spatial correlation reduces the gains that MIMO can provide. Regardless, the HTC 10 somehow manages to beat the Galaxy S7 through much of the curve, but for some reason suffers from a reduction in throughput at higher transmit power. It's worth mentioning though that this test doesn't allow for testing of antenna gain or similar tests. Given various levels of futzing about with the device positioning in the test chamber I'm fairly confident that the Galaxy S7 is consistently better with regard to path loss, so even if it doesn't perform as well at a given RSSI it tends to have a higher RSSI than the HTC 10 by about 5 dBm which is fairly significant.
Finally, the other test that we can run at this time is the roaming latency test, which tests how well a device can hop from one access point to another as the received transmit power rises and falls. If you ever rely on WiFi to work as you walk around any building larger than a single apartment unit, you’re going to feel the effects of high roaming latency as VOIP calls or any real-time network application will either experience interruption or drop altogether if roaming is not implemented properly.
In the case of the Galaxy S7, roaming latency is honestly rather wanting. In the best case the Galaxy S7's roaming latency appears to be acceptable, but it's still significantly worse than the best we've seen so far. It seems that Samsung's algorithms have issues with edge cases as I've seen multiple instances so far where the device just can't handle roaming consistently. Despite consistent positioning and identical test setup I've seen cases where the Galaxy S7 has problems with consistent roaming. Even with the simple case of 10 dBm to -45 dBm at 3 dBm drop per second, I've encountered weirdness where the device drops from the network altogether claiming that the password given was incorrect (it wasn't) or a few successful handovers followed by getting stuck on a single access point or dropping from the network entirely. Even in the best set of trials performed I still saw 3 of 64 trials fail to roam correctly. The performance is certainly far better than something like the Google Pixel C, but Samsung should really be focusing on improving here.
ncG1vNJzZmivp6x7orrAp5utnZOde6S7zGiqoaenZH5xfZhvZq2glWLAornSrqWgZZeWuaLE2GaqcGWRo7Fuv5ZmnJ2flWK%2FpsLInq5mqJGnwW5%2BjnA%3D