Following up on our first webinar for the Americas on “Addressing the Top 10 Challenges of LTE EPC Testing,” we ran a second webinar for Europe. This again generated more questions from the participants, all of which could not be answered during the webinar itself. So, we’ve collected and answered all the questions below. We hope they are informative for those of you seeking to better understand the nuances of LTE EPC testing.
Q.1. What are the KPIs for testing the performance and quality of LTE networks?
Validating the performance and quality of an LTE network requires a high rate and varied mix of data traffic to judge the end user experience. Traffic mix can be based on type of subscribers, applications/services used, usage etc. Some of the key KPIs which need to be validated are:
- Latency and packet loss
- Perceptual video and voice quality
- Number of active sessions, users, and transactions per second
Q.2. What are the challenges in testing IPv6 in an LTE network?
Although IPv6 is mandated in LTE, as of now, most of the existing networks and their components use IPv4. As a result, we have nodes with support for both IPv4 and IPv6 stacks in the network. The integration testing of IPv4/IPv6 poses a big challenge because all network services will need to be tested for interoperability without any issues between IPv4 and IPv6. Added to this, some tunneling schemes from IPv6 to IPv4 are another area of focus.
Q.3. What are the key attributes to focus on during testing, to minimize the issues with LTE EPC nodes after deployment?
Few of the key attributes to focus on for better solution reliability are:
- The LTE EPC node should be validated with an optimized set of invalid scenarios. Prior experience in interop and field testing is a definite help for identifying such scenarios.
- While testing for the scalability and benchmarking, next to real time scenarios should be simulated, such as, different mix of traffic, simultaneous calls, new calls per second etc).
- Inter-RAT handover or inter-working with non-3GPP must be tested based on the existing coverage in the deployment area
- Redundancy and resiliency should be given more focus during testing from a deployment perspective.
Q.4. There are diameter based interfaces in earlier technologies. How is testing for those different from LTE EPC testing?
There are around 12-14 diameter based interfaces in LTE. There are many new interfaces like SWx, STa, and SWa that have been introduced to support non-3GPP interworking. At the same time there are certain interfaces like Gx, Rx, and Sh which also exist in IMS for interfacing with the PCRF. But in the LTE context, there are a multitude of additional AVPs which require the test tools to be upgraded and complicate testing. The test scenarios are quite different in terms of message flow, handling, and parameter validation.
Q.5. Do you recommend that conformance testing as per the specs by GCF/MSF is enough for an LTE node?
No, it is not enough. Conforming as per these specifications is necessary but not sufficient. It is advisable to conform to these specs as they are standardized and different stakeholders expect nodes to comply with these. But at the same time, based on our experience, the node should also be tested against many field-like complex scenarios, optimized negative scenarios, and interop testing scenarios.
Q.6. What are the challenges in identifying the right traffic mix for load testing?
The traffic mix for LTE testing needs to consider different signalling events as well as data formats.
There are multiple interfaces which can initiate signalling messages with the EPC. Testing for these requires synchronization of triggering the messages from these interfaces. There are different frequencies for each kind of message and the number of such triggers for an attached UE is also different for different events. The messages also depend on the number of subscribers normally active at any moment and signalling events differ for different number and kinds of subscribers.
The area where the EPC is going to serve also results in different kinds of traffic. In an urban area, there would be a high rate of handovers, but in rural areas the rate of handovers may be less. In an enterprise deployment, the number of active users may be more than in a consumer market and so on.
The data formats also differ a lot based on the usage of different applications. Video download may result in large packets and high data throughput, but email usage may result in larger active duration, but smaller packet sizes.
Identifying the different use cases and traffic mixes for different deployment thus becomes a challenge.
Q.7. Can you give some examples of applications that would use dedicated bearers? Do these applications typically use network initiated dedicated bearers or user initiated?
Some examples of using dedicated bearers can be found in voice and video traffic. Applications like IPTV and video on demand, which require guaranteed bandwidth, may also require dedicated bearers. The dedicated bearers can be user initiated or network initiated. Mostly for voice and video applications the network would initiate dedicated bearers based on the SDP exchanged for the media for IMS calls. For applications like IPTV and video on demand it could be either UE or network initiated.
Q.8. Can you share some of the important metrics collected for a high available EPC solution in carrier grade networks?
The important metrics for carrier grade EPC solutions validate the level of reliability and switchover time. Switchover could happen for both the control and the data plane. These important metrics are:
- Switchover time for control plane
- Switchover time for data plane
- Amount of memory for buffered packets during switchover
- Transmission time for buffered packets
- Level of reliability
- Check all levels of redundancy for all resources, interfaces, and software solutions
- Signalling and data throughput after switchover
Q.9. Is there a way to monitor the UE messages from a simulated eNodeB?
Yes. Although the UE messages are encrypted, with a simulated eNodeB it is possible to collect the UE keys from the eNodeB simulator and UE exchanges. Alternatively one can use pre-shared information. The eNodeB simulator can also decipher the NAS messages exchanged between the UE and the EPC.
Q.10. How is the voice quality measured in EPC testing and what aspects are covered as part of voice testing through EPC?
Voice quality is measured using multiple voice test tools to test the following:
- Voice quality
- Echo cancellation
- Comfort noise generation
- Voice break at handovers
Q.11. What are the essential scenarios which must be tested for an MME?
Following are the necessary scenarios which need to be tested for an MME:
- Default bearer creation/deletion/modification
- Dedicated bearer creation/deletion/modification
- Network initiated triggers (default/dedicated bearers)
- Handover (S1, X2)
- TA updates
- Release/Idle mode
Q.12. What is major challenge in testing IOT for voice scenarios like CS fall back?
For voice testing, all the different scenarios like CS fallback, IMS calls, SRVCC etc. have to be tested. The different voice application nodes like the IMS network, CS network of 3G network, SRVCC handling IMS application server, and SCP nodes have to be simulated such that they can receive and initiate triggers.
The nodes need to be tested for different user capacities and different kinds of voice applications, like emergency calls, voice and video calls, multiband audio; and require multimedia application support in the simulators. This poses a major challenge.
Q.13. How it is ensured that voice and data go on simultaneously; is it possible on CS fall back?
Voice and data can go on simultaneously in CS fallback, depending on the network and UE capability. If the PS session is going on in LTE and voice is transferred to 3G, then the UE needs to support two kinds of radio at the same time. This may or may not be possible depending on the UE’s capability. Even in the case where both PS and CS are handed over to 3G, depending on the UE’s and the network’s capability, either the PS session may be suspended when the CS session is ongoing or both PS and CS sessions may go on simultaneously.
Q.14. What are the best commercially available test tools for EPC core interoperability, inter-RAT, or non-3GPP testing?
Normally for interop, customers prefer to test with a real node from another vendor, rather than using commercially available test tools.
Altran has frameworks for different LTE EPC core elements (i.e. MME, SGW, PGW, HSS, and PCRF) and eNodeB. We have used these for interop for some of our customers.
Additionally, IxCatapult M500, NetHawk EAST EPC, Polaris LTE Functional Test Tool, Spirent Landslide, and Spirent Test Centre are some of the other tools used for EPC node testing. Selection of the tool depends on different aspects like the node to be simulated, testing phase to be simulated for, scope of testing, cost of the tool etc.
Q.15. Are there any end-end network monitoring solutions you would recommend? Which vendors?
From the monitoring or protocol analyser perspective many tools are there which focus mainly on the access network side. For example: Anritsu, R&S, Tektronix, and Aeroflex.
From an end-to-end perspective, the few vendors can be considered for network monitoring solutions are JDSU and Tektronix. Often, Wireshark or other tools used for testing can fulfill these requirements.