Pakistan Telecommunication Authority (PTA) has planned to procure automated quality of service (QoS) monitoring and benchmarking tool which would have also the support measuring audio and videos services, including live streaming as well as social media including emerging Over-The-Top (OTT) communication services testing.
According to the official documents, the automated system shall bear the specifications including the capacity to test and benchmark QoS across number of mobile operators (i.e. simultaneous and continuous measurement of at least five mobile networks), with the objective to reproduce end-user experience in different mobile networks, device types and wireless technologies (i.e. 2G/3G/4G etc.) operational in Pakistan.
PTA to Deploy Automated System to Monitor Telecoms’ QoS
The system will have data collection and data analysis (post-processing) modules and related services to benchmark voice, SMS and data QoS KPIs for at least five operators simultaneously and shall have the capability to expand to include more operators if required. It would have the capability to perform field tests, measure, benchmark and report QoS Key Advance Indicators (KPIs) as defined in mobile services licensees issued from time to time to Cellular Mobile Operators (CMOs) in Pakistan, as well as global industry standards and best practices.
The system will be required to measure all the QoS KPIs and should support all the technologies including but not limited to Global System for Mobile (GSM)/ Code Division Multiple Access (CDMA)/ Wideband Code Division Multiple Access (WCDMA)/High-Speed Download Packet Access (HSDPA)/High-Speed Upload Packet Access (HSUPA)/High-Speed Packet Access+ (HSPA+)/ Long Term Evolution (LTE)/ Voice over Long Term Evolution (VoLTE)/ Long Term Evolution Advanced (LTE-Advance) etc. (i.e. 2nd, 3rd & 4th generation wireless technologies) in all bands/spectrum, carrier aggregation in contiguous/non-contiguous bands/technologies and should be upgradeable to 5G, as and when required by the PTA.
The equipment would be able to determine and plot the route and map it with respect to its GPS coordinates. Plotting shall only show the relative movements with reference to a starting reference point. The equipment should be interfaced able with an off-the-shelf digitized map of the region, at any time in future with minimum hardware and software up-gradation. All instances have to be time and day stamped. The system should have a power measurement accuracies of up to ± ½ dB (min.). The system should have time-dependent measurements accuracies of one-second minimum. The specification shall explicitly indicate the minimum and maximum levels which can be measured by such an instrument. The system should have an extension for indoor / walk test service quality and coverage measurement.
The system would have the capability of being upgradable to consider upcoming mobile network technologies and related Advance parameters, such as VoLTE and 5G etc and active and optimized power control for phone charging. The installation of the QoS system should be in such a way that the set up provides optimal conditions for uninterrupted data collection in a stable environment for reliable test results.
The equipment should be capable of benchmarking multiple scenarios of Voice, SMS and Data services of at least five networks simultaneously and continuously. Moreover, it should have the capability to expand and include more operators, if required.
It should support both the Perceptual Evaluation of Speech Quality (PESQ) and Perceptual Objective Listening Quality Analysis (POLQA) voice/video quality Mean Opinion Score (MOS) for both narrow and wideband.
It should support Application based testing like, Facebook, Twitter, Snapchat, WhatsApp, Netflix etc and should provide flexibility to support multiple device types of multiple vendors including but not limited to smartphones, modems and mini cards etc.
It should be able to benchmark all possible testing scenarios in auto technology and locked mode. All the technologies 2G/3G/4G and all the bands can be locked also there should be flexibility to lock/unlock any desired technologies. Moreover, the tool should be able to lock and test specific Broadcast Control Channel (BCCH)/ Primary Scrambling Code (PSC)/ Physical Cell Identity (PCI)/ Absolute Radio Frequency Channel Number(ARFCN)/ Universal Mobile Telecommunication Service (UMTS) Absolute Radio Frequency Channel Number (UARFCN)/ Evolved Universal Terrestrial Radio Access (E-UTRA) Absolute Radio Frequency Channel Number (EARFCN) etc.
It should support a minimum 16 handsets for Voice, SMS and Data services simultaneously. It should have the capability to benchmark data services of all the operators simultaneously without any dependency on Laptop’s modem and without any degradation.
The equipment casing should be compact and capable of holding the mobile handsets securely in such a way that all mobile handsets are equally exposed to the outer environment for fair benchmarking. Most of the cabling should be fixed and reliable. The cabling that needs to be done by the end-user should be minimum and easy. The tool should be easy to carry around and can be used in any vehicle.
The system should have the ability to load Geo Maps, Google Maps, tab/shape and network files etc. Any parameter can be exported in multiple formats like CSV, txt, tab, KML/KMZ etc. both as image and point.
Following is the list of QoS KPIs/ parameters which has to be provided in addition to other RF related KPIs/parameters for voice, SMS and Data services for all technologies (2G/3G/4G) by both Data Collection and Post Processing Tool: NGMS Licensed QoS KPIs
- (1) Network Downtime
- (2) Grade of Service
- (3) Service Accessibility
- (4) Call Connection Time
- (5) Call Completion Ratio
- (6) Mean Opinion Score (Average of A2B and B2A, POLQA/PESQ)
- (7) Inter System Handover
- (8) SMS Success Rate
- (9) SMS End-to-End Delivery Time
- (10) Data Throughput (3G & 4G)
- (11) Signal Strength (RSCP & RSRP)
- (12) RAB/E-RAB Setup Success Rate
- (13) Session Abnormal Release Rate,
Other QoS KPIs
- (1). Drop Call Rate (DCR)
- (2). Inter System Handover Success Rate (CS)
- (3) Inter System Handover Success Rate (PS)
- (4) Call Setup Success Rate (CSSR)
- (5) 2G Signal Strength RxLev (No of samples in customizable Bins)
- (6) 2G Signal Quality RxQual (No of samples in customizable Bins)
- (7) 3G Signal Strength RSCP (No of samples in customizable Bins)
- (8) 3G Signal Quality Ec/No (No of samples in customizable Bins)
- (9) 4G Signal Strength RSRP (No of samples in customizable Bins)
- (10) 4G Signal Quality RSRQ (No of samples in customizable Bins)
- (11). Number of Call Attempts
- (12) Number of Calls Established
- (13) Number of Calls Dropped
- (14) Number of Calls Blocked
- (15) Number of Soft Handover Failures
- (16) Number of Hard Handover Failures
- (17) Number of Intra RAT Handover Failures
- (18) Number of Inter RAT Handover Failure
- (19) Number of Intra Frequency Handover Failures
- (20) Number of Inter Frequency Handover Failures
- (21) Number of Total Handover Attempts
- (22) CSFB Call Setup Time/Call Connection Time
- (23) End-to-End SMS Delivery time
- (24) SMS e2e Delivery Success Rate
- (25) SMS Send Success Rate
- (26) SMS Receive Success Rate
- (27) SMS Send time
- (28) SMS attempts
- (29) Number of SMS attempts
- (30) Number of successful SMS
- (31) RLC Data Throughput (DL/UL)
- (32) MAC Throughput (DL/UL)
- (33) Application Throughput (DL/UL)
- (34) Modulation Scheme(DL/UL)
- (35) User Data Throughput (DL/UL)
- (36) CQI
- (37). SINR
- (38) Session Abnormal Release Rate
- (39) RAB Setup Success Rate
- (40) E-RAB Setup Success Rate
- (41) Round Trip Time (Latency)
- (42) Web Page Access Success Rate
- (43) Web Page Completion Success Rate
- (44) Web Page Download Time
- (45) Packet Loss
- (46) Jitter
- (47). FTP Throughput (DL/UL)
- (48) HTTP Throughput (DL/UL)
- (49) Streaming Service Non-Accessibility
- (50) Streaming Service Access Time
- (51) Streaming Reproduction Cut-off Ratio
- (52) Streaming Reproduction Start Failure Ratio
- (53) Streaming Reproduction Start Delay
- (54) Streaming Setup Success Rate
- (55) Streaming Completion Success Rate
- (56) Streaming Audio Quality
- (57) Streaming Video Quality
- (58) Streaming Audio/Video De-Synchronization
- (59) VoLTE Setup Success Rate
- (60) VoLTE Call Setup Time /Call Connection Time
- (61) VoLTE Call Completion Success Rate