As a telecommunications engineer the wow-factor of the LoRaWAN™ technology came just after powering up a new gateway. I was watching my phone - which was monitoring a number of LoRaWAN devices in an application group - and the first device that appeared I recognised as a sensor inside a server rack, inside a building 2km away. I checked and yes, this sensor had connected via the gateway I had just turned on. While I was fully aware of the range of the technology, having walked miles mapping the LoRa® reception, the idea of a low power RF transmitter buried inside a metal cabinet in an invisible building reaching the rooftop where I was standing was just amazing.
This white paper documents some of the experiments that were carried out when initially setting up a LoRaWAN network in Glasgow. Prior to setting up the network some experiments were conducted measuring the range of the LoRa technology in a built up area.
The initial network involved three Kerlink gateways mounted on the roofs of three buildings in Glasgow. For the testing related to the linked map, unfortunately one of the gateways had been removed to a location 60 feet lower due to roof work. This has affected the coverage but doesn't detract from the purpose of the experiment.
In operation the LoRaWAN protocol creates a network similar in many ways to a cellular mobile network. For the most typical mode of operation and in simplistic terms, the LoRaWAN network listens for authorized devices that either want to join the network or have already joined the network. In this context 'joining the network' means a device authorized on this network contacts the network, identifies itself and then partakes in a key exchange. Once the key exchange has occurred the device can transmit data via the network. The actual Radio Frequency (RF) listening is done by LoRaWAN Gateways. The Gateways hear all LoRaWAN traffic and regardless of the source, any valid LoRaWAN packet received by a Gateway is forwarded to the Network Server. The Network Server decides if the device belongs to this network and if it does the packet is processed further. When a device joins a network the Network Server can give a list of channels to use and in this way the spectrum in use around different gateways can be managed by the Network Server.
If a LoRaWAN end point or mote transmits a message that requires an acknowledgement (ACK) the Network Server looks at metadata associated with that message and if the message was received from multiple gateways the ACK is returned via the 'nearest' gateway where 'nearest' is the gateway with the best Received Signal Strength Indication (RSSI) for that message. Transmitting messages to a mote depends on the type of the mote, messages for the mote might be sent within an ACK packet immediately after a mote transmission ("Class A" operation) or sent in a scheduled timeslot ("Class B" operation) or sent anytime ("Class C" operation).
The different classes (A, B, & C) and their modes of operation enable the developer of the sensor or actuator device to tailor the operation to the power and application requirements.There are also modulation parameters that can be adjusted, for example adjusting the "Spreading Factor" (SF) trades off transmission speed for improved receiver sensitivity. A higher SF should mean a better chance of packet delivery in a border line situation, it does mean a much higher energy cost as the transmitter must be on for considerably longer.
Starting from zero knowledge about LoRa and LoRaWAN there were numerous parameters that were considered for measurement. From the claims made about the LoRa technology the range was the first consideration with data reliability the second. From there various other tests were developed including a mapping of the networking performance.
Some initial range testing was conducted and some of the results reported here.
Individual gateways were connected to a LoRaWAN network and fixed location motes configured to repeatedly transmit via a specific gateway using a range of spreading factors (SF). Individual link performance was measured. Multiple gateways were then connected to the network and the mote to network data delivery performance measured.
GPS-tracked motes were used to transmit to the network and the combination of GPS location and gateway reception used to identify coverage of the network and individual gateways.
All the testing used European frequencies (using the '868MHz' band) with 125kHz channel bandwidths using the default coding rate. Only the three default channels were used. All SF combinations were used.
The reliability testing used two distant non-line-of-sight gateways (1.9km and 2.1km) both using the mobile network for the back-haul connection. The mobile network connections used two different network providers.
Without a comparative measure it was unknown if the initial results were good or bad. From the test site to the gateway at Glasgow Caledonian University (GCU, 1.9km) the success in getting a packet to the gateway was about 47% (all spreading factors) (Fig.2) while the success in connecting to the gateway and receiving an ACK was about 38% (Fig.3). Similar results were observed when connecting to the gateway at Strathclyde University (SU, 2.1km).
After analyzing the connection by time of day it became obvious that the connection rates at night were extremely poor (Fig.4). Counting the number of transmitted but not received packets it was found that there were periods exceeding an hour where no packets were received and periods in the night covering several hours where very few packets were received.
The night time packet loss was observed affecting both SU and GCU gateways and the network server logs showed no packets received by these gateways during the periods in question. The issue could possibly be attributed to a number of factors:
(1) seemed the most likely option. (2) didn't seem likely as latency might stop the ACK reaching the mote but the transaction should still have been recorded in the network server and it wasn't, (3) didn't seem as likely since it appeared to concurrently affect two gateways on two different carriers at two geographic locations (4) was considered but the network server was receiving packets from other gateways during the periods in question.
In an attempt to measure RF noise two spectrum analysers were run (GQRX software defined radio and a TTi portable spectrum analyser) with peak hold enabled. This would leave a trace of the highest level signal received in the section of spectrum being monitored. This test was really an inconclusive test since the ability of the LoRa receivers to select out the signal of interest operates down as far as -130dBm but the narrowest filters used on the spectrum analyzers gives a noise floor of about -75dBm. After 24 hours of operation the worst case noise floor recorded by the spectrum analyzers was around -70dBm.
To measure latency between the gateways and the networks, Internet Control Message Protocol (ICMP) pinging was set up with a one second interval and the ping results logged. The ICMP ping was run continuously (Network Server Host to Gateways) and during the period of operation the LoRaWAN packet loss dropped from greater than 50% to less than 5%.
This result suggested that the Kerlink gateways were losing connection to the network and not reconnecting automatically or not connecting in time to prevent the loss of the User Datagram Protocol (UDP) message to the Network Server. With ICMP ping running the internet connection was kept alive and the UDP message delivery rate improved.
When plotting ping latency and ping loss against LoRaWAN packet loss there did not appear to be any correlation.
Experimenting with the ICMP ping interval, increasing ping intervals to 10 seconds and higher increased LoRaWAN packet loss. From these experiments an interval as short as 5 seconds appeared to be necessary to keep the network connection between the Gateway and the Network Server alive. This indicated that the network carriers were dropping connections that were inactive for periods as short as 6 seconds.
Interim test results that were generated during 'ping tuning' gave network connection results (mote to gateway) of 98% and full two way connection at 95.5% (Fig.5). This data was generated from 6000 transmitted packets using both SF9 and SF10 with a ping interval of one second.
The following figures (6 & 7) graph the mote to gateway connection percentages based on channel frequency and spreading factor. These results identified two unexpected results, the first was the distinctive difference in results based on frequency. These channels were only separated by 200kHz but there was a significantly different result for the 868.1MHz channel. While the GCU results had better results for the higher spreading factors (as designed) the SU gateway had better results at lower spreading factors.
Further analysis was conducted, for example, plotting the time series Signal-to-Noise Ratio (SNR) for both gateways and distinctive differences were observed. It appeared that there was some noise source close to the SU gateway and it was theorized that it was bursty in nature causing disruption to the longer packets that occur at higher spreading factors.Occasionally high level, wide band noise was observed in the 868MHz band. The source of this signal was never identified.
The network testing was initially an ad-hoc experiment to see how the different gateways covered the different localities withing the Glasgow CBD. For these tests a single or multiple LoRaWAN devices were carried through the CBD streets in a backpack, at the same time the location of the devices were recorded by GPS. The GPS data was merged with the Network Server data to generate a map where each received transmission could be visually linked to the gateway that received the transmission. There were three Kerlink gateways used in this test, the two gateways used in previous testing and a gateway located in a commercial building 'Skypark'. The third gateway had initially been located on the roof but due to roof work, during these tests the gateway was located 15-20 metres below the original height (this limited the coverage from this gateway).
In the map extract above, the markers represent the position when a transmission occurred and the lines link back to the gateway or gateways that received that transmission. Clicking on a marker indicates the transmission parameters (SF7-12), the receiving gateway and the RSSI for that gateway for that message. If multiple transmissions occurred from the same spot these will appear in the pop-up marker message.
Since it was known that the test was non-optimal due to the third gateway location, the testing has been used to evaluate the test procedure for a later more complete test. Therefore, some tests are just an SF12 device, or a single device alternating between different SF values. Some walk-throughs include multiple devices operating concurrently and independently.
View the full map here. Different levels of plot display can be created by selecting the appropriate check boxes at the top of the screen. For example, it is possible to only show connections related to a single gateway for a single SF.
While there were a few hiccups in the initial testing and despite the temporary relocation of a gateway the tests performed were highly encouraging. The data packet transfer success rate was good and would require minimal resending of data. The biggest issue for the successful reception of data was the problem with the mobile network back-haul connection and this could be ameliorated by pinging the gateway modem.
The initial tests conducted for network mapping made it obvious that the mapping needed multiple units transmitting to capture sufficient data. Because SF12 packets have such a long transmit period the duty-cycle limitations for this Industrial, Scientific and Medical (ISM) radio band makes the transmit repetition rate too slow for useful mapping of SF12 data. This is less of an issue for mapping since the reason for using SF12 is to exploit the additional receiver sensitivity available for this modulation parameter. Mapping of SF12 data would only really be required once the reception started becoming problematic at the lower spreading factors.
A minor issue with the mapping is that the GPS locations can get erratic and a walk down a CBD street is recorded as a wildly oscillating path, also, because GPS doesn't respond well to being in an underpass it isn't obvious from the map but coverage included a 100 metre underpass beneath the Glasgow Central Station (Argyle St). The markers for the underpass are displayed either side of the underpass, depending on the where the GPS imagined itself at the time.
The LoRaWAN technology appears to be well developed technology capable of operating as a wide area network for wireless sensor devices. The implementation process is not particularly complex. Gateways can be connected via cabled connections or, where the complexity of connecting the gateway to the fixed network is too high (for example, in a hospital or university) then connecting via a mobile network connection is a fast-to-implement solution. The sensors and gateways have a long reach considering their low power and only a few gateways are required to cover a city CBD.
Dr Andrew Wixted,
Glasgow Caledonian University
This work funded by Stream Technologies and Innovate UK as a Knowledge Transfer Partnership with Glasgow Caledonian University