A monitor for wireless internets that use BGP as their inter-AS Routing Protocol
Background and the AWMN model
In the AWMN and other community owned wireless internets network enthusiasts own, setup ,and maintain network and
RF equipment on a Best Effort Basis. Most wireless nodes are Autonomous Systems (AS) and the vast majority of inter-AS links are wireless.
Most of the networking and RF equipment are exposed to harsh environments. Equipment failures, configuration errors , equipment incompatibilities , and plain old pranks are much much more common than in professional ISP networks.
There is a Number and Name Authority aka Host-master per community but there is not one authority that has administrative access to all the networking equipment. Usually there are Node Databases but when they exist they are incomplete and contain a large amount of false information. Many wireless communities in Greece are interconnected through wireless and Internet connections.
There are not peering agreements between nodes and communities and there are not established protocols to deal with problems. Community spirit and good interpersonal relationships between the node operators are not always enough. The vast coverage areas and the large number of network nodes increase the complexity of monitoring.
This post discusses a Monitor for wireless internets that use BGP as their inter-AS Routing Protocol. The Monitor has access to the Node Database, BGP daemon tables, Routers, and Sensors across the internet. The Monitor alarms problems to Number and Name Authorities and Node Operators, draws a near real time connectivity graph and potentially automatically acts upon events to correct or contain problems.
A Database with IP Numbers, Node geographical coordinates (latitude,longitude,elevation), Node Operator Contact Information, DNS zones Link information and information for services provided by various nodes across the internet. The Number and Names Authorities aka Host-masters of the internet allocate and assign resources through the Node Database and most information is put there by the node operators. In the AWMN the Wireless Node Database WiND is used.
An Autonomous System as defined in RFC 4271 where the most inter-AS links are wireless and most links within the Autonomous System are wired.
A community member who acts as a node administrator to one or more nodes. He usually has one wireless node on the rooftop of his residence.
1 Border Gateway Protocol Monitor
Our internet monitoring system has access to many BGP Speakers across the internet. In a wireless not-professional, not-homogeneous internet as-paths change very often and it is very common to find wrong as-paths, ghost links and ghost prefixes in a BGP table. We have access to many BGP tables but we need a way to attach levels of certainty to the information they contain. Also in community based internets configuration mistakes and pranks such as IP hijacks are common. The following procedure attempts to deal with ghost links, ghost prefixes, configuration mistakes, and IP hijackings by using the information on BGP tables from across the internet.
We get the BGP tables from across the internet in short regular intervals eg: 30 minutes
We detect Prepends log them and filter them out from the input to the rest of the process
We detect BGP Communities log them and filter them out from the input to the rest of the process
Now every path shorter than 255 ASes that contains an Autonomous System many times indicates a Loop and it should be alarmed and excluded from the following process. RFC 4271 25.1
We split the table in pairs of AS numbers to get the links.
To attach a level of certainty to a pair we calculate a weight by adding a point on every hop starting from the table in which we found it.
eg: consider the following as-path on the AS 1
2 3 4 5 6
the weight w of the pair 1 2 is equal to 0+1=1 w(1,2)=1 and w(2,3)=3 the smaller the weight is the most probable is that the link exists When an AS number x has a w(x,y)=1 , meaning we have access to its table a pair x,z with w(x,z)=33 is invalid and it should be alarmed.
For every prefix in the tables we check if it is assigned by the Number Authority and if not we alarm the host-master.
For every prefix in the tables we check if it is assigned by the Number Authority to the AS advertising it and if not we alarm the host-master.
If we find an AS not assigned to anyone by the Number Authority we alarm the host-masters.
We detect prefixes announced multiple times and according to our Number Authority should not be Anycast If we find such prefixes we can figure out using information from the Number Authority who is cheating or messed up and alarm host-masters and operators.
If we see a path leading to a node-AS where we have access to a BGP table with a prefix not announced there, then this is a Ghost Prefix this is the easy part. To Guess Ghost Prefixes with paths leading to Autonomous Systems in which we do not have access we will use the weights to attach certainty.
For the prefix weight we use the size of the as-path.
Again, the smaller the weight, the higher the probability the prefix exists.
While we traverse the tables lower weights replace higher weights.
We are trying to figure out if the small prefixes announced are valid. (An easy way to do this would be to check against a list maintained by the Number Authority) Invalid small prefixes can create parallel internet spaces for large prefixes announced by ASes with one or a few not stable links.
Detect prefixes that should not pass the BGP filters agreed by the community.
2 Classic and Not So classic Nagios Stuff
We will need to complement the Monitor with Classic nagios ICMP echo stuff since BGP tables may quickly become nothing more than tables of forwarding intentions in a wireless internet.
We could make use of tracepath scripts to detect asymmetries.
Since this is a wireless internet we will need to monitor Connection Quality CCQ, signal/noise ratios,noise level,and signal strength. These are available through SNMP in many routers so adding them to a nagios like system is simple ( examples )
3 Smart Routers
Most wireless routers now days are harsh embedded systems that have plenty of CPU and RAM.
We may use scripts to add some brains and network healing capabilities to them.
eg: In the AWMN is common enough for a dish antenna to move by strong winds or a feeder fill with water after a storm. The link quality may drop drastically but the BGP session remains active. However, since the link quality is very poor many frames and IP packets drop, the ones using it suffer, and the quality of the internet drops. Another as long or even longer but higher in quality route could be used instead. A script may monitor the signal strength of each link and close the appropriate BGP session if the link quality drops drastically (an example script)
A smart router may prepepend its autonomous system to the AS-path as a first measure and close the BGP session as a last measure. The router could also act based on Connection Quality and other link quality metrics.
URI: A monitor for wireless internets that use BGP as their inter-AS Routing Protocol