clean up awstats referral spam

The 'Links from an external page (other web sites except search engines) ' section on the few awstats installations I get to look at has been chaotic the past few months. Tonnes of referral spam overshadow any useful information in the `links from an external page` section.

AWstats comes with a simple mechanism to block ( not include referral spam ) based on a black-list. To enable referral spam blocking set the SkipReferrersBlackList var-directive to the path of a file containing the spam domains you want to exclude.
However, this change is effective only for new updates.

I wrote a little script that can clean up awstats logs from referral spam using the same blacklist.txt file used by the `SkipReferrersBlackList`.

Usage instructions for the clean_awstats_rspam.pl program:

Get the clean_awstats_rspam.pl script

# wget kod.ipduh.com/lib/clean_awstats_rspam.pl

set $blacklist to the path of your blacklist.txt.

# vim clean_awstats_rspam.pl

You may change $bl_subdomain to 0 if you don't want to get rid off sub-domains of blacklisted domains. To print-log the count of referral spam links removed to the bottom of the awstatsYYMMMM.example.com.txt file set $rlog to 1.

The awstats logs files are usually in the awstats data directory and usually look like awstats062013.example.com.txt. The clean_awstats_rspam.pl script will read a log file and print to the standard out a `clean` log. You will have to direct the output to an intermediate log file, copy the intermediate log file to the original and then refresh your awstats page.

# cp clean_awstats_rspam.pl /var/www/sites/example.com/awstats/data
# cd /var/www/sites/example.com/awstats/data
# chmod 700 clean_awstats_rspam.pl
# ./clean_awstats_rspam.pl  awstats062013.example.com.txt >> clean_awstats062013.example.com.txt
# cp clean_awstats062013.example.com.txt awstats062013.example.com.txt

The clean_awstats_rspam.pl script:

 #!/usr/bin/perl
#g0 2013
#clean_awstats_rspam.pl
#clean up awstats referral spam
#http://alog.ipduh.com/2013/06/clean-up-awstats-referral-spam.html
# usage: 
# ./clean_awstats_rspam.pl awstats062013.example.com.txt >> clean_awstats062013.example.com.txt;
# cp clean_awstats062013.example.com.txt awstats062013.example.com.txt

use strict;

#set to 1 if you want to get rid off subdomains of blaclisted domains
my $bl_subdomains=1;
#location of your blacklist.txt file
my $blacklist="/var/www/sites/example.com/awstats/lib/blacklist.txt";
#set to 1 if you want to log the count of spam referral links removed to the awstatsMMYYYY.example.com.txt file
my $rlog=0;

my %spamdoms=();
my $logtxt=$ARGV[0];
my $foundspam=0; 
my $crap=0;
my $me="clean_awstats_rspam";

open FH, "$blacklist" or die "$me:I could not open $blacklist ($!)";
while (<FH>)
{
 chomp;
 if(/^#/) { next; }
 $_=~s/^\s+//;
 $_=~s/\s+$//;
 unless($spamdoms{$_})
 {
   $spamdoms{$_}=1;
 }  
}
close FH;

my @spamdoms=keys %spamdoms;
%spamdoms=();

open FH, "$logtxt" or die "$me:I could not open $logtxt ($!)";
while (<FH>)
{

 if(/^#/) { print $_ ; next; }
 unless(/^http/) { print $_ ; next; }
OOF:{
 $foundspam=0; 
 for my $spamdom (@spamdoms)
 {
    if( $bl_subdomains )
           {
  if( /^http:\/\/$spamdom/ || /^http:\/\/[a-z0-9A-Z\-\.]*\.$spamdom/ )
  {  
   $crap++;
   $foundspam=1; 
   last OOF;
  }
           }else{
    if( /^http:\/\/$spamdom/ )
  {
   $crap++;
                        $foundspam=1;
                        last OOF;
  }
           }
 }
 print $_;
    }
 
}
close FH;

print "$me:removed $crap referrals\n" if($rlog);




     

a script that cleans awstats referral spam

simple web statistics

A placeholder ... it' s nice and ready , written in bash .... have to put it here some time soon



Simple Web Statistics ...

centos add ipduh_intel simple awstats plugin

Notes on adding the ipduh_intel simple plugin to awstats on CentOS.

The system:

# cat /etc/issue; uname -r
CentOS release 5.8 (Final)
Kernel \r on an \m

2.6.18-308.20.1.el5

Install ipduh_intel

# cd /usr/local/awstats/wwwroot/cgi-bin/plugins/
# wget http://kod.ipduh.com/lib/ipduh_intel.pm
# chmod 755 ipduh_intel.pm
# cd /etc/awstats/ 

Add

LoadPlugin="ipduh_intel"

to every awstats.name.tld.conf configuration file in /etc/awstats.

Disable DNSLookup if enabled. Make sure that

DNSLookup=0

in every awstats.name.tld.conf configuration file in /etc/awstats.

PTR DNSLLookup is slow and useless if not hurtful.

Someone can easily put whatever.whatever , googlebot.google.com , or i.am.you.net in the PTR of an IP address.

The best persistent information of a visit ( the IP address ) is discarded when DNSLookup is used.

Also when DNSLookup is enabled the ipduh_intel plugin does not work well --it is a feature protecting you from silly features.



ipduh_intel simple --an awstats plugin-- centos

vim favorites

Vi Improved , Vim , is a just right programmer's editor.

This is not a good place to learn how to use or about the vim editor -- if you are starting with vim try here and here -- This is not even a good ,yet another, vim cheat-sheet.

It is just a list of stuff that I like about vim and I think that you should know if you are using vim to write programs.



On the table below I call:
the Normal Mode NMode:,
the Insert Mode IMode:,
the Command Mode : ,
the Visual Mode VMode.

For More on modes

:h vim-modes

#g0 vim favorites
>>	NMode: Moves current line one Tab to the right.
<<	NMode: Moves current line one Tab to the left.
J	NMode: Joins current line with the next line.
~	NMode: Flip the case of the current (under the cursor) letter
$	NMode: Jump to the last character of the line
0	NMode: Jump to the first column of the line
Ctrl+P	IMode: Keyword Completion - Previous Matching Words First
Ctrl+N	IMode: Keyword Completion - Next Matching Words First
Ctrl+x Ctrl+l	IMode: Line Completion - Complete with previous Matching Line
Ctrl+V	NMode: Enter Vertical Visual Mode. Select Vertical-Visual Block using the arrows
I	VMode: Insert Text Before the selected Visual Block
x	VMode: Delete the selected Visual Block
s	VMode: Replace the selected Visual Block with some other text
:u	: Undo
Ctrl+r	NMode: Undo Undo
.	NMode: Repeat Previous Command
:!	: See out of External Command entered after !
:r!	: Insert output of the external command entered after !
/	NMode: Search Forward - use n to repeat search forward and N to repeat search backward
?	NMode: Search Backward - use n to repeat search backward and N to repeat search forward
Ctrl+a	NMode: Increment by 1 the current (under the cursor) number
Ctrl+x	NMode: Decrement by 1 the current (under the cursor) number
:set cindent	: Enable Automatic Indentation ( good in default but configurable :h C-indenting )
:set ingnorecase	: Ignore Case in Following Searches
:set noingnorecase	: Do Case Sensitive Search in Following Searches
:set wildmenu	: Command Tab Completion - Show Possible Commands
:%s/old/new/g	: Search and Replace Globally
:%s/old/new/cg	: Search and Ask Before Replacing Globally
:s/old/new/g	VMode: Search and Replace In the Selected Area Globally
:set showmatch	: Higlight Matching { ( [
:set spell	: Enable Spell Checking
:set nospell	: Disable Spell Checking
Ctrl+x Ctrl+k	IMode: Dictionary Lookup of current word
:set number on	: Show Line Cumbers
:set nonumber	: Hide Line Cumbers
:setlocal spell spelllang=el	: Set Spell Checking Dictionary to el.utf-8.spl (Greek)
vim -d file1 file2 fileN	Command Line: Vim Diff files file1 file2 fileN
mn	NMode: Mark register n ( the xy position under the cursor )
`n	NMode: Jump to the register n marked earlier with mn
:sp	: Show file in two subwindows
:sp filename	: Open file filename in a subwindow
Ctrl+w Ctrl+w	NMode: Jump to other subwindow
Ctrl+wq	NMode: Quit a subwindow
y	VMode: yank - copy - Save selected text
d	VMode: delete selected text
>	VMode: move selected text to the right
<	VMode: move selected text to the left
~	VMode: flip case of selected text
p	NMode: paste - saved-yanked text after the cursor
:w filename	: Save file subwindow etc to filename

Also available in plain ascii

$ wget kod.ipduh.com/lib/g0-vi-cheatsheet.txt

url: vim favorites

Vim - Greek Spell Checking

Notes on setting up Greek Spell Checking on Vim

I have seen and used a few different ways on setting up Greek Spelling on Vim. This is an easy one

Get it

# wget http://ftp.vim.org/pub/vim/runtime/spell/el.utf-8.spl

Move it to the appropriate place --usually the following

# mv el.utf-8.spl /usr/share/vim/vim72/spell/

Use it

# vi test-greek-spelling
:setlocal spell spelllang=el

on some systems you just need to type

:setlocal spell spelllang=el

in Normal Mode as root and the downloading and installation of the Greek MySpell will happen auto-magically



Some related resources:
OpenOffice Greek Dictionaries
ftp.vim.org Greek Spell Checking Recipe
vim README_el.txt (use ISO-8859-7 to see the Greek Characters )
vim Myspell
a few ways to generate a spell
and of course ispell.math.upatras.gr





Vim Greek Spell Checking

sick ip address lookup

The last three days the ipduh ip address lookup was not working well for at least half of the ipduh users. Of course stuff like these happen when I am focused on other stuff, being away from computers, jabber clients , email agents , cellphones, etc ...

Why? Some of the daemons, written on early morning hours in loose style and prototype specs, figured out, after 408 days of running fine non-stop, that 16 GB of RAM are not enough and they want more ...

Everything is back in order now ( I hope ),
please let me know if you notice something weird ...
yes! an ip lookup taking more than 1 second to respond is weird!



ipduh address lookup UFO bug

2x2 MIMO Nv2 Measurements

I have been using 2x2 MIMO Nv2 on a few ~1km, ~2km, ~5km links for months and I always wanted to quantify my perceptions of how good I think it is.

This is a post with low-on-effort bandwidth and latency measurements on a 2x2 MIMO wireless link using 802.11 N and Nv2.

The link used is a point to point ~1km link in the 5GHz band used by the Athens Wireless Metropolitan Network (AWMN).


Later this week I will add a another wireless interface on the other side and use the same hardware on my side for one more point-to-point link. Then, I will repeat the same measurements.

We are trying to find out how good Nv2 really is.

The Nv2 wireless protocol is a proprietary Time Division Multiple Access ( TDMA ) protocol available on MikrotTik-routerOS systems with Atheros based cards. Ubiquity has a TDMA protocol called airMAX.

RouterOS systems come with a Bandwidth Test Server by default. However, I think that putting the Bandwidth Test Server and the Bandwidth Test Client on other systems `behind` the routers produces accurate results. Especially, when it comes to TCP bandwidth throughput testing using little busy routers.

80cm off-axis --offset-- satellite dishes and double polarization `awmn-nvac` type feeders are used on both sides. One of the routers is a PC and the other router is a MIPS machine (RB433GL). Unfortunately, at this moment the PC router is connected to the iperf server in his LAN using a Fast Ethernet 100Mb/s NIC. Hence, I have to use the Bandwidth Server and the Bandwidth Tester made by MikroTik. Gladly, the Standalone MikroTik Windows Bandwidth Test runs on Linux with Wine and the LAN bottleneck is on the PC router side. The MIPS machine has a Gigabit NIC and it is connected to a Gigabit LAN. The PC router is an x86 at 2673MHz and the MIPS router is a RB433GL at 680MHz.

For the UDP tests I will just use the Bandwidth Servers and Clients on the routers.

For the TCP tests I will use the Bandwidth Server on the PC router and a Bandwidth Client on a PC behind the MIPS router.

PCrouter <-air-> MIPSrouter <-GigabitLAN-> PC-BWclient

Nv2 N - One Client

Latency with Low Amounts of Traffic --no BW tests
From the AP to the Client

$ ping 10.21.241.67 -c 100
PING 10.21.241.67 (10.21.241.67) 56(84) bytes of data.
64 bytes from 10.21.241.67: icmp_req=1 ttl=63 time=2.27 ms
64 bytes from 10.21.241.67: icmp_req=2 ttl=63 time=6.70 ms
64 bytes from 10.21.241.67: icmp_req=3 ttl=63 time=6.27 ms
...
64 bytes from 10.21.241.67: icmp_req=100 ttl=63 time=2.67 ms

--- 10.21.241.67 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 99141ms
rtt min/avg/max/mdev = 1.610/5.470/9.517/2.111 ms

Latency with Large Amounts of UDP Traffic --while BW testing
From the AP to the Client

$ ping 10.21.241.67 -c 100
PING 10.21.241.67 (10.21.241.67) 56(84) bytes of data.
64 bytes from 10.21.241.67: icmp_req=1 ttl=63 time=16.6 ms
64 bytes from 10.21.241.67: icmp_req=2 ttl=63 time=9.52 ms
64 bytes from 10.21.241.67: icmp_req=3 ttl=63 time=9.02 ms
64 bytes from 10.21.241.67: icmp_req=4 ttl=63 time=5.60 ms
...
64 bytes from 10.21.241.67: icmp_req=100 ttl=63 time=22.2 ms

--- 10.21.241.67 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 99148ms
rtt min/avg/max/mdev = 4.754/10.402/37.854/4.615 ms

while testing UDP bandwidth


Latency with Low Amounts of Traffic --no BW tests
From the Client to the AP.

$ ping 10.27.224.237 -c 100
PING 10.27.224.237 (10.27.224.237) 56(84) bytes of data.
64 bytes from 10.27.224.237: icmp_seq=1 ttl=63 time=3.42 ms
64 bytes from 10.27.224.237: icmp_seq=2 ttl=63 time=3.06 ms
64 bytes from 10.27.224.237: icmp_seq=3 ttl=63 time=7.85 ms
...
64 bytes from 10.27.224.237: icmp_seq=100 ttl=63 time=5.79 ms

--- 10.27.224.237 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 99116ms
rtt min/avg/max/mdev = 1.617/4.341/15.546/2.508 ms

Latency with Large Amount of UDP traffic
From the Client to the AP

$ ping 10.27.224.237 -c 100
PING 10.27.224.237 (10.27.224.237) 56(84) bytes of data.
64 bytes from 10.27.224.237: icmp_seq=1 ttl=63 time=5.73 ms
64 bytes from 10.27.224.237: icmp_seq=2 ttl=63 time=2.75 ms
64 bytes from 10.27.224.237: icmp_seq=3 ttl=63 time=6.67 ms
64 bytes from 10.27.224.237: icmp_seq=4 ttl=63 time=10.9 ms
...
64 bytes from 10.27.224.237: icmp_seq=100 ttl=63 time=8.68 ms

--- 10.27.224.237 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 99070ms
rtt min/avg/max/mdev = 2.752/7.739/24.250/3.273 ms

while running a UDP bandwidth test

> /tool bandwidth-test protocol=udp direction=transmit address=10.27.224.237 
                status: running
              duration: 2m27s
            tx-current: 205.3Mbps
  tx-10-second-average: 189.3Mbps
      tx-total-average: 184.0Mbps
           random-data: no
             direction: transmit
               tx-size: 1500
-- [Q quit|D dump|C-z pause]

Ping while testing TCP bandwidth throughput.
From the client to the AP.

$ ping 10.27.224.237 -c 100
PING 10.27.224.237 (10.27.224.237) 56(84) bytes of data.
64 bytes from 10.27.224.237: icmp_seq=100 ttl=63 time=3.83 ms
...
64 bytes from 10.27.224.237: icmp_seq=100 ttl=63 time=4.02 ms

--- 10.27.224.237 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 99108ms
rtt min/avg/max/mdev = 2.726/5.791/21.109/3.734 ms

802.11 N One Client

Latency with `low` amounts of traffic --no BW testing
Client to the AP

$ ping 10.27.224.237 -c 100
PING 10.27.224.237 (10.27.224.237) 56(84) bytes of data.
64 bytes from 10.27.224.237: icmp_seq=1 ttl=63 time=0.934 ms
64 bytes from 10.27.224.237: icmp_seq=2 ttl=63 time=0.664 ms
64 bytes from 10.27.224.237: icmp_seq=3 ttl=63 time=0.572 ms
...
64 bytes from 10.27.224.237: icmp_seq=100 ttl=63 time=14.0 ms

--- 10.27.224.237 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 99029ms
rtt min/avg/max/mdev = 0.486/1.043/15.114/1.977 ms

Latency with large amounts of UDP traffic
Client to the AP

$ ping 10.27.224.237 -c 100
PING 10.27.224.237 (10.27.224.237) 56(84) bytes of data.
64 bytes from 10.27.224.237: icmp_seq=1 ttl=63 time=12.4 ms
64 bytes from 10.27.224.237: icmp_seq=2 ttl=63 time=9.88 ms
64 bytes from 10.27.224.237: icmp_seq=3 ttl=63 time=7.68 ms
64 bytes from 10.27.224.237: icmp_seq=5 ttl=63 time=89.8 ms
...
64 bytes from 10.27.224.237: icmp_seq=100 ttl=63 time=45.4 ms

--- 10.27.224.237 ping statistics ---
100 packets transmitted, 89 received, 11% packet loss, time 99089ms
rtt min/avg/max/mdev = 7.688/42.859/141.709/16.173 ms

while testing UDP bandwidth throughput

 /tool bandwidth-test protocol=udp direction=receive address=10.27.224.237   
                status: running
              duration: 2m51s
            rx-current: 207.5Mbps
  rx-10-second-average: 200.0Mbps
      rx-total-average: 108.7Mbps
          lost-packets: 2308
           random-data: no
             direction: receive
               rx-size: 1500

With one client to the AP Nv2 increases the latency on the link when there is not traffic but behaves well --in terms of latency-- when we stress the link using a bandwidth tester.

Hmm, Nv2 seems better already ... more tests by me , hence suffering ( sorry ) for the West and South West Athens Neighborhoods are coming.

References:
MikroTik Nv2



2x2 MIMO nv2 measurements

AWMN Anycast DNS

Notes on setting a slave root awmn. Anycast IPv4 DNS server

I am using a KVM that has redundant Internet Connectivity, redundant AWMN Connectivity and does both the AWMN routing and the Hybrid Internet-AWMN DNS resolving. This way, if the KVM ceases to exist in the AWMN so does the 10.0.0.1/32 route to it.

# apt-get install quagga bind9

Configure Bind

set /etc/bind/named.conf

# vi /etc/bind/named.conf
# cat /etc/bind/named.conf

include "/etc/bind/named.conf.options";
include "/etc/bind/named.conf.local";

set /etc/bind/named.conf.options

# cat /etc/bind/named.conf.options 
options {
 directory "/var/cache/bind";
 allow-recursion { any; };
        allow-query { any; };
 version "slave awmn.";
 auth-nxdomain no;    # conform to RFC1035
 listen-on { 127.0.0.1; 10.21.241.100; 10.0.0.1;  };
 listen-on-v6 { any; };
};

set /etc/bind/named.conf.local

# cat /etc/bind/named.conf.local 

zone "awmn" IN {
 type slave;
 file "/var/cache/bind/db.awmn";
 masters{ 10.19.143.12; };
};

zone "10.in-addr.arpa" IN {
 type slave;
 file "/var/cache/bind/db.10.in-addr.arpa";
 masters { 10.19.143.12; };
};

zone "." {
        type hint;
        file "/etc/bind/db.root";
};

use this by the system name resolver

# echo "nameserver 127.0.0.1" > /etc/resolv.conf

Restart Bind

# /etc/init.d/bind9 restart

Test

# dig +short ipduh.com
85.25.242.245
# dig +short ipduh.awmn
10.21.241.4
# dig +short www.awmn
srv1.awmn.
10.19.143.13

Add 10.0.0.1 to the loopback interface.
Adjust the lo stanza in /etc/network/interface

# The loopback network interface
auto lo
iface lo inet loopback
ip addr add 10.0.0.1 dev lo

Configure Quagga

set zebra, bgpd , and ospfd to yes in /etc/quagga/daemons

# grep "=yes" /etc/quagga/daemons 
zebra=yes
bgpd=yes
ospfd=yes

Copy to /etc/quagga/ some skeleton files

# cp /usr/share/doc/quagga/examples/bgpd.conf.sample /etc/quagga/bgpd.conf
# cp /usr/share/doc/quagga/examples/ospfd.conf.sample /etc/quagga/ospfd.conf
# cp /usr/share/doc/quagga/examples/zebra.conf.sample /etc/quagga/zebra.conf

Configure Routing

Zebra example configuration file

# cat /etc/quagga/zebra.conf 
! -*- zebra -*-
!
! zebra configuration file
!
hostname anydns
password kodikos1
enable password kodikos2
!
ip route 10.0.0.1/32 10.21.241.100
ip route 0.0.0.0/0 192.168.30.1
!

log file /var/log/quagga/zebra.log

BGP example configuration file

hostname anydns

log file /var/log/quagga/bgpd.log
log monitor
log stdout
log syslog

password kodikos1
enable password kodikos2

router bgp 20305
 bgp router-id 10.21.241.100
 network 10.0.0.1/32

! ipduh002011
 neighbor 10.21.241.69 remote-as 20305
 neighbor 10.21.241.69 description ipduh AWMN BGP Feed
 neighbor 10.21.241.69 timers 10 30
 neighbor 10.21.241.69 capability dynamic
 neighbor 10.21.241.69 capability orf prefix-list both
 neighbor 10.21.241.69 soft-reconfiguration inbound

! neighbor 10.21.241.69 prefix-list awmn-bgp in
! neighbor 10.21.241.69 filter-list maxaslength out

! ipduh04711 
 neighbor 10.21.241.68 remote-as 20305
 neighbor 10.21.241.68 description ipduh AWMN BGP Feed
 neighbor 10.21.241.68 timers 10 30
 neighbor 10.21.241.68 capability dynamic
 neighbor 10.21.241.68 capability orf prefix-list both
 neighbor 10.21.241.68 soft-reconfiguration inbound

! neighbor 10.21.241.68 prefix-list awmn-bgp in
! neighbor 10.21.241.68 filter-list maxaslength out

! ipduh01433
 neighbor 10.21.241.67 remote-as 20305
 neighbor 10.21.241.67 description ipduh AWMN BGP Feed
 neighbor 10.21.241.67 timers 10 30
 neighbor 10.21.241.67 capability dynamic
 neighbor 10.21.241.67 capability orf prefix-list both
 neighbor 10.21.241.67 soft-reconfiguration inbound

!
!

! ipduh03711
 neighbor 10.21.241.66 remote-as 20305
 neighbor 10.21.241.66 description ipduh AWMN BGP Feed
 neighbor 10.21.241.66 timers 10 30
 neighbor 10.21.241.66 capability dynamic
 neighbor 10.21.241.66 capability orf prefix-list both
 neighbor 10.21.241.66 soft-reconfiguration inbound

!
!

! ares
 neighbor 10.21.241.126 remote-as 20305
 neighbor 10.21.241.126 description ipduh AWMN BGP Feed
 neighbor 10.21.241.126 timers 10 30
 neighbor 10.21.241.126 capability dynamic
 neighbor 10.21.241.126 capability orf prefix-list both
 neighbor 10.21.241.126 soft-reconfiguration inbound


!need to update these filters ... 
!not important, this is an inner router in full mess with the border routers
!ip prefix-list awmn-bgp seq 5 permit 10.0.0.0/8 ge 9 le 24
!ip prefix-list awmn-bgp seq 10 permit 10.0.0.0/15 le 32
!ip prefix-list awmn-bgp seq 15 deny any
!
!!ip as-path access-list maxaslength deny ( [0-9]+){250}$
ip as-path access-list maxaslength permit .*
!
line vty
!
end

OSPF configuration

# cat /etc/quagga/ospfd.conf 
! -*- ospf -*-
!
! OSPFd configuration file
!
!
hostname anydns
password kodikos1
enable password kodikos2
!
router ospf
   network 10.21.241.0/25 area 0
ospf router-id 10.21.241.100
!
log stdout

Cool now add this BGP peer to your other routers and get rid off any 0/0 route in configuration files other than the zebra.
Obviously you do not need any 10/8 route any more.

Firewall

# iptables -t filter -A INPUT -p udp -d 10.0.0.1/32 --dport domain -j ACCEPT
# iptables -t filter -A INPUT -p tcp -d 10.0.0.1/32 -j REJECT --reject-with tcp-reset
# iptables -t filter -A INPUT -d 10.0.0.1/32 -j REJECT

Make it stick

# iptables-save > /etc/rules.iptables
# vi /etc/network/if-pre-up.d/ipv4init.sh
# cat /etc/network/if-pre-up.d/ipv4init.sh 
#!/bin/sh
/sbin/iptables-restore < /etc/rules.iptables

Test, test again, and then test some more ...







AWMN Anycasta DNS How-To

OpenVPN on wireless networks

Notes on setting simple static key OpenVPN tunnels that work over wireless networks.

In the server

# apt-get install openvpn
# cd /etc/openvpn
# openvpn --genkey --secret static.key
# vi /etc/openvpn/tun0.conf
# cat /etc/openvpn/tun0.conf 
dev tun0
proto tcp-server
ifconfig 192.168.50.1 192.168.50.2
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
secret /etc/openvpn/static.key

In the client

# apt-get install openvpn
# cd /etc/openvpn
# scp root@10.13.157.13:/etc/openvpn/static.key .
# vi /etc/openvpn/tun0.conf
# cat /etc/openvpn/tun0.conf 
remote 10.13.157.13
dev tun0
proto tcp-client
ifconfig 192.168.50.2 192.168.50.1
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
secret /etc/openvpn/static.key

Assuming that the OpenVPN server is at 10.13.157.13

Testing the tunnel

You may find it useful to run the following commands in a screen or another ssh session ( in case you are allergic to screen ).

In the server

# openvpn --config /etc/openvpn/tun0.conf --verb 6

In the client

# openvpn --config /etc/openvpn/tun0.conf --verb 6

still in the client

# ping 192.168.50.1
PING 192.168.50.1 (192.168.50.1) 56(84) bytes of data.
64 bytes from 192.168.50.1: icmp_req=1 ttl=64 time=2.72 ms
64 bytes from 192.168.50.1: icmp_req=2 ttl=64 time=2.94 ms
^C

Forward IP traffic through the OpenVPN tunnel

In the server

# echo net.ipv4.ip_forward=1 >> /etc/sysctl.conf
# sysctl -p
# iptables -t nat -I POSTROUTING -s 192.168.50.0/24 -o eth0 -j MASQUERADE

In the client

# route del default gw 192.168.48.1
# route add default gw 192.168.50.1
# ping ipduh.com
PING ipduh.com (85.25.242.245) 56(84) bytes of data.
64 bytes from archimedes.ipduh.com (85.25.242.245): icmp_req=1 ttl=55 time=74.7 ms
^C

where 192.168.48.1 is the previous LAN default gateway

Make it stick

In the server

# vi /etc/network/if-pre-up.d/ipv4init.sh
# cat /etc/network/if-pre-up.d/ipv4init.sh 
#!/bin/sh
/sbin/iptables-restore < /etc/rules.iptables

# iptables -A FORWARD -i eth0 -o tun0 -m state --state ESTABLISHED,RELATED -j ACCEPT
# iptables -A FORWARD -s 192.168.50.0/24 -o eth0 -j ACCEPT
# iptables -t nat -I POSTROUTING -s 192.168.50.0/24 -o eth0 -j MASQUERADE
# iptables-save > /etc/rules.iptables

Both systems

Set the AUTOSTART parameter in /etc/default/openvpn ... "all" works
/etc/default/openvpn is the configuration file for /etc/init.d/openvpn

# /etc/init.d/openvpn start

In the client

Add a route to the OpenVPN server(s) in /etc/network/interface eg:

up route add -net 10.0.0.0 netmask 255.0.0.0 gw 10.21.241.88

and a script that starts the vpn tun and adds the default route in /etc/network/if-up.d/

# cat /etc/network/if-up.d/routes.sh 
#!/bin/sh
/etc/init.d/openvpn start 
#there is also another script in if-up.d that should start openvpn 

/sbin/route add default gw 192.168.50.1

To test restart networking

# /etc/init.d/networking restart
# ping ipduh.com
PING ipduh.com (85.25.242.245) 56(84) bytes of data.
64 bytes from archimedes.ipduh.com (85.25.242.245): icmp_req=1 ttl=55 time=73.4 ms
^C

that's all folks!



OpenVPN ... simple setup that works on wireless community networks ...

ads usb to ethernet adapter linux - debian

Notes on setting up the

# lsusb |grep -i ads
Bus 004 Device 009: ID 06e1:0008 ADS Technologies, Inc. UBS-10BT Ethernet [klsi]

ethernet adapter on Debian.

The ADS usb to ethernet adapter works with the kaweth driver. The kaweth driver is contained in the firmware-linux-nonfree debian package. If you want to apt install it add 'non-free' and 'contrib' to the repositories in /etc/apt/sources.list and

# apt-get update
# apt-get install firmware-linux-nonfree

Plug in the usb-to-ethernet adapter and look in messages for the name of the ethernet interface created

# dmesg |grep kaweth |grep created
[1613044.197148] kaweth 4-2:1.0: kaweth interface created at eth1

Set up IP and test

# ifconfig eth1 192.168.88.88 netmask 255.255.255.0
# ifconfig

URI:http://alog.ipduh.com/2013/06/ads-usb-to-ethernet-adapter-linux-debian.html



ADS USB to Ethernet adapter on debian Linux

apache reverse proxy

Apache2 Reverse Proxy on Debian Notes

Enable the apache modules

#cd /etc/apache2/mods-available/
#a2enmod proxy
#a2enmod proxy_http

An example Virtual Host

<VirtualHost 10.21.241.4:80>
 ServerAdmin spamme@ipduh.com
 ServerName  ipduh.ipduh.awmn
 
 ProxyRequests off
 
 <Proxy *>
 Order deny,allow
 Allow from all
 </Proxy>

 ProxyPass / http://ipduh.com/
 ProxyPassReverse / http://ipduh.com/
 
</VirtualHost>

Apache Reverse Proxy