ipduh search





i. Sum of all decline

ii. Lately,

iii. apropos

iiii. Torrent Search

v. Shops Search

vi. AWMN Search

viii. RFC Search

ix. Minor Cyberbrain Sclerosis

x. Anonymity Check













































i. Sum of all decline


The ipduh site has been on the autopilot for years.

The increasing politeness of the big crawling bots
and the decline in traffic from Google
matched and fed the decline of the ipduh infrastructure.

At it's peak the ipduh site was running on dozens of VMs on seven physical servers
on four different internets and the AWMN.

Now, a reduced infrastructure runs a degraded service.











ii. Lately,

I fixed a few glitches and restored some of the broken services.











iii. apropos

Now, most of apropos works.

Most errors return five random examples,
in case you need to fuzz ipduh or learn how it works.

Exempli gratia

I meant first five random usage examples :)



five examples that I got around 1546476964
instagram.com
google.co.jp
qq.com
google.com.br
t.co



(btw, more about apropos)























v. Torrent Search

The bittorrent Search



examples:
!t debian
!t slackware












vi. Shops Search

The Online Shops Search examples:
!shops SDR
!shops drones
!shops fixed wing drones
!shops fixed wing aircraft
!shops bath bombs
!shops parachute
!shops helicopter
!shops hovercraft
!shops aircraft
!shops 3D printer
!shops android
!shops camera
!shops robot
!shops robotic arm












vii. AWMN Search

The Athens Wireless Metropolitan Network, AWMN Search


examples:
!awmn ipduh
!awmn MiCkOf
!awmn node
!awmn tor
!awmn voip
!awmn bgp
!awmn wifi












viii. RFC Search

The Internet Engineering Task Force Request for Comments, RFC Search

examples:
!rfc GPRS
!rfc DNS
!rfc VoIP
!rfc BGP












ix. Minor Cyberbrain Sclerosis


ipduh.com/el, which is a "some of ipduh in Greek" version of ipduh
does not work.


Now, that the minor cyberbrain sclerosis query
shows ipduh.com/el first on Google :)
it's time to restore ipduh.com/el.
(
btw, ipduh.com/el is restored
https://ipduh.com/el/apropos/?acropolis
https://ipduh.com/el/apropos/?ギャンブル.com
https://ipduh.com/el/apropos/?yahoo.de
The old el home page is at ipduh.com/el/old, the /el runs another experiment now.
Hopefully, some time soon I will have a nice mobile friendly version for /el.
).



But nothing is forever,
the following picture shows ipduh first on the Google orgarnic results for "Minor Cyberbrain Sclerosis"







Usually, when one googles ipduh from Greece with
https://www.google.com/search?q=ipduh
gets ipduh.com/el first in his organic results.
If you are located in Greece, please try and report.


ipduh.com/el is a descent resource given its size and the links to it.
Taking it down in order to achieve
the minor cyberbrain sclerosis promotion
was a "shoot your foot" SEO tactic.

Since, all of us leave in some Google bubble,
I wonder for how many of you does the minor cyberbrain sclerosis google query returns ipduh.com/el first in the organic results.





The following picture shows how a google query for ipduh shows in a friends' Google bubble somewhere in Greece.































x. Anonymity Check



Nothing is forever, so let's take some pictures.

The anonymity checker
in various caches and through various proxies across the web.


the anonymity checker in the google cache.

A picture of http://webcache.googleusercontent.com/search?q=cache:http://ipduh.com/anonymity-check






the anonymity checker through the baidu translate service



the anonymity checker through the bing translate service







through some sogu service



and through google traslate











































to be continued
I haven't really worked on IPduh for years.



The original Google powered search of IPduh was queried for everything that apropos was unable to solve. That broke a while ago, but recently I put together a ten minute work around. I may be able to put it together like it was using the new Google Search API, but I am working on other things now and I am not going to have the time to play with it anytime soon.

For what it's worth, the original Google powered search would sieve on IPduh, Security Sites and Lists, RFCs, GNU/Linux sites and lists, Finance sites, BSD sites and lists, and others. Some pictures showing it working in action are following.























The Torrent search was pretty funny too, I google-dorked on it until my little nephew was able to find her favorite kids movies with Greek subtitles on her own. Unfortunately, I do not have a picture showing the "Subtitles" button.



Memento mori buddies.

tankos code







Remember tankos?



I wrote Tankos in C and I used the ATMEL Studio to put it together,
the code is on github ...





Hardware:
  • one ATmega328 Arduino ( UNO ) board
  • one SR04 ultrasonic range detector
  • tamiya tracks ( got them from a bulldozer toy )
  • one tamiya twin-motor gearbox that I geared to go slow with a lot of torque
  • one small tower pro servo
  • fuses, buttons , switches , breadboard cables , a tiny breadbord, 4 18650s, a twin 18650 case that connects them in series, two single 18650 cases , dc jacks , H bridges , plywood , screws , random pieces of clear plastic, etc




if you want to recreate tankos and you need help in choosing hardware or find your way around the code please contact me .







tankos code



notes on moving xen guests to KVM

Set up the KVM guest in the KVM host
virt-install -n xxenguest -r 4096 --vcpus=2 --os-type=linux --disk /home/vm/xxenguest/xxenguest-copy.img --network=bridge:br0 --hvm --vnc --noautoconsole --import








Switch to rtl8139 Ethernet controller module in /etc/modprobe.conf in the KVM guest
# cat /etc/modprobe.conf
#alias scsi_hostadapter ata_piix
alias scsi_hostadapter xen-vbd
alias sym53c8xx off
alias net-pf-10 off
alias ipv6 off
options ipv6 disable=1
alias eth0 8139cp
#alias eth0 xen-vnif







notes on moving xen guests to KVM

upgrade to debian 8

Upgrade wheezy ( debian 7 ) to jessie ( debian 8 )

adjust /etc/apt/sources.list to something like this
deb http://ftp.gr.debian.org/debian/ jessie main non-free contrib
deb-src http://ftp.gr.debian.org/debian/ jessie main non-free contrib

deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free

# jessie-updates, previously known as 'volatile'
deb http://ftp.gr.debian.org/debian/ jessie-updates main contrib non-free
deb-src http://ftp.gr.debian.org/debian/ jessie-updates main contrib non-free

Most likely, NTUA 's debian mirror in athens is not your closest mirror.
You may want to use another debian mirror.

upon adjusting /etc/apt/sources.list update apt index
# apt-get update


and upgrade all the packages and the base system
# apt-get upgrade

or
# apt-get dist-upgrade


You will have to quit a few informative pagers and answer a few questions, but everything works out fine.





upgrade to debian 8



netem notes

tc qdisc netem notes

Usage: ... netem [ limit PACKETS ]
                 [ delay TIME [ JITTER [CORRELATION]]]
                 [ distribution {uniform|normal|pareto|paretonormal} ]
                 [ corrupt PERCENT [CORRELATION]]
                 [ duplicate PERCENT [CORRELATION]]
                 [ loss random PERCENT [CORRELATION]]
                 [ loss state P13 [P31 [P32 [P23 P14]]]
                 [ loss gemodel PERCENT [R [1-H [1-K]]]
                 [ reorder PRECENT [CORRELATION] [ gap DISTANCE ]]
                 [ rate RATE [PACKETOVERHEAD] [CELLSIZE] [CELLOVERHEAD]]



Show default queueing discipline
lithium:~# tc -s qdisc ls dev eth0
qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 5029 bytes 21 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0


pfifo_fast
    Standard qdisc for 'Advanced Router' enabled kernels. Consists of a three-band queue which honors Type of  Service  flags,
    as well as the priority that may be assigned to a packet.


Baseline Measurements:

lithium# iperf -c 172.31.1.2
------------------------------------------------------------
Client connecting to 172.31.1.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 172.31.1.1 port 44196 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   994 MBytes   834 Mbits/sec
lithium# iperf -c 172.31.1.2 -P2
------------------------------------------------------------
Client connecting to 172.31.1.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  4] local 172.31.1.1 port 44198 connected with 172.31.1.2 port 5001
[  3] local 172.31.1.1 port 44197 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   570 MBytes   478 Mbits/sec
[  3]  0.0-10.0 sec   548 MBytes   460 Mbits/sec
[SUM]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec


lithium# iperf -u -c 172.31.1.2 -b 1000M
------------------------------------------------------------
Client connecting to 172.31.1.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  4] local 172.31.1.1 port 36755 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   943 MBytes   791 Mbits/sec
[  4] Sent 672968 datagrams
[  4] Server Report:
[  4]  0.0-10.0 sec   940 MBytes   788 Mbits/sec   0.009 ms 2631/672967 (0.39%)
[  4]  0.0-10.0 sec  1 datagrams received out-of-order
lithium# iperf -u -c 172.31.1.2 -b 1000M -P2
------------------------------------------------------------
Client connecting to 172.31.1.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  5] local 172.31.1.1 port 46261 connected with 172.31.1.2 port 5001
[  4] local 172.31.1.1 port 46175 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   570 MBytes   478 Mbits/sec
[  5] Sent 406871 datagrams
[  4]  0.0-10.0 sec   570 MBytes   478 Mbits/sec
[  4] Sent 406570 datagrams
[SUM]  0.0-10.0 sec  1.11 GBytes   957 Mbits/sec
[  5] Server Report:
[  5]  0.0-10.0 sec   570 MBytes   478 Mbits/sec   0.019 ms  111/406870 (0.027%)
[  5]  0.0-10.0 sec  29 datagrams received out-of-order
[  4] Server Report:
[  4]  0.0-10.0 sec   570 MBytes   478 Mbits/sec   0.011 ms  281/406569 (0.069%)
[  4]  0.0-10.0 sec  1 datagrams received out-of-order


lithium# ping 172.31.1.2 -c3
PING 172.31.1.2 (172.31.1.2) 56(84) bytes of data.
64 bytes from 172.31.1.2: icmp_req=1 ttl=64 time=0.444 ms
64 bytes from 172.31.1.2: icmp_req=2 ttl=64 time=0.530 ms
64 bytes from 172.31.1.2: icmp_req=3 ttl=64 time=0.515 ms

--- 172.31.1.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.444/0.496/0.530/0.041 ms


Configure netem based (fixed) 50ms delay on egress
lithium# tc qdisc add dev eth0 root netem delay 50ms
Measure
lithium# ping 172.31.1.2 -c3
PING 172.31.1.2 (172.31.1.2) 56(84) bytes of data.
64 bytes from 172.31.1.2: icmp_req=1 ttl=64 time=50.6 ms
64 bytes from 172.31.1.2: icmp_req=2 ttl=64 time=50.7 ms
64 bytes from 172.31.1.2: icmp_req=3 ttl=64 time=50.7 ms

--- 172.31.1.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 50.600/50.698/50.772/0.197 ms


lithium# iperf -c 172.31.1.2
------------------------------------------------------------
Client connecting to 172.31.1.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  4] local 172.31.1.1 port 44199 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   170 MBytes   142 Mbits/sec
lithium# iperf -c 172.31.1.2 -P2
------------------------------------------------------------
Client connecting to 172.31.1.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  5] local 172.31.1.1 port 44201 connected with 172.31.1.2 port 5001
[  4] local 172.31.1.1 port 44200 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   177 MBytes   148 Mbits/sec
[  4]  0.0-10.1 sec  73.4 MBytes  61.2 Mbits/sec
[SUM]  0.0-10.1 sec   250 MBytes   209 Mbits/sec


lithium# iperf -u -c 172.31.1.2 -b 1000M
------------------------------------------------------------
Client connecting to 172.31.1.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  4] local 172.31.1.1 port 42788 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   886 MBytes   743 Mbits/sec
[  4] Sent 631941 datagrams
[  4] Server Report:
[  4]  0.0-10.0 sec   273 MBytes   229 Mbits/sec   0.015 ms 437415/631940 (69%)
[  4]  0.0-10.0 sec  1 datagrams received out-of-order
lithium# iperf -u -c 172.31.1.2 -b 1000M  -P2
------------------------------------------------------------
Client connecting to 172.31.1.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  5] local 172.31.1.1 port 32920 connected with 172.31.1.2 port 5001
[  4] local 172.31.1.1 port 52721 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   495 MBytes   415 Mbits/sec
[  5] Sent 353252 datagrams
[  5] Server Report:
[  5]  0.0-10.2 sec   139 MBytes   114 Mbits/sec  15.732 ms 253866/352872 (72%)
[  5]  0.0-10.2 sec  30 datagrams received out-of-order


Change the netem root qdisc to 50ms +- 25ms delay on egress
lithium# tc qdisc change dev eth0 root netem delay 50ms 25ms


Measure

lithium# ping -c6 172.31.1.2
PING 172.31.1.2 (172.31.1.2) 56(84) bytes of data.
64 bytes from 172.31.1.2: icmp_req=1 ttl=64 time=46.2 ms
64 bytes from 172.31.1.2: icmp_req=2 ttl=64 time=62.0 ms
64 bytes from 172.31.1.2: icmp_req=3 ttl=64 time=74.2 ms
64 bytes from 172.31.1.2: icmp_req=4 ttl=64 time=75.3 ms
64 bytes from 172.31.1.2: icmp_req=5 ttl=64 time=47.5 ms
64 bytes from 172.31.1.2: icmp_req=6 ttl=64 time=57.6 ms

--- 172.31.1.2 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5008ms
rtt min/avg/max/mdev = 46.284/60.522/75.317/11.461 ms


lithium# iperf -c 172.31.1.2
------------------------------------------------------------
Client connecting to 172.31.1.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  4] local 172.31.1.1 port 44202 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  36.4 MBytes  30.5 Mbits/sec
lithium# iperf -c 172.31.1.2 -P2
------------------------------------------------------------
Client connecting to 172.31.1.2, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  4] local 172.31.1.1 port 44203 connected with 172.31.1.2 port 5001
[  5] local 172.31.1.1 port 44204 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  36.1 MBytes  30.2 Mbits/sec
[  5]  0.0-10.0 sec  25.2 MBytes  21.1 Mbits/sec
[SUM]  0.0-10.0 sec  61.4 MBytes  51.3 Mbits/sec
lithium# iperf -u -c 172.31.1.2 -b 1000M
------------------------------------------------------------
Client connecting to 172.31.1.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  4] local 172.31.1.1 port 49905 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   943 MBytes   791 Mbits/sec
[  4] Sent 672834 datagrams
[  4] Server Report:
[  4]  0.0-4466.0 sec   280 MBytes   526 Kbits/sec  16.734 ms 472969/672832 (70%)
[  4]  0.0-4466.0 sec  191972 datagrams received out-of-order
lithium# iperf -u -c 172.31.1.2 -b 1000M -P2
------------------------------------------------------------
Client connecting to 172.31.1.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  5] local 172.31.1.1 port 57588 connected with 172.31.1.2 port 5001
[  4] local 172.31.1.1 port 50446 connected with 172.31.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   484 MBytes   406 Mbits/sec
[  4] Sent 344967 datagrams
[  4] Server Report:
[  4]  0.0-4544.3 sec   141 MBytes   260 Kbits/sec  16.698 ms 244564/344965 (71%)
[  4]  0.0-4544.3 sec  95803 datagrams received out-of-order


Lots of latency and crazy jitter, the network is on crack
lithium# tc qdisc change dev eth0 root netem delay 400ms 300ms
lithium# ping -c20 172.31.1.2
PING 172.31.1.2 (172.31.1.2) 56(84) bytes of data.
64 bytes from 172.31.1.2: icmp_req=1 ttl=64 time=224 ms
64 bytes from 172.31.1.2: icmp_req=2 ttl=64 time=157 ms
64 bytes from 172.31.1.2: icmp_req=3 ttl=64 time=283 ms
64 bytes from 172.31.1.2: icmp_req=4 ttl=64 time=633 ms
64 bytes from 172.31.1.2: icmp_req=5 ttl=64 time=256 ms
64 bytes from 172.31.1.2: icmp_req=6 ttl=64 time=313 ms
64 bytes from 172.31.1.2: icmp_req=7 ttl=64 time=275 ms
64 bytes from 172.31.1.2: icmp_req=8 ttl=64 time=477 ms
64 bytes from 172.31.1.2: icmp_req=9 ttl=64 time=553 ms
64 bytes from 172.31.1.2: icmp_req=10 ttl=64 time=416 ms
64 bytes from 172.31.1.2: icmp_req=11 ttl=64 time=637 ms
64 bytes from 172.31.1.2: icmp_req=12 ttl=64 time=594 ms
64 bytes from 172.31.1.2: icmp_req=13 ttl=64 time=647 ms
64 bytes from 172.31.1.2: icmp_req=14 ttl=64 time=315 ms
64 bytes from 172.31.1.2: icmp_req=15 ttl=64 time=356 ms
64 bytes from 172.31.1.2: icmp_req=16 ttl=64 time=211 ms
64 bytes from 172.31.1.2: icmp_req=17 ttl=64 time=215 ms
64 bytes from 172.31.1.2: icmp_req=18 ttl=64 time=256 ms
64 bytes from 172.31.1.2: icmp_req=19 ttl=64 time=638 ms
64 bytes from 172.31.1.2: icmp_req=20 ttl=64 time=453 ms

--- 172.31.1.2 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 19026ms
rtt min/avg/max/mdev = 157.775/395.904/647.801/165.088 ms


Big latency and big jitter but attempt 90% correlation of each random element with his next random element
lithium# tc qdisc change dev eth0 root netem delay 400ms 300ms 90%
lithium# ping -c20 172.31.1.2
PING 172.31.1.2 (172.31.1.2) 56(84) bytes of data.
64 bytes from 172.31.1.2: icmp_req=1 ttl=64 time=398 ms
64 bytes from 172.31.1.2: icmp_req=2 ttl=64 time=283 ms
64 bytes from 172.31.1.2: icmp_req=3 ttl=64 time=135 ms
64 bytes from 172.31.1.2: icmp_req=4 ttl=64 time=202 ms
64 bytes from 172.31.1.2: icmp_req=5 ttl=64 time=475 ms
64 bytes from 172.31.1.2: icmp_req=6 ttl=64 time=685 ms
64 bytes from 172.31.1.2: icmp_req=7 ttl=64 time=320 ms
64 bytes from 172.31.1.2: icmp_req=8 ttl=64 time=347 ms
64 bytes from 172.31.1.2: icmp_req=9 ttl=64 time=692 ms
64 bytes from 172.31.1.2: icmp_req=10 ttl=64 time=687 ms
64 bytes from 172.31.1.2: icmp_req=11 ttl=64 time=356 ms
64 bytes from 172.31.1.2: icmp_req=12 ttl=64 time=390 ms
64 bytes from 172.31.1.2: icmp_req=13 ttl=64 time=269 ms
64 bytes from 172.31.1.2: icmp_req=14 ttl=64 time=375 ms
64 bytes from 172.31.1.2: icmp_req=15 ttl=64 time=178 ms
64 bytes from 172.31.1.2: icmp_req=16 ttl=64 time=618 ms
64 bytes from 172.31.1.2: icmp_req=17 ttl=64 time=248 ms
64 bytes from 172.31.1.2: icmp_req=18 ttl=64 time=236 ms
64 bytes from 172.31.1.2: icmp_req=19 ttl=64 time=375 ms
64 bytes from 172.31.1.2: icmp_req=20 ttl=64 time=595 ms

--- 172.31.1.2 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 19010ms
rtt min/avg/max/mdev = 135.251/393.653/692.645/171.976 ms


in order to hide his crack-pipe, we may just need reduce the delay variation
lithium# tc qdisc change dev eth0 root netem delay 200ms 20ms 50%
lithium# ping -c20 172.31.1.2
PING 172.31.1.2 (172.31.1.2) 56(84) bytes of data.
64 bytes from 172.31.1.2: icmp_req=1 ttl=64 time=206 ms
64 bytes from 172.31.1.2: icmp_req=2 ttl=64 time=196 ms
64 bytes from 172.31.1.2: icmp_req=3 ttl=64 time=196 ms
64 bytes from 172.31.1.2: icmp_req=4 ttl=64 time=211 ms
64 bytes from 172.31.1.2: icmp_req=5 ttl=64 time=191 ms
64 bytes from 172.31.1.2: icmp_req=6 ttl=64 time=188 ms
64 bytes from 172.31.1.2: icmp_req=7 ttl=64 time=213 ms
64 bytes from 172.31.1.2: icmp_req=8 ttl=64 time=219 ms
64 bytes from 172.31.1.2: icmp_req=9 ttl=64 time=185 ms
64 bytes from 172.31.1.2: icmp_req=10 ttl=64 time=196 ms
64 bytes from 172.31.1.2: icmp_req=11 ttl=64 time=193 ms
64 bytes from 172.31.1.2: icmp_req=12 ttl=64 time=194 ms
64 bytes from 172.31.1.2: icmp_req=13 ttl=64 time=215 ms
64 bytes from 172.31.1.2: icmp_req=14 ttl=64 time=209 ms
64 bytes from 172.31.1.2: icmp_req=15 ttl=64 time=184 ms
64 bytes from 172.31.1.2: icmp_req=16 ttl=64 time=182 ms
64 bytes from 172.31.1.2: icmp_req=17 ttl=64 time=183 ms
64 bytes from 172.31.1.2: icmp_req=18 ttl=64 time=213 ms
64 bytes from 172.31.1.2: icmp_req=19 ttl=64 time=187 ms
64 bytes from 172.31.1.2: icmp_req=20 ttl=64 time=207 ms

--- 172.31.1.2 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 19028ms
rtt min/avg/max/mdev = 182.506/198.908/219.301/11.804 ms


See qdisc statistics
lithium# tc -s qdisc ls dev eth0
qdisc netem 8005: root refcnt 2 limit 1000 delay 200.0ms  20.0ms 50%
 Sent 2367236851 bytes 1567250 pkt (dropped 2837991, overlimits 0 requeues 0)
 backlog 106b 1p requeues 0


Use a normal distribution for the pseudorandom delay variation
lithium# tc qdisc change dev eth0 root netem delay 200ms 20ms distribution normal
lithium# ping -c20 172.31.1.2
PING 172.31.1.2 (172.31.1.2) 56(84) bytes of data.
64 bytes from 172.31.1.2: icmp_req=1 ttl=64 time=195 ms
64 bytes from 172.31.1.2: icmp_req=2 ttl=64 time=196 ms
64 bytes from 172.31.1.2: icmp_req=3 ttl=64 time=183 ms
64 bytes from 172.31.1.2: icmp_req=4 ttl=64 time=183 ms
64 bytes from 172.31.1.2: icmp_req=5 ttl=64 time=166 ms
64 bytes from 172.31.1.2: icmp_req=6 ttl=64 time=208 ms
64 bytes from 172.31.1.2: icmp_req=7 ttl=64 time=224 ms
64 bytes from 172.31.1.2: icmp_req=8 ttl=64 time=216 ms
64 bytes from 172.31.1.2: icmp_req=9 ttl=64 time=211 ms
64 bytes from 172.31.1.2: icmp_req=10 ttl=64 time=221 ms
64 bytes from 172.31.1.2: icmp_req=11 ttl=64 time=206 ms
64 bytes from 172.31.1.2: icmp_req=12 ttl=64 time=219 ms
64 bytes from 172.31.1.2: icmp_req=13 ttl=64 time=214 ms
64 bytes from 172.31.1.2: icmp_req=14 ttl=64 time=197 ms
64 bytes from 172.31.1.2: icmp_req=15 ttl=64 time=216 ms
64 bytes from 172.31.1.2: icmp_req=16 ttl=64 time=182 ms
64 bytes from 172.31.1.2: icmp_req=17 ttl=64 time=179 ms
64 bytes from 172.31.1.2: icmp_req=18 ttl=64 time=223 ms
64 bytes from 172.31.1.2: icmp_req=19 ttl=64 time=209 ms
64 bytes from 172.31.1.2: icmp_req=20 ttl=64 time=220 ms

--- 172.31.1.2 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 19030ms
rtt min/avg/max/mdev = 166.480/203.929/224.446/16.774 ms


... pinging lithium from another host we should observe a similar behavior but now his ECHO_REPLYs are slowed down
beryllium# ping -A -c20 172.31.1.1
PING 172.31.1.1 (172.31.1.1) 56(84) bytes of data.
64 bytes from 172.31.1.1: icmp_req=1 ttl=64 time=203 ms
64 bytes from 172.31.1.1: icmp_req=2 ttl=64 time=204 ms
64 bytes from 172.31.1.1: icmp_req=3 ttl=64 time=207 ms
64 bytes from 172.31.1.1: icmp_req=4 ttl=64 time=221 ms
64 bytes from 172.31.1.1: icmp_req=5 ttl=64 time=185 ms
64 bytes from 172.31.1.1: icmp_req=6 ttl=64 time=224 ms
64 bytes from 172.31.1.1: icmp_req=7 ttl=64 time=182 ms
64 bytes from 172.31.1.1: icmp_req=8 ttl=64 time=178 ms
64 bytes from 172.31.1.1: icmp_req=9 ttl=64 time=220 ms
64 bytes from 172.31.1.1: icmp_req=10 ttl=64 time=236 ms
64 bytes from 172.31.1.1: icmp_req=11 ttl=64 time=180 ms
64 bytes from 172.31.1.1: icmp_req=12 ttl=64 time=184 ms
64 bytes from 172.31.1.1: icmp_req=13 ttl=64 time=205 ms
64 bytes from 172.31.1.1: icmp_req=14 ttl=64 time=197 ms
64 bytes from 172.31.1.1: icmp_req=15 ttl=64 time=227 ms
64 bytes from 172.31.1.1: icmp_req=16 ttl=64 time=173 ms
64 bytes from 172.31.1.1: icmp_req=17 ttl=64 time=180 ms
64 bytes from 172.31.1.1: icmp_req=18 ttl=64 time=199 ms
64 bytes from 172.31.1.1: icmp_req=19 ttl=64 time=192 ms
64 bytes from 172.31.1.1: icmp_req=20 ttl=64 time=204 ms

--- 172.31.1.1 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 3842ms
rtt min/avg/max/mdev = 173.172/200.557/236.417/17.955 ms, pipe 2, ipg/ewma 202.210/198.474 ms


Availabe probability distributions
lithium# ls -1 /usr/lib/tc/*dist
/usr/lib/tc/experimental.dist
/usr/lib/tc/normal.dist
/usr/lib/tc/pareto.dist
/usr/lib/tc/paretonormal.dist


Drop random 0.5% of packets ( tc, not layer3, packets )
lithium# tc qdisc change dev eth0 root netem loss 0.5%
lithium# ping -A -c10000 -q beryllium
PING beryllium (172.31.1.2) 56(84) bytes of data.

--- beryllium ping statistics ---
10000 packets transmitted, 9948 received, 0% packet loss, time 3538ms
rtt min/avg/max/mdev = 0.065/0.159/3.710/0.094 ms, ipg/ewma 0.353/0.145 ms


beryllium# ping -A -c10000 -q lithium
PING lithium (172.31.1.1) 56(84) bytes of data.

--- lithium ping statistics ---
10000 packets transmitted, 9949 received, 0% packet loss, time 3470ms
rtt min/avg/max/mdev = 0.069/0.166/3.337/0.088 ms, ipg/ewma 0.347/0.156 ms


Show qdisc statistics
lithium# tc -s qdisc ls dev eth0
qdisc netem 8007: root refcnt 2 limit 1000 loss 0.5%
 Sent 1962385 bytes 20015 pkt (dropped 103, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0


Remove our netem queing discipline
lithium# tc qdisc del dev eth0 root


Baseline measurements
lithium# ping -A -c10000 -q beryllium
PING beryllium (172.31.1.2) 56(84) bytes of data.

--- beryllium ping statistics ---
10000 packets transmitted, 10000 received, 0% packet loss, time 2730ms
rtt min/avg/max/mdev = 0.062/0.146/0.602/0.079 ms, ipg/ewma 0.273/0.117 ms


beryllium# ping -A -c10000 -q lithium
PING lithium (172.31.1.1) 56(84) bytes of data.

--- lithium ping statistics ---
10000 packets transmitted, 10000 received, 0% packet loss, time 2868ms
rtt min/avg/max/mdev = 0.066/0.163/3.487/0.091 ms, ipg/ewma 0.286/0.220 ms


1% (random) drop with probality(n) = 0.8 * propablity(n-1) + 0.2 * Random
lithium# tc qdisc add dev eth0 root netem loss 1% 80%


skipping the measurements --they are inconclusive
... hmm , maybe I need to put sodium --one more host-- in the game

~3% of duplicates
lithium# tc qdisc change dev eth0 root netem duplicate 3%
lithium# ping -A -c10000 -q beryllium
PING beryllium (172.31.1.2) 56(84) bytes of data.

--- beryllium ping statistics ---
10000 packets transmitted, 10000 received, +317 duplicates, 0% packet loss, time 2844ms
rtt min/avg/max/mdev = 0.066/0.149/3.409/0.075 ms, pipe 2, ipg/ewma 0.284/0.131 ms


~1% of single bit errors in the packets ( frames? )
lithium# tc qdisc change dev eth0 root netem corrupt 1%
lithium# ping -A -c10000 -q beryllium
PING beryllium (172.31.1.2) 56(84) bytes of data.

--- beryllium ping statistics ---
10000 packets transmitted, 9909 received, 0% packet loss, time 3874ms
rtt min/avg/max/mdev = 0.064/0.156/0.648/0.078 ms, ipg/ewma 0.387/0.165 ms


force out of order with gap
gap uses a fixed sequence and reorders every Nth packet
eg the following should cause ~20% of every 3rd packet to be sent immediately while the ~previous two packets are delayed for 10ms
lithium# tc qdisc change dev eth0 root netem reorder 20% gap 3 delay 10ms
lithium# ping -A -c10000 -q beryllium
PING beryllium (172.31.1.2) 56(84) bytes of data.

--- beryllium ping statistics ---
10000 packets transmitted, 9895 received, 1% packet loss, time 102198ms
rtt min/avg/max/mdev = 0.142/9.139/13.900/3.517 ms, pipe 2, ipg/ewma 10.220/8.481 ms
lithium# tc -s qdisc ls dev eth0
qdisc netem 8008: root refcnt 2 limit 1000 delay 10.0ms reorder 20% corrupt 1% gap 3
 Sent 2009781566 bytes 1745324 pkt (dropped 10, overlimits 0 requeues 2653)
 backlog 0b 0p requeues 2653


Force out of order
25% of packets (with a correlation of 50%) will get sent immediately, (all) others will be delayed by 10ms
lithium# tc qdisc change dev eth0 root netem delay 10ms reorder 25% 50%
lithium# ping -A -c10000 -q beryllium
PING beryllium (172.31.1.2) 56(84) bytes of data.

--- beryllium ping statistics ---
10000 packets transmitted, 9899 received, 1% packet loss, time 104680ms
rtt min/avg/max/mdev = 0.109/9.829/13.955/2.632 ms, pipe 2, ipg/ewma 10.469/7.666 ms
lithium# ping -A -c10 beryllium
PING beryllium (172.31.1.2) 56(84) bytes of data.
64 bytes from beryllium (172.31.1.2): icmp_req=1 ttl=64 time=10.6 ms
64 bytes from beryllium (172.31.1.2): icmp_req=2 ttl=64 time=10.5 ms
64 bytes from beryllium (172.31.1.2): icmp_req=3 ttl=64 time=10.5 ms
64 bytes from beryllium (172.31.1.2): icmp_req=4 ttl=64 time=10.4 ms
64 bytes from beryllium (172.31.1.2): icmp_req=5 ttl=64 time=10.4 ms
64 bytes from beryllium (172.31.1.2): icmp_req=6 ttl=64 time=10.6 ms
64 bytes from beryllium (172.31.1.2): icmp_req=7 ttl=64 time=10.5 ms
64 bytes from beryllium (172.31.1.2): icmp_req=8 ttl=64 time=10.6 ms
64 bytes from beryllium (172.31.1.2): icmp_req=9 ttl=64 time=10.8 ms
64 bytes from beryllium (172.31.1.2): icmp_req=10 ttl=64 time=10.5 ms

--- beryllium ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 100ms
rtt min/avg/max/mdev = 10.486/10.582/10.813/0.150 ms, ipg/ewma 11.116/10.600 ms


Reorder with delay eg delay each packet 100ms +-rand(75)ms Jitter
lithium# tc qdisc change dev eth0 root netem delay 100ms 75ms
lithium# ping -A -c10 beryllium
PING beryllium (172.31.1.2) 56(84) bytes of data.
64 bytes from beryllium (172.31.1.2): icmp_req=1 ttl=64 time=152 ms
64 bytes from beryllium (172.31.1.2): icmp_req=2 ttl=64 time=168 ms
64 bytes from beryllium (172.31.1.2): icmp_req=3 ttl=64 time=88.1 ms
64 bytes from beryllium (172.31.1.2): icmp_req=4 ttl=64 time=137 ms
64 bytes from beryllium (172.31.1.2): icmp_req=5 ttl=64 time=145 ms
64 bytes from beryllium (172.31.1.2): icmp_req=6 ttl=64 time=105 ms
64 bytes from beryllium (172.31.1.2): icmp_req=7 ttl=64 time=138 ms
64 bytes from beryllium (172.31.1.2): icmp_req=8 ttl=64 time=164 ms
64 bytes from beryllium (172.31.1.2): icmp_req=9 ttl=64 time=174 ms
64 bytes from beryllium (172.31.1.2): icmp_req=10 ttl=64 time=123 ms

--- beryllium ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 1307ms
rtt min/avg/max/mdev = 88.111/139.794/174.547/26.414 ms, pipe 2, ipg/ewma 145.244/143.915 ms
lithium# ping -A -c1000 -q beryllium
PING beryllium (172.31.1.2) 56(84) bytes of data.

--- beryllium ping statistics ---
1000 packets transmitted, 991 received, 0% packet loss, time 102193ms
rtt min/avg/max/mdev = 26.115/102.891/175.749/43.053 ms, pipe 3, ipg/ewma 102.295/92.228 ms






netem







alog netem

cpan : Terminal does not support AddHistory. on debian

# cpan
Terminal does not support AddHistory.


Term::ReadLine::Perl is missing

Term::ReadLine::Perl is debian-packaged as libterm-readline-perl-perl

# apt-get install libterm-readline-perl-perl
# cpan

cpan shell -- CPAN exploration and modules installation (v2.05)
Enter 'h' for help.

cpan[1]>         




Debian cpan Terminal does not support AddHistory



install gnome on jessie

Install gnome on Debian Jessie

The system
# cat /etc/issue /etc/debian_version 
Debian GNU/Linux 8 \n \l

8.2



Install gnome
# tasksel install gnome-desktop --new-install


which is (tasksel test)
# tasksel -t install gnome-desktop
debconf-apt-progress -- apt-get -q -y -o APT::Install-Recommends=true -o APT::Get::AutomaticRemove=true -o APT::Acquire::Retries=3 install task-gnome-desktop









install gnome on jessie

django notes

Notes on setting up a basic django app on a debian system using MySQL and Apache2 ...

Install the apache2 HTTP daemon
# apt-get install apache2


Install the WSGI apache2 module ( Python Web Server Gateway Interface )
# apt-get install libapache2-mod-wsgi


Install MySQL, the MySQL database development files, and the Python interface to MySQL
# apt-get install mysql-server libmysqlclient-dev python-mysqldb


Install gcc, build-essential and python headers
# apt-get install gcc build-essential python-dev


Install Python Virtual Environment Creator to avoid system Python
# apt-get install python-virtualenv


Create a virtual python environment
$ virtualenv django


Create or copy a file listing the python requirements.
e.g.
# cat requirements.txt 
Django==1.8.4                                                                                            
Markdown==2.6.2
MySQL-python==1.2.5
argparse==1.2.1
django-filter==0.11.0
djangorestframework==3.2.3
mysql==0.0.1
mysqlclient==1.3.6
wsgiref==0.1.2


And install them into the django virtual python environment
# source django/bin/activate
(django):~# pip install -r ./django/requirements.txt 


Check python modules installed into the django python virtual environment
(django):~# pip freeze
Django==1.8.4
Markdown==2.6.2
MySQL-python==1.2.5
argparse==1.2.1
django-filter==0.11.0
djangorestframework==3.2.3
mysql==0.0.1
mysqlclient==1.3.6
wsgiref==0.1.2


Create a django_project and configure the Apache HTTP daemon
(django):~# cd /var/www/sites/myproject
(django):~# django-admin startproject myproject .
(django):~# deactivate
#


Create a virtual host, eg.
# cat /etc/apache2/sites-available/000-myproject.conf 
<VirtualHost *:80>

  ServerAdmin g0@spam.bot.ipduh.com
  DocumentRoot /var/www/sites/myproject/www


 WSGIDaemonProcess wids python-path=/var/www/sites/myproject:/home/myproject/django/lib/python2.7/site-packages
 WSGIProcessGroup  myproject
 WSGIScriptAlias / /var/www/sites/myproject/myproject/wsgi.py

 Alias /static/  /var/www/sites/myproject/www/static/

  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

# vim: syntax=apache ts=2 sw=2 sts=2 sr noet



Enable the new virtual apache host and restart apache
# a2ensite 000-myproject.conf
# /etc/init.d/apache2 restart


use your browser to see the django welcome page,
you should see: It worked! Congratulations ...

Configure django to use MySQL, it uses SQLite by default.

add a mysql user and a mysql database
# mysql -u root -p
mysql>create database myproject;
mysql> grant all on myproject.* to myproject_user;
mysql> set password for myproject_user=password('somepasswd');
mysql>flush privileges;
mysql> exit
Bye
#


Configure the django project to use MySQL, edit settings.py
DATABASES = {
    'default': {
  'ENGINE': 'django.db.backends.mysql',
#   'ENGINE': 'mysql.connector.django',
        'NAME': 'myproject',
        'USER': 'myproject_user',
        'PASSWORD': 'somepasswd',
        'HOST': 'localhost',
        'PORT': '3306',
    }
}



Create the MySQL schema
(django):# python manage.py migrate


Copy the django-admin-app static files directory to /var/www/sites/myproject/www/static/
The directory that contains the admin static files for admin should be in django/lib/python2.7/site-packages/django/contrib/admin/static

Create a django super user
(django):# python manage.py createsuperuser




django - MySQL - apache2 on debian notes



mozilla thunderbird - icedove hide user agent

To hide (obfuscate or not send) the User agent header in thunderbird and icedove





open the config editor and look for general.useragent.override

If it does not exist add it ( type: string )

You may set it to a value --string or leave it empty.

If you leave it empty Thunderbird/Icedove will not add a User agent Header to your outgoing emails.