20141120

squeeze container on a wheezy host

Notes on putting a Debian squeeze Linux Container on a Debian Wheezy host.

The requirement; the Squeeze system in the container runs a TCP/IP application that should be accessible only by the host.



Create a bridge between a dummy network interface and the network interface used by the container.

Load dummy module (with numdummies=1) at startup.
# echo dummy >> /etc/modules


Install the Linux ethernet bridge utilities.
# apt-get install bridge-utils


Add stanzas that create an inhost bridge that contains a dummy to /etc/network/interfaces
auto dummy0
  iface dummy0 inet static

auto etherisland
  iface etherisland inet static
  address 172.16.17.18
  netmask 255.255.255.128
  bridge_ports dummy0
  bridge_stp  off
  bridge_waitport 0
  bridge_fd 0



Load one dummy interface and restart networking.
# modprobe dummy
# /etc/init.d/networking restart




Install lxc and prerequisites.
# apt-get install lxc
which also installs debootstrap libcap2-bin and libpam-cap.

Mount control groups hierarchy now and at boot.
# mount /sys/fs/cgroup/
# echo "cgroup /sys/fs/cgroup  cgroup  defaults  0  0" >> /etc/fstab


Check your kernel for lxc support.
# lxc-checkconfig


Get the squeeze template.
# wget https://raw.githubusercontent.com/ipduh/lxc-squeeze/master/lxc-squeeze -O /usr/share/lxc/templates/lxc-squeeze


Allow execution to all.
# chmod 755 /usr/share/lxc/templates/lxc-squeeze


Create the Squeeze Container.
# lxc-create -n squeezie -t squeeze


Start the container in the background.
# lxc-start -n squeezie -d


Console into the squeezie container.
# lxc-console -n squeezie

Type <Ctrl+a q> to exit the console, <Ctrl+a Ctrl+a> to enter Ctrl+a itself

Debian GNU/Linux 6.0 squeezie tty1

squeezie login: root
Password: 
Linux squeezie 3.2.0-4-amd64 #1 SMP Debian 3.2.63-2 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

root@squeezie:~#
The password set by the template is squeezie.
Alternatively, you may ssh to squeezie from the host.

Change the root password
root@squeezie:~# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully



Give Temporary Internet Connectivity to the Squeeze Container.
root@squeezie:~# route add default gw 172.16.17.18
and in the host
# echo 1 > /proc/sys/net/ipv4/ip_forward
# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -s 172.16.17.0/25
To disable the Internet Connectivity reset your Firewall e.g.
# /etc/bif




Squeeze LXC on Wheezy

20141118

linux dummy interface



Linux Pseudo Device i.e. Dummy Interface Notes.



Create interface dummy0.
# modprobe dummy


I need (more dummies)
# rmmod dummy
# modprobe dummy numdummies=3
to Create 3 pseudo interfaces.

Set the Pseudo Interface(s ') MAC address(es).
# ifconfig dummy0 hw ether fc:de:ad:be:ef:10
# ifconfig dummy1 hw ether fc:de:ad:be:ef:11
# ifconfig dummy2 hw ether 00:00:0c:f0:00:0d
(: 00:00:0c:f0:00:0d :)

Set the Pseudo Interface(s ') IP address(es).
# ifconfig dummy0 172.16.17.18/25
# ifconfig dummy1 172.16.17.19/25
# ifconfig dummy2 192.0.2.8/26


Show dummy0 configuration.
# ifconfig dummy0
dummy0    Link encap:Ethernet  HWaddr fc:de:ad:be:ef:10  
          inet addr:172.16.17.18  Bcast:172.16.17.127  Mask:255.255.255.128
          inet6 addr: fe80::fede:adff:febe:ef10/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:210 (210.0 B)


Create a bridge.
# brctl addbr etherisland


Set forwarding delay to 0 seconds.
# brctl setfd etherisland 0


Bridge dummy0 and dummy1 dummies.
# brctl addif etherisland dummy0 dummy1


List bridge(s).
# brctl show
bridge name bridge id          STP enabled interfaces
etherisland 8000.fcdeadbeef10  no          dummy0
                                           dummy1







Make an ethernet inhost island --a bridge that contains a pseudo interface-- stick.

Load dummy module (with numdummies=1) at system startup.
# echo dummy >> /etc/modules


The /etc/network/interfaces stanzas.
auto dummy0
  iface dummy0 inet static

auto etherisland
  iface etherisland inet static
  address 172.16.17.18
  netmask 255.255.255.128
  bridge_ports dummy0
  bridge_stp  off
  bridge_waitport 0
  bridge_fd 0
  bridge_hello 1





Linux pseudo interface i.e. dummy



20141112

PostgreSQL Notes

PostgreSQL Debian Notes.

Install PostgreSQL on debian
Login, login using psql, list databases and users , set postgres password.
Authentication Modes and Access.

Create a new user and a new database.
Delete database
Vacuum
Enable remote Access
Logical backup with pg_dump




Install PostgreSQL from debian packages.
# apt-get install postgresql
Now, the debian package postgresql installs all dependencies and the client.
libpq5 postgresql-9.1 postgresql-client-9.1 postgresql-client-common postgresql-common
Alternatively, you may install PostgreSQL from apt repositories maintained by the PostgreSQL Global Development Group.



Login as the postgres user using psql--the PostgreSQL interactive terminal.
# su - postgres -c psql 
psql (9.1.14)
Type "help" for help.

postgres=#


The default authentication mode is 'ident' or 'peer' i.e. system user x can only login as PostgreSQL user x.

The PostgreSQL Client Authentication Configuration File in debian is at
/etc/postgresql/*/main/pg_hba.conf


The 'trust' authentication mode allows connections unconditionally and the mode 'password' requires the client to supply an unencrypted password.

Since the default local authentication mode is 'peer', applications that make use of the database locally may give authentication errors if running as another system user. You may want to check if setting the mode to 'trust' fixes the problem.
# "local" is for Unix domain socket connections only
#local   all             all                                     peer
local   all             all                                     trust 
But not settle for 'trust' i.e. allow local connections unconditionally.

By default PostgreSQL binds to 127.0.0.1:5432 --you can change it at
/etc/postgresql/*/main/postgresql.conf 
'listen_addresses' if you need to enable remote access.

List databases.
postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(3 rows)

Most new databases are created by copying tempate1 of the templates.

List users.
postgres=# \du
                             List of roles
 Role name |                   Attributes                   | Member of 
-----------+------------------------------------------------+-----------
 postgres  | Superuser, Create role, Create DB, Replication | {}



Set the password for the PostgreSQL user postgres.
postgres=# \password postgres
Enter new password: 
Enter it again: 


Exit psql.
postgres-# \q
#


Create a new PostgreSQL user.
# su - postgres -c "createuser -P puser"
Enter password for new role: 
Enter it again: 
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n


Create the database puser that belongs to user puser.
# su - postgres -c "createdb -O puser puser"


List PostgreSQL databases and user roles.
# su - postgres -c psql
psql (9.1.14)
Type "help" for help.

postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 puser     | puser    | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(4 rows)

postgres=# \du
                             List of roles
 Role name |                   Attributes                   | Member of 
-----------+------------------------------------------------+-----------
 postgres  | Superuser, Create role, Create DB, Replication | {}
 puser     |                                                | {}

postgres=# \q
#


Delete the database puser.
# su - postgres -c "dropdb puser"


Calculate statistics for use by the optimizer(-z) and Garbage-Collect i.e. full vacuum(-f) all(-a) PostgreSQL databases in this host.
# su - postgres -c "vacuumdb -a -f -z"
Vacuumdb is a VACUUM wrapper (pg_wrapper) written in Perl.



pg_dump - Logical Backups

Pg_dump extracts a PostgreSQL database into a script file or other archive file.

Create a compressed sql script file with the schema and data of the database puser that belongs to the PostgreSQL user puser in host a.
a# pg_dump -o -U puser -h localhost puser |gzip > puser.dump.gz
Password: 


Restore PostgreSQL database in host b.
b# gunzip  puser.dump.gz
b# su - postgres -c "createuser -P puser"
Enter password for new role: 
Enter it again: 
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) y
Shall the new role be allowed to create more new roles? (y/n) n
b# su - postgres -c "createdb -O puer puser"
b# psql -U puser -h localhost < puser.dump 
Password for user puser:




PostgreSQL Notes



20141107

devz howto II

devz Howto





Get devz
$ git clone https://github.com/ipduh/devz.git
$ cd devz


Install devz as root
$ su
# ./install_devz_as_root.sh
# source ~/.bashrc


Install devz for a user
# exit
$ ./install_devz_as_user.sh
$ source ~/.bashrc


Configure devz
  • 1) Copy public SSH key to remote system(s)
  • 2) Adjust ~/.devzconfig/production-servers
1) Create SSH key pair and copy the public key to a remote "production" server.
$ ssh-keygen -t dsa
$ scp ~/.ssh/id_dsa.pub production_server:~/.ssh/authorized_keys2
2) An example ~/.devzconfig/production-servers
# production servers
# IP address , SSH TCP port, user
192.0.2.22,44,usar
192.0.2.23,22,usar


Use devz

Initialize SSH agent.
$ devz-setagent


Stor
$ stor blah
devz:The directory ./stor does not exist! I will create it.
devz:blah is at ./stor/blah.0


Toprod
$ toprod blah 
devz:/home/usar/blah to usar@192.0.2.22:44:/home/usar/blah
blah                                                                                                          100%    6     0.0KB/s   00:00  
devz:/home/usar/blah to usar@192.0.2.23:22:/home/usar/blah
blah                                                                                                          100%    6     0.0KB/s   00:00


Ctoprod
$ ctoprod "cat blah"
devz: usar@192.0.2.22:44 "cat blah"
***Start 192.0.2.22***
blah

***End 192.0.2.22***
devz: usar@192.0.2.23:22 
***Start 192.0.2.23***
blah

***End 192.0.2.23***


Fromprod
$ fromprod blah
devz:blah exists! Please stor it and delete it or rename it.
$ rm blah
$ fromprod blah
devz:ipduh@192.0.2.22:44:/home/usar/blah to /hom/usar/blah
blah                                                                                                          100%    6     0.0KB/s 00:00                                       


get help-cheatsheet
$ devz
******
devz
DEVeloper'S Stupid Servant.
A bash extention that helps the administrator of similar dev and production systems.
g0 2010 - http://ipduh.com/contact
http://sl.ipduh.com/devz-howto
******
devz verbs:
*
'toprod' or 'devz toprod'
 toprod file
 scp a file to the production server(s)
*
'ctoprod' or 'devz ctoprod'
 ctoprod 'command;command;'
 send command(s) to poduction server(s)
*
'fromprod' or 'devz fromprod'
 fromprod file
 scp a file from the first production server here.
*
'stor' or 'devz stor'
 stor file
 creates the directory stor in the current directory if it does not exist.
 makes a copy of the file in stor
 the file gets a version number like file.n where n [0,n]
*
'devz-setagent' or 'devz setagent'
 setagent
 start an ssh-agent login session
*
'devz-showconfig' or 'devz showconfig'
 showconfig
 See the Current devz configuration
*
'devz-setconfig' or 'devz setconfig'
 setconfig
 add server to the production-servers list file
 setconfig cannot configure much, check the devz-howto for your first setup
*
'devz-prodsrvexists' or 'devz prodsrvexists'
 prodsrvexists
 check if ${DEVZ_PRO_SRV} exists and  print an example ${DEVZ_PRO_SRV} file
*
******





devz-howto



20141029

mysql administration notes

MySQL server on debian administration notes .



Install on debian
debian-sys-maint user
Passwordless administration commands or SQL statements
Configuration files and MySQL system variables
mysqladmin
Using the standard client for an elementary database exploration.
Binary Backups
mysqlhotcopy
mysqldump --Logical Backups.
Master-Slave Replication.
mysqlcheck.



Now, with the debian MySQL server packet installation
# apt-get install mysql-server
you get the MySQL server version 5.5 along with its prerequisites, basic MySQL tools like the standard MySQL client and the Perl DBI, and some other stuff like mailx.
heirloom-mailx libaio1 libdbd-mysql-perl libdbi-perl libhtml-template-perl
libmysqlclient18 mysql-client-5.5 mysql-common mysql-server-5.5 mysql-server-core-5.5




The debian-sys-maint user.

The debian package adds the debian-sys-maint MySQL user that can do pretty much everything if logging in locally.
mysql> show grants for 'debian-sys-maint'@'localhost';
+----------------------------------------------------------------------------------------------------------------------------------------------------+
| Grants for debian-sys-maint@localhost                                                                                                              |
+----------------------------------------------------------------------------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY PASSWORD '*0123456789ABCDEF12346789082F1970A47EDCBA' WITH GRANT OPTION |
+----------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> select Host,Super_priv,Create_priv,Grant_priv,Drop_priv from mysql.user where user='debian-sys-maint';
+-----------+------------+-------------+------------+-----------+
| Host      | Super_priv | Create_priv | Grant_priv | Drop_priv |
+-----------+------------+-------------+------------+-----------+
| localhost | Y          | Y           | Y          | Y         |
+-----------+------------+-------------+------------+-----------+
1 row in set (0.01 sec)



The debian-sys-maint user has a "random" password stored at /etc/mysql/debian.cnf.
# ls -l /etc/mysql/debian.cnf 
-rw------- 1 root root 333 Oct 23 16:04 /etc/mysql/debian.cnf




Skip the Password Prompts.

The debian-sys-maint user is used by system scripts, but it is convenient for an administrator to use it in his commands and scripts.
# mysqladmin --defaults-file=/etc/mysql/debian.cnf create yo
# mysqladmin --defaults-file=/etc/mysql/debian.cnf drop yo
Dropping the database is potentially a very bad thing to do.
Any data stored in the database will be destroyed.

Do you really want to drop the 'yo' database [y/N] y
Database "yo" dropped


Another way to skip the password prompt when running a SQL command.
# mysql -u root -p"password" -e "command;"
As far as I know since at least version 5.1.41-3 and upwards commands like the above, do not reveal your password in the current processes snapshot (ps)
root     30510  0.0  0.1  40280  2696 pts/0    S+   07:41   0:00          \_ mysql -u root -px xxxxxx




Configuration files and MySQL system variables.

Default options are read from the following files in the given order: /etc/my.cnf /etc/mysql/my.cnf /usr/etc/my.cnf ~/.my.cnf

Within the MySQL configuration or option files we may define groups for which we want to set options. A Group name often matches a MySQL helper program name. An options group named "group" starts with [group] in the options or configuration files.

The debian debian-package puts the configuration files in /etc/mysql.

The database table and DB specific options files are stored in /var/lib/mysql

An easy way to display MySQL system variables and their values.
# mysqladmin --defaults-file=/etc/mysql/debian.cnf variables


A quick reference of the MySQL daemon options along with some MySQL system variables and their values.
# mysqld --h -v


MySQL has many logging options (look for log in the MySQL system variables).
General query log keeping is expensive and disabled by default. It may may be enabled permanently in the configuration file /etc/mysql/my.cnf
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1


As of version 5.1, general log keeping may be enabled or disabled at runtime.
# mysql -u root -p"root_paswd"
mysql> show variables like 'general%';
+------------------+-------------------------------+
| Variable_name    | Value                         |
+------------------+-------------------------------+
| general_log      | OFF                           |
| general_log_file | /var/lib/mysql/anaxagoras.log |
+------------------+-------------------------------+
2 rows in set (0.00 sec)
mysql> SET GLOBAL general_log=1;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like 'general%';
+------------------+-------------------------------+
| Variable_name    | Value                         |
+------------------+-------------------------------+
| general_log      | ON                            |
| general_log_file | /var/lib/mysql/anaxagoras.log |
+------------------+-------------------------------+
2 rows in set (0.00 sec)
mysql> SET GLOBAL general_log=0;
Query OK, 0 rows affected (0.08 sec)
mysql> quit
Bye
# cat /var/lib/mysql/anaxagoras.log 
/usr/sbin/mysqld, Version: 5.5.40-0+wheezy1 ((Debian)). started with:
Tcp port: 3306  Unix socket: /var/run/mysqld/mysqld.sock
Time                 Id Command    Argument
141026 13:21:22    37 Query show variables like 'general%'
141026 13:24:35    37 Query SET GLOBAL general_log=0




mysqladmin and a few usage examples.
mysqladmin is an agent-client suitable for administering MySQL servers.


Check whether the server is alive.
# mysqladmin --defaults-file=/etc/mysql/debian.cnf ping
mysqld is alive



See status.
# mysqladmin --defaults-file=/etc/mysql/debian.cnf status
Uptime: 10189  Threads: 2  Questions: 172  Slow queries: 0  Opens: 171  Flush tables: 1  Open tables: 41  Queries per second avg: 0.016


To view an extended status try.
# mysqladmin --defaults-file=/etc/mysql/debian.cnf extended-status |less


List processes.
# mysqladmin --defaults-file=/etc/mysql/debian.cnf processlist
+----+------------------+-----------+----+---------+------+-------+------------------+
| Id | User             | Host      | db | Command | Time | State | Info             |
+----+------------------+-----------+----+---------+------+-------+------------------+
| 40 | root             | localhost |    | Sleep   | 1940 |       |                  |
| 47 | debian-sys-maint | localhost |    | Query   | 0    |       | show processlist |
+----+------------------+-----------+----+---------+------+-------+------------------+


Kill process with ID 40 and show processes (proc).
# mysqladmin --defaults-file=/etc/mysql/debian.cnf kill 40 proc
+----+------------------+-----------+----+---------+------+-------+------------------+
| Id | User             | Host      | db | Command | Time | State | Info             |
+----+------------------+-----------+----+---------+------+-------+------------------+
| 46 | debian-sys-maint | localhost |    | Query   | 0    |       | show processlist |
+----+------------------+-----------+----+---------+------+-------+------------------+


Reload the MySQL server mysql database grant tables.
# mysqladmin --defaults-file=/etc/mysql/debian.cnf reload
or
# mysqladmin -u root -p"root_passwd" flush-privileges


Clear MySQL status variables in a MySQL instance running on host 192.0.2.10
# mysqladmin -h 192.0.2.10 -u root -p"0210_root_passwsd" flush-status


Shutdown MySQL server.
# mysqladmin --defaults-file=/etc/mysql/debian.cnf ping
mysqld is alive
# mysqladmin --defaults-file=/etc/mysql/debian.cnf shutdown
# mysqladmin --defaults-file=/etc/mysql/debian.cnf ping 2>/dev/null
# echo $?
1


Start MySQL server.
# /etc/init.d/mysql start
[ ok ] Starting MySQL database server: mysqld ..
[info] Checking for tables which need an upgrade, are corrupt or were 
not closed cleanly..




Use the standard client in an elementary database exploration.

Log in to MySQL as the root using the standard mysql client.
# mysql -u root -p"root_passwd"


Show databases.
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| foodb              |
| mysql              |
| performance_schema |
+--------------------+
4 rows in set (0.00 sec)
The mysql , information_schema and performance_schema databases come with the MySQL server and they are used by the MySQL server in its operation. The mysql database holds information about users,servers,plugins,timezone,etc and the users may write to it (eg: this is how you add a MySQL user ). The information_schema (read-only to the users) stores information about all the other databases that MySQL maintains. The performance_schema database is used by the MySQL system to provide low level execution monitoring.

Use the foodb database.
mysql> use foodb;


Show all tables in the foodb database.
mysql> show tables;
+------------------+
| Tables_in_foodb  |
+------------------+
| exits            |
+------------------+
1 row in set (0.00 sec)


Describe the schema of the 'exits' table in the 'foodb' DB.
mysql> describe exits;
+------------+------------------+------+-----+---------+-------+
| Field      | Type             | Null | Key | Default | Extra |
+------------+------------------+------+-----+---------+-------+
| su         | int(10) unsigned | NO   | PRI | NULL    |       |
| first_test | datetime         | YES  |     | NULL    |       |
| last_test  | datetime         | YES  |     | NULL    |       |
+------------+------------------+------+-----+---------+-------+
3 rows in set (0.00 sec)


Find out the number of rows in the table exits.
mysql> SELECT COUNT(*) FROM exits;
+----------+
| COUNT(*) |
+----------+
|   260472 |
+----------+
1 row in set (0.00 sec)



exit
mysql> quit
Bye
#




Binary Database Backup.
If a database contains only MyISAM tables (*.frm,*.MYD,or *.MYI) and the db.opt file you may simply copy it.

Lock tables before copying.
# mysql --defaults-file=/etc/mysql/debian.cnf -e "LOCK TABLES foodb.exits READ;"


The database tables are in /usr/lib/mysql. To copy.
# cp -rp /var/lib/mysql/foodb /bak/mysql/foodb


Unlock tables after copying.
# mysql --defaults-file=/etc/mysql/debian.cnf -e "UNLOCK TABLES;"


To restore the foodb to the same or another MySQL server
copy it to /var/lib/mysql.
# cp -rp  /bak/mysql/foodb /var/lib/mysql/foodb
If you are copying to another MySQL server and you are missing or you do not want to mess with the old /var/lib/mysql/mysql you may want to create a user for the foodb.
mysql> grant all on foodb.* to foodbuser;
mysql> set password for foodbuser = password('foodbuser_passwd');


When I did copy the foodb DB directory to another MySQL server everything worked fine, the origin MySQL server was version 5.1.41-3, and the destination MySQL was version 5.5.40-0.



mysqlhotcopy

A more robust way for doing binary backups of MyISAM tables is mysqlhotcopy --a Perl script that comes with the standard MySQL distribution.
Eg: to copy foodb to another MySQL server using mysqlhotcopy.
dest# mkdir /var/lib/mysql/foodb
orig#mysqlhotcopy --method='scp' --user=root --password=mysqlrootpasswd foodb root@192.0.2.26:/var/lib/mysql


mysqldump --Logical Backups

Good for all storage engines. Logical Backups are text files that contain SQL statements used to restore schemata and data.

Dump backup of foodb in a file.
orig# mysqldump -u root -p"root_passwd" foodb > foodb.sql


Restore the foodb to another MySQL server.
dest# mysqladmin --defaults-file=/etc/mysql/debian.cnf create foodb
dest# mysql --defaults-file=/etc/mysql/debian.cnf foodb < foodb.sql




Master-Slave Replication

Prepare Master ( anaxagoras ) and Slave ( democritus ) MySQL servers.
Enable binary logging, set a server ID number and listen on all interfaces.

Master --host anaxagoras
add in /etc/mysql/my.cnf -configuration group mysqld
[mysqld]
server-id               = 11
log_bin                 = /var/log/mysql/mysql-bin.log
bind-address            = 0.0.0.0
innodb_flush_log_at_trx_commit=1
sync_binlog             = 1  
binlog_do_db            = foodb


Restart MySQL
anaxagoras# service mysql restart


Punch Firewall holes.

Slave --host democritus
Add in /etc/mysql/my.cnf -configuration group mysqld
[mysqld]
server-id               = 12
log_bin                 = /var/log/mysql/mysql-bin.log
binlog_do_db            = foodb

Restart MySQL
democritus# service mysql restart 


Create a user for replication on the master.
anaxagoras# mysql -u root -p"root_passwd"
mysql> CREATE USER 'repuser'@'192.0.2.0/255.255.255.128';
mysql> SET PASSWORD FOR 'repuser'@'192.0.2.0/255.255.255.128' = password('repuser_password');
mysql> GRANT REPLICATION SLAVE ON *.* TO 'repuser'@'192.0.2.0/255.255.255.128';
mysql> FLUSH PRIVILEGES;


Obtain master's binary log coordinates.
mysql> USE foodb
mysql> FLUSH TABLES WITH READ LOCK;
Query OK, 0 rows affected (0.17 sec)

mysql> SHOW MASTER STATUS;
+------------------+----------+--------------+------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000003 |      107 | foodb        |                  |
+------------------+----------+--------------+------------------+
1 row in set (0.00 sec)



Copy the database with mysqldump , mysqlhotcopy , a cold copy, or something else.

Cold Super Copy.
Copy raw data files to the slave MySQL.
If you are using any InnoDB tables, shutdown MySQL
anaxagoras# mysqladmin --defaults-file=/etc/mysql/debian.cnf shutdown 
anaxagoras# rsync -avz -e ssh /var/lib/mysql/ root@democritus:/var/lib/mysql
anaxagoras# scp /etc/mysql/debian.cnf root@democritus:/etc/mysql/


Remove master' s lock.
mysql> UNLOCK TABLES;


Configure the slave MySQL server and start the replication.
democritus# mysql -u root -p"root_passwd"
mysql> CHANGE MASTER TO MASTER_HOST='anaxagoras', MASTER_USER='repuser', MASTER_PASSWORD='repuser_password', MASTER_LOG_FILE='mysql-bin.000003', MASTER_LOG_POS=107;
mysql> START SLAVE;



See the status in the slave MySQL server.
mysql> SHOW SLAVE STATUS\G




mysqlcheck --A MySQL tables maintance program.

Check all tables in all databases.
# mysqlcheck --defaults-file=/etc/mysql/debian.cnf -A


Analyze tables in database foodb.
# mysqlcheck --defaults-file=/etc/mysql/debian.cnf --analyze foodb


Optimize tables in database foodb.
# mysqlcheck --defaults-file=/etc/mysql/debian.cnf -0 foodb


Repair tables in foodb. Make backups of the tables before 'repairing' them.
# mysqlcheck --defaults-file=/etc/mysql/debian.cnf --debug-info --auto-repair foodb




MySQL administration notes



20141021

DELETE tun interfaces

A quick note on killing a bug in a previous recipe and deleting protocol 41 tunnel interfaces in linux.

datun is an interface used as one of the edges in a 6in4 tunnel set with
ip tunnel add datun mode sit remote 192.0.2.49 local 198.51.100.50 ttl 64 
ip link set datun up


seen as
# ifconfig datun
datun Link encap:IPv6-in-IPv4 


and taken down with
# ip link set datun down
at the 6in4 tunnels to the IPv6 Internet how-to, even in places we needed to delete the tunnel instead of putting it down, causing all kinds of errors and confusion.

To delete a tunnel interface.
ip tunnel del datun


Instead of "restarting" the 6in4 tunnel it may be better to destroy it and set it again.
# ip tunnel del datun
# /etc/network/if-up.d/ipv6-tunnel.sh




delete tun interface

20141020

tripwire notes

Yet another tripwire ( as in the open source file integrity checker for Unix Systems ) how-to for debian , like tripwire ... but, hopefully, easier to follow.

Assuming you trust your repositories, your distribution, etc
# apt-get install tripwire
and then click the no, no, and OK buttons.

Ideally, the tripwire binaries and the tripwire database are stored in a read only medium that can be mounted as read-write for updates. I would use an SD card or some other medium that I can set "mechanically" to read-only. Some administrators put the binaries and the DB in an NFS.I think that putting the binaries and the DB in an NFS would increase the attack surface. If you are not in the mood or do not have the resources to take the extra steps to secure further the integrity of the tripwire binaries and the tripwire DB at least save copies of the files and their cryptographically secure checksums in other hosts.

In debian the tripwire binaries are statically linked and located in /usr/sbin and the DB is located in /var/lib/tripwire.
# sha256sum /usr/sbin/tripwire |tee ~/twsums
0e4791bb58dfc4095dba902621b72111d61bf1838d77aff4ae00d3c7432d5739  /usr/sbin/tripwire
# sha256sum /usr/sbin/tw* |tee -a ~/twsums
bc01ac66aa421d2e5324983150bea573b2e2d3ee004293501b0dcc4ce1560898  /usr/sbin/twadmin
e1b097eaf28f3ec54114cba7cc82a1ab4122a9fb82590422d9820711c884e5e9  /usr/sbin/twprint
# sha256sum /usr/sbin/siggen |tee -a ~/twsums 
e5e72b264f9b4fa86aa88e0f893b6031457e30b510f28bcb31ea1296b38566bd  /usr/sbin/siggen


Tripwire uses $HOSTNAME a lot in the configuration and policy files. Make sure that you are happy with hostname, if not change the hostname before continuing the tripwire configuration.

Create a site key.
# cd /etc/tripwire/
# twadmin --generate-keys --site-keyfile site.key
# chmod 400 site.key
The site key is used to secure the integrity of the tripwire configuration files.

Create a local key.
# twadmin --generate-keys --local-keyfile `hostname`-local.key
# chmod 400 *cal.key
The local key is used to protect the integrity of the local tripwire database.

Create and sign tw.cfg --the tripwire configuration file.
# stor twcfg.txt
# vi twcfg.txt 
# twadmin --create-cfgfile --cfgfile tw.cfg --site-keyfile site.key twcfg.txt
Please enter your site passphrase: 
Wrote configuration file: /etc/tripwire/tw.cfg


Create and sign tw.pol --the tripwire policy file.
# stor twpol.txt
# vi twpol.txt
# twadmin --create-polfile --cfgfile tw.cfg --site-keyfile site.key twpol.txt
Please enter your site passphrase: 
Wrote policy file: /etc/tripwire/tw.pol


Make the policy and configuration files readable and writable only by the root user.
# chmod 600 *txt
# chmod 600 *cfg
# chmod 600 *pol


Initialize the tripwire database.
# tripwire --init
Please enter your local passphrase: 
Parsing policy file: /etc/tripwire/tw.pol
Generating the database...
*** Processing Unix File System ***
# ...
Wrote database file: /var/lib/tripwire/anaxagoras.twd
The database was successfully generated.


Test that tripwire can send email.
# tripwire --test --email example@example.net


Check integrity and produce report.
# tripwire --check


View report.
# twprint -m r --twrfile /var/lib/tripwire/report/hostname-latest.twr |less


It is highly unlikable that you are using all the files in the "Root config files" rulename in the debian default twpol.txt. Also you may want to adjust the "Devices & Kernel information" rulename since /proc (meaning recursive /proc/*) may be too much to track in normal servers.

Adjust the tripwire policy and initialize a new tripwire database.
# stor twpol.txt
# vi twpol.txt
# twadmin -m P -S site.key twpol.txt 
# tripwire --init


Check for integrity, create a report and OK changes if any.
Once the editor opens look for [x] and delete the x if you are not OK with that change.
# tripwire --check --interactive
Integrity check complete.
Please enter your local passphrase: 
Wrote database file: /var/lib/tripwire/anaxagoras.twd


You may enter an `interactive` mode from a report as well. eg:
# tripwire --update --twrfile /var/lib/tripwire/report/hostname-date-time.twr
and again look for [x] and delete the x if you are not OK with that change.

Email alerts.
To email an alert we need to add an emailto definition to at least one rulename.
So we need to update the tripwire policy. eg:
#
# Critical Libraries
#
(
  rulename = "Root file-system libraries",
  severity = $(SIG_HI),
  emailto = root,
  emailto = systembot@ares.ipduh.rox
)
{
        /lib                    -> $(SEC_BIN) ;
}

If /lib is changed an alert will be sent to root and systembot@ares.ipduh.rox.


Check integrity, produce report and email alerts.
# tripwire --check --email-report


The debian package sets a cronjob that creates reports and emails alerts daily.
#!/bin/sh -e

tripwire=/usr/sbin/tripwire

[ -x $tripwire ] || exit 0

umask 027

$tripwire --check --quiet --email-report


View the tripwire database.
# twprint -m d --print-dbfile |less


View tripwire information for a file eg:/var/test
# twprint -m d --print-dbfile /lib/test


The system used in this how-to.
ii  tripwire                           2.4.2.2-2                     amd64        file and directory integrity checker
# cat /etc/issue /etc/debian_version 
Debian GNU/Linux 7 \n \l

7.6





Links:
The Design and Implementation of Tripwire: A File System Integrity Checker





Tripwire how-to

20141018

apache disable SSLv3

Notes on disabling SSLv3 in apache.

Test if SSLv3 is available.
$ openssl s_client -connect ipduh.com:443 -ssl3


In Debian SSLv2 is disabled by default but SSLv3 is available.

# grep SSLProtocol /etc/apache2/mods-available/ssl.conf
SSLProtocol all -SSLv2


To disable SSLv3 add '-SSLv3' in /etc/apache2/mods-available/ssl.conf
# vi /etc/apache2/mods-available/ssl.conf


If you are using SSL Virtual Hosts you may need to add
SSLProtocol All -SSLv2 -SSLv3
in each VirtualHost definition.

Restart Apache
# /etc/init.d/apache2 restart


Test again if SSLv3 is disabled.
$ openssl s_client -connect ipduh.com:443 -ssl3
CONNECTED(00000003)
140330958718632:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1258:SSL alert number 40
140330958718632:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:596:




apache disable SSLv3

20141017

libguestfs notes

Libguestfs basics.

Install.
# apt-get install libguestfs-tools 
# apt-get install guestfish



guestfish

The libguestfs Filesystem Interactive SHell.

An example: explore, read and write to disk image file within the libguestfs VM.
# guestfish --rw -a /home/vm/anaxagoras.qcow2
> run
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ 00:00
> list-filesystems
/dev/vda1: ext4
/dev/vda2: unknown
/dev/vda5: swap
> mount /dev/vda1 /


Add a file to the disk image file system.
> touch /etc/guestfish_play
> edit /etc/guestfish_play
> quit
#



virt-cat

Display files in a virtual machine.

eg:
# virsh list
 Id    Name                           State
----------------------------------------------------
 9     anaxagoras                     running

# virt-cat anaxagoras /etc/issue
Debian GNU/Linux 7 \n \l
# virt-cat anaxagoras /etc/hostname
anaxagoras




guestmount

Mount a guest filesystem on the host using FUSE and libguestfs

Install
# apt-get install guestmount


Mount rw a filesystem contained in a disk image file.
# mkdir /mnt/anax
# guestmount -a /home/vm/anaxagoras.qcow2 -m /dev/vda1 --rw /mnt/anax/


# cat /mnt/anax/etc/guestfish_play
hi
# echo "hi kosme" > /mnt/anax/etc/guestfish_play
# mv /mnt/anax/etc/guestfish_play /mnt/anax/etc/guestmount_play
# umount /mnt/anax


guestmount is and looks traditionally-scriptable. However, guestfish is as or more scriptable. Also, libguestfs has cute C , Perl and Python APIs.


virt-df

List free space on virtual filesystems.
# virt-df anaxagoras
Filesystem                           1K-blocks       Used  Available  Use%
anaxagoras:/dev/sda1                  19751804     840608   17907832    5%



virt-filesystems

List filesystems, partitions, block devices, LVM in a virtual machine or a disk image file.

eg:
# virt-filesystems --long --parts --blkdevs -a /home/vm/anaxagoras.qcow2 -h
Name       Type       MBR  Size  Parent
/dev/sda1  partition  83   19G   /dev/sda
/dev/sda2  partition  05   1.0K  /dev/sda
/dev/sda5  partition  82   880M  /dev/sda
/dev/sda   device     -    20G   -



or

# virt-filesystems --long -h --all -a anaxagoras.qcow2 
Name       Type        VFS      Label  MBR  Size  Parent
/dev/sda1  filesystem  ext4     -      -    19G   -
/dev/sda2  filesystem  unknown  -      -    1.0K  -
/dev/sda5  filesystem  swap     -      -    880M  -
/dev/sda1  partition   -        -      83   19G   /dev/sda
/dev/sda2  partition   -        -      05   1.0K  /dev/sda
/dev/sda5  partition   -        -      82   880M  /dev/sda
/dev/sda   device      -        -      -    20G   -



virt-list-filesystems

List filesystems in a virtual machine or disk image.

eg:
# virt-list-filesystems anaxagoras.qcow2 
/dev/sda1
# virt-list-filesystems anaxagoras
/dev/sda1






virt-resize

Resize a virtual disk image file.

Eg: Expand the 20GB anaxagoras qcow2 disk image file to a 30GB qcow2 disk image file.
# truncate -r anaxagoras.qcow2 anaxagoras30G.qcow2
# truncate -s +10G anaxagoras30G.qcow2
# virt-resize --expand /dev/sda1 anaxagoras.qcow2 anaxagoras30G.qcow2 
Examining anaxagoras.qcow2 ...
**********

Summary of changes:

/dev/sda1: This partition will be resized from 19.1G to 29.1G.  The 
    filesystem ext4 on /dev/sda1 will be expanded using the 'resize2fs' 
    method.

/dev/sda2: This partition will be left alone.

**********
Setting up initial partition table on anaxagoras30G.qcow2 ...
Copying /dev/sda1 ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ 00:00
Copying /dev/sda2 ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ 00:00
Expanding /dev/sda1 using the 'resize2fs' method ...

Resize operation completed with no errors.  Before deleting the old 
disk, carefully check that the resized disk boots and works correctly.

Test resized image.
# cd /etc/libvirt/qemu/
# stor anaxagoras.xml
# virsh
virsh # edit anaxagoras
virsh # define anaxagoras.xml
virsh # start anaxagoras
virsh # quit
# ssh anaxagoras
root@anaxagoras:~# df -h
Filesystem                                              Size  Used Avail Use% Mounted on
rootfs                                                   29G  822M   27G   3% /
udev                                                     10M     0   10M   0% /dev
tmpfs                                                   202M  172K  202M   1% /run
/dev/disk/by-uuid/8ca4bd34-120c-45ff-bd0b-86d8de552d10   29G  822M   27G   3% /
tmpfs                                                   5.0M     0  5.0M   0% /run/lock
tmpfs                                                   579M     0  579M   0% /run/shm


More virt-.* tools.
virt-alignment-scan    virt-filesystems       virt-ls                virt-tar-in
virt-cat               virt-format            virt-make-fs           virt-tar-out
virt-clone             virt-host-validate     virt-pki-validate      virt-viewer
virt-convert           virt-image             virt-rescue            virt-win-reg
virt-copy-in           virt-inspector         virt-resize            virt-xml-validate
virt-copy-out          virt-install           virt-sparsify          
virt-df                virt-list-filesystems  virt-sysprep           
virt-edit              virt-list-partitions   virt-tar               


The system used.
# cat /etc/debian_version /etc/issue
7.6
Debian GNU/Linux 7 \n \l





libguestfs basics