20140929

Perl RegEx Cheatsheet





yaprec is Yet Another (Incomplete) Perl Regular Expressions (Reference) Cheatsheet





















... and posting it here gives me a good excuse to play with POD HTMLification.
I have been putting this reference together for a while. I will keep on working on yaprec and publish the POD once it looks a bit better and makes more sense ...



















NAME

 yaprec

Description

 Yet Another (Incomplete) Perl Regular Expressions (Reference) Cheatsheet.

Metacharacters

 \     Quote-literalize-disable the next metacharacter.
 .     Match any character except newline.
 ()    Grouping and Capturing.
 []    Bracketed Character Class.
 |     Infix 'operator'. Alternate, match one or the other.
 [^]   Negate bracketed character class

Anchors

 ^     True at beginning of string.
 $     True at the end of string.

Quantifiers

 {n}   Match the Previous exactly n times
 {n,}  Match the Previous n or more times
 {n,m} Match the previous at least n times, but not more than m times
 *     Match the previous 0 or more times   {0,}
 ?     Match the previous 0 or 1 times      {0,1}
       ? also changes any maximal quantifier to a minimal quantifier
 +     Match the previous 1 or more times   {1,}
       + also changes a maximal quantifier to a posessive quantifier (that does not backtrack)
 Maximal     Minimal     Possessive      Allowed Range
 {MIN,MAX}   {MIN,MAX}?  {MIN,MAX}?+     Must occur at least MIN times but no more than MAX times
 {MIN,}      {MIN,}?     {MIN,}?+        Must occur at least MIN times
 {COUNT}     {COUNT}?    {COUNT}?+       Must match exactly COUNT times
 *           *?          *+              0 or more times ( {0,}  )
 +           +?          ++              1 or more times ( {1,}  )
 ?           ??          ?+              0 or 1 time     ( {0,1} )

Operators

 =~    Bind the scalar exression to the right on the m// xor s// xor tr/// xor y/// operator to the left
 !~    Bind the scalar exression to the right on the m// xor s// xor tr/// xor y/// operator to the left
 m//   Match. Scalar, Boolean, and List context. Quote operator.
 s///  Substitute. Quote operator.
 qr//  Quote REx and return 'compiled' regex for future use.

The tr/// operator does not speak Real Perl REx

 tr///   Transliterate. Quote operator.
 y///    tr/// synonym

Delimeters and Quotes

 //    Search the string in the scalar EXPR for PATTERN , EXPR =~ /PATTERN/modifiers  ( m operator implied )
 ??    Once only match, EXPR =~ ?PATTERN?modifiers  ( m operator implied )
 ''    Suppress variable interpolation and the six translation escapes
 {}    Quotes
 []    Quotes
 ##    Quotes
 ()    Quotes
 <>    Quotes
 ""    Quotes
 ``    Quotes

Character Class Shortcuts

                                            In ASCII with the /a modifier
 \t  Horizontal character tabulation
 \n  Newline (LF,NL)
 \r  Return
 \f  Line feed
 \w  Word character                         [A-Za-z0-9_]
 \W  Non-(word-character)                   [^A-Za-z0-9_]
 \s  Whitespace                             [\f\n\r\t]
 \S  Non-whitespace                         [^\f\n\r\t]
 \d  Digit                                  [0-9]
 \D  Nondigit                               [^0-9]
 \h  Horizontal whitespace character
 \H  Non horizontal whitespace character
 \v  Vertical whitespace character
 \V  Non-vertical whitespace character

Pattern Modifiers

 /i   Ignore alphanumeric case
 /x   Ignore (most) whitespace and permit comments in pattern
 /c   Keep current position after a failed match
 /g   Global match/substitude as often as possible
 /cg  Allow continued search after failed /g match
 /s   Treat string as a single line and let . match newline
 /m   Treat string as multiple lines and let ^ and $ also match next to embedded newline, not just the edges of the string
 /o   Compile pattern once only
 /p   Preserve ${^PREMATCH} , ${^MATCH} , ${^POSTMATCH} as in $`, $& and $' without penalizing the entire program
 /d   Dual ASCII-Unicode mode charset behavior (old default)
 /a   ASCII charset behavior
 /aa  ASCIIer charset behavior without Unicode Casefolding
 /u   Unicode charset behavior
 /l   The runtime locale's charset behavior (defult under use locale)
 /e   Evaluate the right side as a Perl expression, use with the s// operator to create a do{Perl code} block on the right
 /ee  Like /e but adds and eval around the do{Perl code} block on the right
 /r   Return substitution and leave the original string untouched ( v >= 5.14 )

Perl REx Pattern Variables

 $`                    Text left of the Match
 ${^PREMATCH}          Like $` minus the performance penalty
 $&                    The Match
 ${^MATCH}             Like $& minus the performance penalty
 $'                    Text right of the Match
 ${^POSTMATCH}         Like $' minus the performance penalty
 $+                    Last capture group
 @-                    Array with offsets of submatches' starts
 @+                    Array with offsets of submatches' ends
 $1                    Holds the 1st captured expression
                       $1 $2 ... $n holds the nth captured expression
 $^N                   Holds the most recent capture
 $^R                   holds the result of the last (?{ CODE } expression
 %+                    Named capture buffers
 %-                    Named capture buffers
 ${^RE_DEBUG_FLAGS}    The current value of the regex debugging flags. Added in 5.10
 ${^RE_TRIE_MAXBUF}    Controls how certain regex optimisations are applied and how much memory they utilize.
                       This value by default is 65536 which corresponds to a 512kB temporary cache.

Other Alphanumeric RegEx Metasymbols and Escape Characters

More Anchors, Zero-width Assertions

 \b  Match between a "word" and a non-"word" character ( \W\w | \w\W )
     In a bracketed class [\b] means backspace.
 \B  Match except at a word boundary ( \W\W )
 \A  Match only at beginning of string
 \Z  Match only at the end of string, or before newline at the end
 \z  Match only at end of string
 \G  True at end-of-match  position  of prior m//g
 \G  Match only at pos() (eg the end-of-matching position of previous //g)

The (8) Six Translation Escapes

 \N{NAME}   Match the named character, alias, or sequence, it requires the charnames module
 \U         Uppercase until \E
 \u         Titlecase next character only
 \l         Lowercase (not foldcase) next character only
 \L         Lowercase (not foldcase) until \E.
 \Q         Quote (de-meta) metacharacters until \E.
 \F         Foldcase (not lowercase) until \E
 \E         End case (\F,\L,\U) or quotemeta (\Q) translation

The Rest Alphanumeric Metasymbols

 \0         Match character number zero (U+0,NULL,NULL)
 \NNN       Match the character given in octal, up to \377
 \n         Match the nth (n is a decimal number) capture group
 \g{n}      Match the nth (decimal) captured group  #
 \g{-n}     Relative backreference to the nth group before  #
 \a         Match the alert character (ALERT, BEL)
 \cX        Match control character Control-X
 \cZ        Match control character Control-Z
 \c[        Match control character ESC
 \c?        Match control character DEL
 \C         Match only one byte (C char) even in UTF-8
 \e         Match the escape character (ESCAPE, ESC , not backslash)
 \f         Match the form feed character (FORM FEED,FF)
 \k<GROUP>  Match the named capture group; also \k'NAME'
 \K         Keep text to the left of \K out of match
 \N         Match any character except newline.
 \o{NNN}    Match the character NNN in octal
 \p{prop}   Match any character with the prop Unicode property
 \pP        Match any character with the P Unicode property
 \P{PROP}   Match any character without the PROP Unicode property
 \PP        Match any character without the P Unicode property
 \pM        Match any character that is considered a Unicode mark character
 \PM        Match any character that is not considered a Unicode mark character
 \r         Match the return character (usually CARRIAGE RETURN, CR)
 \R         Match any linebreak grapheme (not in char classes)
 \x{ABCD}   Match the character given in hexadecimal
 \X         Match grapheme (not in char classes)

Extented RegEx Sequences

 (?#   )        Comment, Discard
 (?:   )        Noncapturing Group
 (?>   )        Possesive Group, no capturing, no backtracking
 (?a-i )        Enable (adlupimsx), Disable (-imsx) Pattern Modifiers (to outer group)
 (?a-i:)        Enable (adlupimsx), Disable (-imsx) Pattern Modifiers and make this a Noncapturing block
 (?^a  )        Reset (to the default option set) and Enable Pattern Modifiers (adlupimsx)
 (?^a: )        Group-only Parentheses plus reset (to the default option set) and enable modifiers (adlupimsx)
 (?=   )        True if lookahead assertion succeeds
 (?!   )        True if lookahead assertion fails
 (?<=  )        True if lookbehind assertion succeeds
 (?<!  )        True if lookbehind assertion fails
 (?| | )        Branch reset for numbered groups
 (?<NAME>   )   Named Capture Group
 (?'NAME'   )   Named Capture Group
 (?{  })        Execute Embeded Perl code , result of last (?{Perl code}) in $^R
 (??{ })        Match REx from embedded Perl code. It allows you to enter code that evaluates to a valid pattern.
 (?NUMBER)      Call the independent subexpression in group NUMBER; also (?+NUMBER) , (?-NUMBER) , (?0) , (?R)
 (?&NAME)       Recurse on group NAME; also (?P>NAME)
 (?(COND)  |  ) Conditional Interpolation, match with if-then pattern , $^R is not set
 (?(DEFINE)   ) Define name groups for later "RegEx subroutine."
                Invocation as (?&NAME) , like (?(COND) ) where COND is always false.
 (*VERB)        Backtracking control verb; also (*VERB:NAME)
 VERBs          (*ACCEPT), (*COMMIT) , (*FAIL) or (*F) ,
                (*MARK:NAME) or (*:NAME) , (*PRUNE) or (*PRUNE:NAME)
                (*SKIP) or (*SKIP:NAME) , (*THEN) or (*THEN:NAME)

Apply Modifiers to all patterns in you scope and other 'use's

 use re "/u";     Unicode mode
 use re "/d";     Dual ASCII–Unicode mode
 use re "/l";     8–bit locale mode
 use re "/a";     ASCII mode, plus Unicode casefolding
 use re "/aa";    ASCIIer mode, without Unicode casefolding
 use re "debug";  See Perl REx preapration and trace. Like -Dr, it needs Perl compiled with -DDEBUGGING
 use utf8;        use UTF-8 in REx literals as well

Pattern Related Perl Builtins

 pos SCALAR        When used with m//g allows you to read or set the offset of the last match.
 quotemeta EXPR    Backslashes [^A-Za-z_0-9], like \Q...\E
 reset;            One-match searches ?pattern? are reset to match again.
 study SCALAR      Analyze string and attempt to optimize matching. It may or may not save time.

POSIX-style Character Classes

 [[:class:]]    Meaning and Equivalencies                        With /a
 [\p{class}]
 alnum          Any alphanumeric character                    [A-Za-z0-9]
                \p{X_POSIX_Alnum}                             \p{POSIX_Alnum}
 alpha          Any alphabetic character                      [A-Za-z]
                Includes Nonletters (Other_Alphabetic)        \p{POSIX_Alpha}
                \p{X_POSIX_Alpha}
 ascii          ASCII 0-127                                   ASCII 0-127
                \p{ASCII}                                     \p{ASCII}
 blank          Any horizontal whitespace                     space or horizontal tab
                \p{X_POSIX_Blank} \p{HorizSpace} \h           \p{POSIX_Blank}
 cntrl          Control Characters                            ASCII 0-31 and 127-159
                ASCII 0-31 and 127-159                        \p{POSIX_Cntrl}
                \p{X_POSIX_Cntrl}
 digit          Characters with the Digit Propery             [0-9]
                and numerical value 0 to 9                    \p{POSIX_Digit} or \d
                \p{X_POSIX_Digit} or \d
 graph          Complement of Control, Surrogate, and         ASCII 33-126
                Unassigned non-whitespace characters.         \p{POSIX_Graph}
                \p{X_POSIX_Graph}
 lower          Lower case characters with                    [a-z]
                GC: Lowercase_Letter                          \p{POSIX_Lower}
                and property Other_Lowercase
                \p{X_POSIX_Lower}
 print          Any graph or non-CNTRL blank character        \p{POSIX_Print}
                \p{X_POSIX_print}
 punct          Any character with                            \p{POSIX_Punct}
                GC: Punctuation
                and the 9 ASCII characters in Symbol
                \p{X_POSIX_Punct} or \pP
                Not equivalent to \p{punct}
 space          Characters with the Whitespace Property       All Whitespace ASCII chars
                \p{X_POSIX_Space} , [\v\h] or [\s\cK]         tab, vertical tab,
                                                              LF , FF, CR and space
                                                              \p{POSIX_Space}
 upper          All uppercase (not Titlecase) characters.     [A-Z]
                GC: Uppercase_Letter or Other_Uppercase       \p{POSIX_Upper}
                \p{X_POSIX_Upper} or \p{Uppercase}
 word           alnum or                                      [a-zA-Z0-9_]
                GC: Mark or Connector_Punctuation             \p{POSIX_Word}
                \p{X_POSIX_Word} or \w
 xdi            Hexadecimal Digits                            [0-9A-Fa-f]
                \p{X_POSIX_XDigit} \p{Hex_Digit}              \p{POSIX_XDigit}
                \p{hex}                                       \p{ASCII_Hex_Digit}
                                                              \p{ahex}

(

The tr/// operator does not speak Real Perl REx

 tr///   Transliteration
 y///    tr/// synonym

tr/// modifiers

 /c  Complement SEARCHLIST
 /d  Delete found but unreplaced characters
 /s  Squash duplicate replaced characters
 /r  Retrun transliteration and leave the original string untouched

)

Author

 g0 L<http://ipduh.com/contact>;

Further

 perlre
 perlreref
 perlreapi
 perlreguts
 perlrequick
 perlrecharclass
 perlrebackslash
 Programming Perl


Thanks

 to the Authors of Perl and Programming Perl for putting them together.


Perl regex cheatsheet



20140928

Greek HAM Exam Notes



Notes in Greek for the Harmonised Amateur Radio Examination Certificate ( HAREC ).

These are notes I put together ~a year ago (1384034400), when I was studying for my HAREC certificate.

A HAREC certificate is needed in Greece and other Countries for an official-legal HAM call-sign starting with the SV prefix ( eg mine is SV1PDM ) and permits for antennas allowed the maximum legal ERP in the HAM frequencies.

A couple of friends found my notes useful while studying for their HAREC
and a lot of acquaintances and aspired HAMs asked me to send them or send to others these notes.


I put this note here because I keep on loosing them.



Greek ham exam notes

20140921

samba

Samba Notes, simple file shares setup, like install samba on debian but for a wheezy server and a bit more thorough.

In this setup I just need a little security ( I don' t want someone to accidently delete stuff from the LAN ) and two file shares, one file share with data stored in a RAID 1 array and one file share with insignificant data.

the server
# cat /etc/issue /etc/debian_version 
Debian GNU/Linux 7 \n \l

7.6


install samba
# apt-get install samba


backup original smb.conf and adjust smb.conf
# cp /etc/samba/smb.conf /etc/samba/smb.conf.0
# vi /etc/samba/smb.conf


save `smb.conf original (debian package default) and initial changes` to smb.conf.master
# cp /etc/samba/smb.conf /etc/samba/smb.conf.master


test smb.conf.master and create smb.conf
# testparm -s /etc/samba/smb.conf.master > /etc/samba/smb.conf
Load smb config files from /etc/samba/smb.conf.master
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
Processing section "[homes]"
Processing section "[share]"
Loaded services file OK.
Server role: ROLE_STANDALONE


the smb.conf
# cat /etc/samba/smb.conf
[global]
 server string = %h samba
 interfaces = 192.0.2.29/32, eth0
 map to guest = Bad User
 obey pam restrictions = Yes
 pam password change = Yes
 passwd program = /usr/bin/passwd %u
 passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
 unix password sync = Yes
 syslog = 0
 log file = /var/log/samba/log.%m
 max log size = 1000
 dns proxy = No
 usershare allow guests = Yes
 panic action = /usr/share/samba/panic-action %d
 idmap config * : backend = tdb

[homes]
 comment = Home Directories
 valid users = %S
 create mask = 0700
 directory mask = 0700
 browseable = No

[share]
 comment = insigdato
 path = /insigdato
 read only = No
 create mask = 0755
 guest ok = Yes


Create the system user smbuser
# adduser smbuser --system --shell /usr/sbin/nologin --home /home/sigdato 
Warning: The home dir /home/sigdato you specified already exists.
Adding system user `smbuser' (UID 110) ...
Adding new user `smbuser' (UID 110) with group `nogroup' ...


Add the smbuser to samba
# smbpasswd -L -a smbuser
New SMB password:
Retype new SMB password:
Added user smbuser.


Adjust ownership of /insigdato
# chown -R smbuser.nogroup /insigdato/


Restart samba
# /etc/init.d/samba restart


Check the new share from another host in the LAN
# apt-get install smbclient
# smbclient -U smbuser -L 192.0.2.29
Enter smbuser's password: 
Domain=[WORKGROUP] OS=[Unix] Server=[Samba 3.6.6]

 Sharename       Type      Comment
 ---------       ----      -------
 share           Disk      insigdato
 IPC$            IPC       IPC Service (kernel samba)
 smbuser         Disk      Home Directories
Domain=[WORKGROUP] OS=[Unix] Server=[Samba 3.6.6]

I can see the insigdato share ( 192.0.2.29:/insigdato ) and the share on the RAID 1 Array ( the smbuser 's home directory )

Mount a samba share in a linux system
# apt-get install samba-common
# mkdir /insigdato_test
# mount -v -t cifs //192.0.2.29/share /insigdato_test -o username=smbuser,password=pas,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=clientuser
where, pas is the samba password for smbuser
and clientuser is the client host unix user that owns the mounted samba share


the insigdato share should be visible and browsable from all hosts in the LAN but one needs to login as smbuser in order to write.



samba file shares

20140920

gitweb 404 Projects Not found

gitweb working for years and suddenly says 404 Projects Not Found

disable selinux
# echo 0 > /selinux/enforce
nope ...

look around ...
nope ...

read Perl and carp print debug ...
nope ...

create a projects list
# cd /data/software/git
# ./create_project_list.sh > projects.list


with the following script
# cat create_project_list.sh 
#!/bin/sh

export GITWEB_CONFIG="gitweb_make_index.perl"
export GATEWAY_INTERFACE="CGI/1.1"
export HTTP_ACCEPT="*/*"
export REQUEST_METHOD="GET"
export QUERY_STRING="a=project_index"

#perl -- /var/www/cgi-bin/gitweb.cgi
perl -- /usr/share/gitweb/index.cgi

and adjust /etc/gitweb.conf
# grep projects.list /etc/gitweb.conf 
#$projects_list = $projectroot;
$projects_list = "/data/software/git/projects.list";
nope ...

check out what was wrong
# ls -l / |grep data
drwx------  10 root  root   4096 Mar 27 19:33 data
I am silly for doing it at the first place ( can see where in history ) and for complicating the debugging

# chmod 755 /data
it works.

Classic overcomplication.





KISS when debugging



NFS notes

basic NFS notes

server: 192.0.2.29
client : 192.0.2.13

server
server# cat /etc/issue /etc/debian_version 
Debian GNU/Linux 7 \n \l

7.6


client
client# cat /etc/issue /etc/debian_version
Debian GNU/Linux 7 \n \l

7.5


Install the kernel space NFS server
server# apt-get install nfs-kernel-server


Allow read only access from 192.0.2.13 to the /insigdato/movies directory
server# echo "/insigdato/movies/  192.0.2.13(ro,sync,no_subtree_check)" >> /etc/exports


Restart nfsd
server# /etc/init.d/nfs-kernel-server restart


Install nfs support files in the client
client# apt-get install nfs-common


Mount the server directory /insigdato/movies in the client
client# mkdir /movies
client# mount 192.0.2.29:/insigdato/movies /movies


adjust fstab in the client
client# echo "192.0.2.29:/insigdato/movies /movies nfs  ro,sync,hard,intr  0  0" >> /etc/fstab
test client fstab
client# umount /movies
client# mount -a




basic NFS debian wheezy

gnome-terminal UTF8

gnome-terminal UTF8

following up on debian argghh! my debian gnome terminal is UTF8 illiterate ...



Set encoding to UTF8 in gnome-terminal






gnome-terminal encoding can be set to UTF8 by default with gconftool
$ gconftool --set --type=string /apps/gnome-terminal/profiles/Default/encoding en_US.UTF-8


showunicode



gnome-terminal Unicode UTF-8

20140919

basic LVM

Linux Logical Volume Manager --LVM basic notes

the system
# cat /etc/issue
Debian GNU/Linux 7 \n \l
# uname -r
3.2.0-4-amd64


Install the LVM, the Linux Kernel Device Mapper userspace library and the partition manipulator.
# apt-get install lvm2 dmsetup parted


Prepare physical volume /dev/sdc for use by the LVM.
Remove MBR and partition table ( most of my drives are used )
# dd if=/dev/zero of=/dev/sdc bs=512 count=1


Initialize sdc for use as an LVM Physical Volume.
# pvcreate /dev/sdc
  Can't open /dev/sdc exclusively.  Mounted filesystem?
# pvcreate -f /dev/sdc
  Can't open /dev/sdc exclusively.  Mounted filesystem?
nope, it's not mounted --not even partitioned ... 5 minutes of looking around ... fakit
# shutdown -r now


...

initialize sdc for use as an LVM Physical Volume.
# pvcreate /dev/sdc
  Writing physical volume data to disk "/dev/sdc"
  Physical volume "/dev/sdc" successfully created


View LVM Physical Volumes
# pvdisplay 
  "/dev/sdc" is a new physical volume of "1.82 TiB"
  --- NEW Physical volume ---
  PV Name               /dev/sdc
  VG Name               
  PV Size               1.82 TiB
  Allocatable           NO
  PE Size               0   
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               LY8rp6-IxmV-Cd4Z-AalR-ZN45-6cp5-U2iiVM
   



Create LVM Volume Group 'insigdato'
# vgcreate insigdato /dev/sdc
  Volume group "insigdato" successfully created


display volume groups and their attributes
# vgdisplay
  --- Volume group ---
  VG Name               insigdato
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               1.82 TiB
  PE Size               4.00 MiB
  Total PE              476932
  Alloc PE / Size       0 / 0   
  Free  PE / Size       476932 / 1.82 TiB
  VG UUID               2j2iVB-ugas-kQyT-5AC1-RcIk-1Lju-sY39Ec



Add LVM logical volume "movies" to the LVM volume group "insigdato"
# lvcreate --name movies --size 150G insigdato
  Logical volume "movies" created


Add LVM logical volume "mp3" to LVM VG insigdato
# lvcreate --name mp3 --size 50G insigdato


Add LVM logical volume "OS.iso" to LVM VG insigdato
# lvcreate --name OS.iso --size 50G insigdato


Display logical volumes
# lvdisplay


format the mp3 logical volume
# mkfs.ext3 /dev/insigdato/mp3


format the movies LVM logical volume
# mkfs.xfs -f /dev/insigdato/movies


format /dev/insigato/OS.iso
# mkfs.ext4 /dev/insigdato/OS.iso


create mount points, adjust fstab and mount
# mkdir /insigdato
# mkdir /insigdato/movies
# mkdir /insigdato/OS.iso
# mkdir /insigdato/mp3
# echo "/dev/insigdato/movies  /insigdato/movies  xfs   rw,noatime   0 0" >> /etc/fstab   
# echo "/dev/insigdato/OS.iso  /insigdato/OS.iso  ext4   rw,noatime   0 0" >> /etc/fstab   
# echo "/dev/insigdato/mp3  /insigdato/mp3  ext3   rw,noatime   0 0" >> /etc/fstab   
# for i in `ls /dev/insigdato`;do mount /dev/insigdato/$i /insigdato/$i;done


Check the LVM physical volume
# pvdisplay 
  --- Physical volume ---
  PV Name               /dev/sdc
  VG Name               insigdato
  PV Size               1.82 TiB / not usable 1.09 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              476932
  Free PE               412932
  Allocated PE          64000
  PV UUID               LY8rp6-IxmV-Cd4Z-AalR-ZN45-6cp5-U2iiVM



hmm, 150GiB for movies? ... need more

extend the movies LVM logical volume
# lvextend -L165G /dev/insigdato/movies
  Extending logical volume movies to 165.00 GiB
  Logical volume movies successfully resized


Extend the xfs filesystem used by movies
# xfs_growfs /dev/insigdato/movies
meta-data=/dev/mapper/insigdato-movies isize=256    agcount=16, agsize=2457600 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=39321600, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=19200, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 39321600 to 43253760
while mounted and without affecting its contents ...

LVM introduction notes

20140918

degraded software RAID 1

Restore a degraded software RAID 1 array on a linux debian wheezy system. That is, replace the failed Hard Drive.

Someone removed the broken Hard Drive from the system and added a working used Hard Drive with the same specs at the same Serial ATA port with the failed Hard Drive.

The system
# uname -a
Linux kernel 3.2.0-4-amd64 #1 SMP Debian 3.2.60-1+deb7u1 x86_64 GNU/Linux
# cat /etc/debian_version 
7.6


Multiple Devices Device(s) Status
~# cat /proc/mdstat 
Personalities : [raid1] 
md0 : active raid1 sda2[0]
      1949474624 blocks super 1.2 [2/1] [U_]
      
unused devices: 



What we are working with?
# grep -v "#" /etc/fstab
UUID=a16c558f-47ec-40ca-8989-5e12ce34a5af /               ext4    errors=remount-ro 0       1
UUID=7e96f64f-2f0b-40ac-9de2-4b4f3ff95376 none            swap    sw              0       0
UUID=15ee3ef4-9097-4b04-a154-076972d2f560 none            swap    sw              0       0
/dev/sdc1       /media/usb0     auto    rw,user,noauto  0       0


# blkid 
/dev/sda2: UUID="fc20fbc0-faef-29d6-9666-35454cbf4447" UUID_SUB="852ff6bf-04a5-bbcd-5b3a-b542a61372a6" LABEL="kernel:0" TYPE="linux_raid_member" 
/dev/sda5: UUID="7e96f64f-2f0b-40ac-9de2-4b4f3ff95376" TYPE="swap" 
/dev/md0: UUID="a16c558f-47ec-40ca-8989-5e12ce34a5af" TYPE="ext4" 
/dev/sdb2: UUID="CEE46C0AE46BF2DF" TYPE="ntfs" LABEL="System Reserved" 



# fdisk -l

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00029b1e

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1      3899215870  3907028991     3906561    5  Extended
/dev/sda2   *        2048  3899213823  1949605888   fd  Linux raid autodetect
/dev/sda5      3899215872  3907028991     3906560   82  Linux swap / Solaris

Partition table entries are not in disk order

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0xdf90f657

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048      206847      102400    7  HPFS/NTFS/exFAT
/dev/sdb2          206848   409602047   204697600    7  HPFS/NTFS/exFAT
/dev/sdb3       409602048  3907024895  1748711424    7  HPFS/NTFS/exFAT

Disk /dev/md0: 1996.3 GB, 1996262014976 bytes
2 heads, 4 sectors/track, 487368656 cylinders, total 3898949248 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

It seems like I am going to hose an old windows system drive (sdb).

A look at the superblock information for /dev/sdb
# mdadm --examine /dev/sdb
/dev/sdb:
   MBR Magic : aa55
Partition[0] :       204800 sectors at         2048 (type 07)
Partition[1] :    409395200 sectors at       206848 (type 07)
Partition[2] :   3497422848 sectors at    409602048 (type 07)


Copy the partition table of sda to sdb
# sfdisk -d /dev/sda | sfdisk --force /dev/sdb


write grub to sdb
# grub-install /dev/sdb
Installation finished. No error reported.


look at the new sdb partition table
# fdisk -l /dev/sdb

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0xdf90f657

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1      3899215870  3907028991     3906561    5  Extended
Partition 1 does not start on physical sector boundary.
/dev/sdb2   *        2048  3899213823  1949605888   fd  Linux raid autodetect
/dev/sdb5      3899215872  3907028991     3906560   82  Linux swap / Solaris

Partition table entries are not in disk order


Add sdb to the RAID 1 array
# mdadm --add /dev/md0 /dev/sdb2


check MD status
# cat /proc/mdstat 
Personalities : [raid1] 
md0 : active raid1 sdb2[2] sda2[0]
      1949474624 blocks super 1.2 [2/1] [U_]
      [>....................]  recovery =  0.3% (7625152/1949474624) finish=431.5min speed=75000K/sec
      
unused devices: 





replace failed HD in a degraded linux RAID 1 array

20140909

a nice Perl regex debugger

A nice Perl Regex debugger -- Regexp::Debugger

Install
# cpan Regexp::Debugger


cpan or cpanm should put rxrx in your path eg:
/usr/local/bin/rxrx
invoke with
$ rxrx
or
$ perl -MRegexp::Debugger -E 'Regexp::Debugger::rxrx(@ARGV)'


a usage example:
$ rxrx
enter a pattern in the "pattern register" eg:
/\s.*(fo{1,2}\s*bar)[\.\,\:\s]*/i
enter a string to match against
'The Foo bar.'
enter m to Match current string against current pattern --start the stepping process
m
Now hit n for single steps xor m for partial match steps --all the steps required up to the next partial match at once. These are not the only stepping options. Anyways, it is a nice debugger with colors, explanations, forward, backward and jump stepping.

A few pictures with the above example.





How coool is that!
( it does not show the /i modifier but it 'obviously' knows about it )

It is easier to have the debugger reading your Regular Expressions from a file or a perl program. The debugger will find them.
eg:
$ echo '"The Foo Bar." =~ /\s.*(fo{1,2}\s*bar)[\.\,\:\s\;\?]*/i;' > rxtest.pl
$ echo '"say Foo Bar;" =~ /\s.*(fo{1,2}\s*bar)[\.\,\:\s\;\?]*/i;' >> rxtest.pl
$ rxrx rxtest.pl


sound|video play style controls description:s > , n > , m >> , c >FullMatch_END , - < , f >FAIL

but better
________________________________________________/ Help \______

  Motion:    s : step forwards
             - : step backwards
             m : continue to next partial match
             f : continue to next partial failure
             c : continue to end of full match
          : repeat last motion

  Display:   v : change to visualization
             e : change to event log
             h : change to heatmaps
             j : change to JSON representation
             d : describe the regex in detail

  Snapshot:  V : take snapshot of current visualization
             E : take snapshot of current event log
             H : take snapshot of current heatmaps
             J : take snapshot of current JSON representation
             D : take snapshot of regex description

  Control:   q : quit debugger and continue program
             x : exit debugger and terminate program

______________________________________________________________



https://metacpan.org/pod/Regexp::Debugger
http://search.cpan.org/~dconway/Regexp-Debugger-0.001020/lib/Regexp/Debugger.pm



A nice Perl Regex Debugger