3.12.2015

Ansible for Configuration Management and IT automation

Wow, Ansible is a great tool for configuration management and IT automation. I got this reference from here and here.
1. Install ansible on your workstation (machine which will be the controller)
$ sudo yum install ansible
$ ansible --version

2. Create env variable
$ export ANSIBLE_HOSTS=~/ansible/ansible_hosts

3. Create hosts file
$ echo -e "perf-c690\nperf-c691" > ~/ansible/ansible_hosts

4. Do a quick ping
$ ansible all -m ping --ask-pass

5. Now, let's create ssh keys
(Should typically run using your own user account, have used root here for convenience) 
# ssh-keygen -t rsa

6. Copy public key to the remote machines
$ ansible all -m copy -a "src=/root/.ssh/id_rsa.pub dest=/tmp/id_rsa.pub" --ask-pass -c paramiko

Warning: error message: "Aborting, target uses selinux but python bindings (libselinux-python) aren't installed!"
Which means we need to install libselinux-python on the remote machine prior to copying the files there. We can use ansible to do that for us.

Install libselinux-python on remote machines
$ ansible all -m yum -a "name=libselinux-python state=latest" -u root --ask-pass -c paramiko

Now we can go back to copying the public key

7. Let's add the public key to the remote machines
$ ansible all -m shell -a "cat /tmp/id_rsa.pub >> /root/.ssh/authorized_keys" --ask-pass -c paramiko

8. Test if we actually don't need any password
$ ansible all -m shell -a "hostname"

9. Now we can do or install whatever on the remote machines, let's try installing expect
$ ansible all -m yum -a "name=expect state=latest" -u root

10. We can remotely reboot all machines at once
$ ansible all -a "/sbin/reboot" -u root

3.10.2015

Install Ruby via RVM on CentOS 6.5

Got install from here.
RVM best practices here.
How to use RVM gemsets here.
How to use Bundler here.
1. Install pre-requisites
# yum groupinstall -y development

2. Install RVM
# curl -L get.rvm.io | bash -s stable

3. Create system environment for RVM
# source /etc/profile.d/rvm.sh

4. Install Ruby
# rvm reload
# rvm install 2.2.1

5. Check version of ruby installed
# ruby --version

6. To see all rubies installed
# rvm list rubies
rvm rubies
=* ruby-2.2.1 [ x86_64 ]
# => - current
# =* - current && default
#  * - default

7. To set a Ruby version as the default
# rvm use 2.2.1 --default
Using /usr/local/rvm/gems/ruby-2.2.1

8. Shortcut - create a new gemset along with a fresh .rvmrc in our project directory
# cd projects/dev/phoenix
# rvm --rvmrc --create use 2.2.1@phoenix
ruby-2.2.1 - #gemset created /usr/local/rvm/gems/ruby-2.2.1@phoenix
ruby-2.2.1 - #generating phoenix wrappers..........
Using /usr/local/rvm/gems/ruby-2.2.1 with gemset phoenix

9. To ignore rmvrc warnings ...
# rvm rvmrc warning ignore all.rvmrcs

10. Now when we cd to our project directory we get a confirmation which ruby-version and gemset we are using...
dev]# cd phoenix/
Using: /usr/local/rvm/gems/ruby-2.2.1@phoenix

11. Let's make use of bundler to make it easy to track and install the exact gems and versions for our project
# gem install bundler

12. Create a Gemfile
# bundle init
Writing new Gemfile to /root/projects/dev/phoenix/Gemfile

13. Add gems to Gemfile
# vi Gemfile
source "https://rubygems.org"
# 1. de facto XML/HTML parsing library
gem 'nokogiri'

14. Now, let's install all gems we listed in the Gemfile
# bundle install

To create and use gemsets the longer way:
a. Create gemset
# rvm gemset create my_proj

b. Use the gemset
# rvm gemset use my_proj

c. Install a gem
# gem install package_name

d. Wipe all gems installed
# rvm gemset empty my_proj

e. Delete the gemset
# rvm gemset delete my_proj

f. Check your current gemset
# rvm gemset name

2.19.2015

Open HTTP port 80 on CentOS

Got from here.
1. Look at the current iptables
# iptables --line -vnL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           udp dpt:53
2        0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           tcp dpt:53
3        0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           udp dpt:67
4        0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           tcp dpt:67
5     195K  212M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED
6        3   252 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0
7       74  5440 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
8       16   864 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22
9     170K   39M REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited

2. Inject rule, as long as it's before the REJECT line
# iptables -I INPUT 5 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT

3. Save
# service iptables save

4. Add httpd_t to the list of permissive domains (So we don't need to turn off selinux)
# yum install policycoreutils-python
# semanage permissive -a httpd_t

Perl specific installs for Centos 6.6

1. Enable RPMForge Repository

# wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
# rpm -Uvh rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
# rpm --import http://dag.wieers.com/rpm/packages/RPM-GPG-KEY.dag.txt

2. Install perl installs

MANUAL:
1. Dependency of Net::SSH::Perl
# wget http://download.opensuse.org/repositories/home:/csbuild:/Perl/CentOS_CentOS-6/x86_64/perl-Math-GMP-2.04-2.5.x86_64.rpm
# rpm -Uvh perl-Math-GMP-2.04-2.5.x86_64.rpm

2. Dependency of gnuplot for Chart::Graph::Gnuplot
# wget http://www.my-guides.net/en/images/stories/fedora12/msttcore-fonts-2.0-3.noarch.rpm
# rpm -Uvh msttcore-fonts-2.0-3.noarch.rpm

YUM:
# yum install perl-Math-BigInt-GMP
# yum install perl-Math-Pari
# yum install perl-Archive-Zip
# yum install perl-Chart
# yum install perl-DBI
# yum install perl-DBD-MySQL
# yum install perl-Data-Compare
# yum install perl-Date-Manip
# yum install perl-JSON
# yum install perl-Net-Telnet
# yum install perl-NetAddr-IP
# yum install perl-Statistics-Descriptive
# yum install perl-Text-Table
# yum install perl-XML-Simple
# yum install perl-XML-Writer
# yum install gnuplot

CPAN:
# cpan Data::Diff
# cpan HTML::TextToHTML
# cpan XML::SemanticDiff
# cpan Chart::Graph - this didn't install

Force installed:
cpan[1]> force install Net::Telnet::Brcd

2.13.2015

Install different versions of Perl on Linux using Perlbrew

Got from here and here. What is perlbrew.
1. Install perfbrew
$ curl -kL http://install.perlbrew.pl | bash

2. Modify .bashrc and add:
source ~/perl5/perlbrew/etc/bashrc

3. Initialize
$ perlbrew init

4. Select mirror
$ perlbrew mirror

5. See what's available
$ perlbrew available
  perl-5.20.1
  perl-5.18.4
  perl-5.16.3
  perl-5.14.4

6. Install perls
$ perlbrew install perl-5.20.1  # did not work for Centos 6.5
$ perlbrew --notest install perl-5.20.1

7. See what were installed
$ perlbrew list

8. Swith to an installation and set it as default
$ perlbrew switch perl-5.20.1

9. Temporarily use another version only in current shell.
$ perlbrew use perl-5.20.1
$ perl -v

10. Or turn it off completely. Useful when you messed up too deep. Or want to go back to the system Perl.
$ perlbrew off

11. Use 'switch' command to turn it back on.
$ perlbrew switch perl-5.20.1

12. Exec something with all perlbrew-ed perls
$ perlbrew exec -- perl -E 'say $]'

13. Install cpanm
$ perlbrew install-cpanm

14. Install cpan modules
$ cpanm install File::Copy::Recursive XML::Simple etc

2.07.2015

Ruby Notes

While reading RubyMonk tutorials

1. Truthiness of objects in Ruby
Only objects false and nil equates to false. Every other object like say 1, 0, "" are all evaluated to be true.

2. Use break in infinite loops
loop do
  monk.meditate
  break if monk.nirvana?
end
3. Append to an array, use << operator
[1, 2, 3, 4, 5] << 'woot'
[1, 2, 3, 4, 5, "woot"]
4. Filter elements of an array, use 'select'
names = ['rock', 'paper', 'scissors', 'lizard', 'spock']
names.select { |word| word.length > 5 }
5. Delete elements of an array, use 'delete_if'
[1,2,3,4,5,6,7,8,9].delete_if { |i| i % 2 == 0 }
6. Methods are themselves objects, and responds using 'call'
next_method_object = 1.method("next")
puts next_method_object.call
7. Beware of shallow object copies, to do a deep copy use Marshal
new_job = Marshal.load(Marshal.dump(job))
8. Use heredoc for convenient multiline strings. The dash ignores spaces before placeholder. The gsub takes care of removing spaces at beginning of each line.
def long_message
  puts <<-EOT.gsub(/^\s*/, '')
    Here goes a very long message...
    Sincerely,
    Dr. Foobear
  EOT
end
7. Use 'send' when testing private methods
# obj.send(:method_name, args)
@parser.send(:running?, test_strings)

1.21.2015

Use iperf to measure network link bandwidth

Nice writeup on how to use iperf. Another example, writeup.
1. Install iperf
$ sudo yum install iperf

2. Start server on another system:
$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 10.1.1.1 port 5001 connected with 10.0.0.1 port 46699
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.3 sec   107 MBytes  87.7 Mbits/sec

3. Start client from your system specifying server ip:
$ iperf -c 10.1.1.1
------------------------------------------------------------
Client connecting to 10.1.1.1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.0.1 port 46699 connected with 10.1.1.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.1 sec   107 MBytes  89.0 Mbits/sec

4. Use options, e.g., -t duration, -i interval between measurements
$ iperf -c 10.1.1.1 -t 20 -i 2
------------------------------------------------------------
Client connecting to 10.1.1.1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.0.1 port 46761 connected with 10.1.1.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 2.0 sec  17.6 MBytes  73.9 Mbits/sec
[  3]  2.0- 4.0 sec  22.5 MBytes  94.4 Mbits/sec
[  3]  4.0- 6.0 sec  21.2 MBytes  89.1 Mbits/sec
[  3]  6.0- 8.0 sec  22.6 MBytes  94.9 Mbits/sec
[  3]  8.0-10.0 sec  22.5 MBytes  94.4 Mbits/sec
[  3] 10.0-12.0 sec  21.2 MBytes  89.1 Mbits/sec
[  3] 12.0-14.0 sec  22.6 MBytes  94.9 Mbits/sec
[  3] 14.0-16.0 sec  22.5 MBytes  94.4 Mbits/sec
[  3] 16.0-18.0 sec  21.2 MBytes  89.1 Mbits/sec
[  3] 18.0-20.0 sec  22.5 MBytes  94.4 Mbits/sec
[  3]  0.0-20.1 sec   217 MBytes  90.6 Mbits/sec

Just want to take note: Bandwidth vs Throughput

Bandwidth : Refers how fast a device can send data over a single communication channel.
Throughput: Refers how fast a device is actually sending data over the communication channel.