RVM Implode

My rvm installation and ruby got messed up after a change in home directory mounts at work. Looks like rvm had referenced everything back in my old home directory full path. The only way I found to easily fix this issue is to re-install rvm, ruby and all gems I need including rails.

1. Remove every trace of rvm
$ rvm implode
Are you SURE you wish for rvm to implode?
This will recursively remove /auto/home13/delacs/.rvm and other rvm traces?
(anything other than 'yes' will cancel) > yes
Removing rvm-shipped binaries (rvm-prompt, rvm, rvm-sudo rvm-shell and rvm-auto-ruby)
Removing rvm wrappers in /auto/home13/delacs/.rvm/bin
Hai! Removing /auto/home13/delacs/.rvm

Note you may need to manually remove /etc/rvmrc and ~/.rvmrc if they exist still.
Please check all .bashrc .bash_profile .profile and .zshrc for RVM source lines and delete or comment out if this was a Per-User installation.
Also make sure to remove `rvm` group if this was a system installation.
Finally it might help to relogin / restart if you want to have fresh environment (like for installing RVM again).

2. Now follow this post to reinstall everything.

Capstone Cloud Computing References

1. Hadoop cluster setup on AWS - http://insightdataengineering.com/blog/hadoopdevops/
2. Cassandra Cluster setup on AWS - http://ealfonso.com/setting-up-a-cassandra-cluster-on-awsubuntu14-04/
3. Python with Hadoop Streaming
- http://www.glennklockwood.com/data-intensive/hadoop/streaming.html
- http://www.michael-noll.com/tutorials/writing-an-hadoop-mapreduce-program-in-python/
4. Python with Cassandra - https://academy.datastax.com/demos/getting-started-apache-cassandra-and-python-part-i 5. Cassandra Tutorial - http://wiki.apache.org/cassandra/GettingStarted

namenode:~$ cd $HADOOP_HOME
namenode:/usr/local/hadoop$ bin/hadoop jar share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar -file /home/ubuntu/mapper.py -mapper /home/ubuntu/mapper.py -file /home/ubuntu/reducer.py -reducer /home/ubuntu/reducer.py -input /user/mobydick.txt -output /user/gutenberg-output
namenode:/usr/local/hadoop$ bin/hadoop jar share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar -D mapreduce.job.maps=4 -D mapreduce.job.reduces=1 -file /home/ubuntu/mapper.py    -mapper /home/ubuntu/mapper.py -file /home/ubuntu/reducer.py   -reducer /home/ubuntu/reducer.py -input /user/mobydick.txt -output /user/gutenberg-output

$ hdfs dfs -ls /user/gutenberg-output/
Found 5 items
-rw-r--r--   3 ubuntu supergroup          0 2016-01-26 07:53 /user/gutenberg-output/_SUCCESS
-rw-r--r--   3 ubuntu supergroup      91541 2016-01-26 07:53 /user/gutenberg-output/part-00000
-rw-r--r--   3 ubuntu supergroup      91157 2016-01-26 07:53 /user/gutenberg-output/part-00001
-rw-r--r--   3 ubuntu supergroup      91940 2016-01-26 07:53 /user/gutenberg-output/part-00002
-rw-r--r--   3 ubuntu supergroup      92025 2016-01-26 07:53 /user/gutenberg-output/part-00003

Delete output before starting a new job:
$ hdfs dfs -rm /user/gutenberg-output/*
$ hdfs dfs -rmdir /user/gutenberg-output/

tmux Terminal Multiplexer

Good tutorial from here and here.

Managing tmux sessions:
$ tmux      # start tmux server
$ tmux at   # attach running sessions to a terminal
$ tmux ls   # list running tmux sessions
$ exit      # close tmux session

Sharing sessions between terminals:
$ tmux new -s session_name # make new named session
$ tmux at -t session_name  # attach to exist session (allowing shared sessions)
$ tmux kill-session -t session_name  # kill named session

Commands (used within a running tmux session):

NOTE: All commands need to be prefixed with the action key.
      By default, this is CTRL-b

 c  - create new window
n/p - move to next/previous window
0-9 - move to window number 0-9
 f  - find window by name
 w  - menu with all windows
 &  - kill current window
 ,  - rename window

 %  - split window, adding a vertical pane to the right
 "  - split window, adding an horizontal pane below
←/→ - move focus to left/right pane
↑/↓ - move focus to upper/lower pane

 !  - Break current pane into new window
 x  - Kill the current pane.
 z  - toggle maximize the current pane.
 d  - detach the current client

 [  - enter copy mode (then use emacs select/yank keys)
      * press CTRL-SPACE or CTRL-@ to start selecting text
      * move cursor to end of desired text
      * press ALT-w to copy selected text

 ]  - paste copied text

 ?  - show tmux key bindings

Install Go and Setup Vim as IDE

1. Install Go - http://www.hostingadvice.com/how-to/install-golang-on-ubuntu/
2. Configure Vim as IDE for Go - http://farazdagi.com/blog/2015/vim-as-golang-ide/
3. Docker image of Vim configured with Go - https://hub.docker.com/r/mbrt/golang-vim-dev/

Setup Notes:
When installing vim-go-ide while in corporate firewall, do a clone in this fashion...
git clone https://github.com/farazdagi/vim-go-ide.git ~/.vim_go_runtime

Additional configurations:
$ vim ~/.bashrc
gvm use go1.5.1
alias vimgo='vim -u ~/.vimrc.go'

$ vim ~/.vimrc.go
set t_Co=256

Using the Docker image (what I prefer):
$ cd your/go/workspace
$ docker run --rm -tiv `pwd`:/go mbrt/golang-vim-dev

How to Create and Use SSL Certificates

From this post.

1. ~]$ mkdir CA
2. ~]$ cd CA
3. CA]$ mkdir newcerts private
4. CA]$ echo '01' >serial
5. CA]$ touch index.txt

Create a Root Certificate
6. CA]$ vi openssl.cnf
#
# OpenSSL configuration file.
#

# Establish working directory.

dir         = .

[ ca ]
default_ca      = CA_default

[ CA_default ]
serial          = $dir/serial
database        = $dir/index.txt
new_certs_dir   = $dir/newcerts
certificate     = $dir/cacert.pem
private_key     = $dir/private/cakey.pem
default_days    = 365 
default_md      = md5 
preserve        = no
email_in_dn     = no
nameopt         = default_ca
certopt         = default_ca
policy          = policy_match

[ policy_match ]
countryName     = match
stateOrProvinceName = match
organizationName    = match
organizationalUnitName  = optional
commonName      = supplied
emailAddress        = optional

[ req ]
default_bits        = 1024          # Size of keys
default_keyfile     = key.pem       # name of generated keys
default_md      = md5           # message digest algorithm
string_mask     = nombstr       # permitted characters
distinguished_name  = req_distinguished_name
req_extensions      = v3_req

[ req_distinguished_name ]
# Variable name       Prompt string
#----------------------     ----------------------------------
0.organizationName    = Organization Name (company)
organizationalUnitName    = Organizational Unit Name (department, division)
emailAddress      = Email Address
emailAddress_max  = 40
localityName      = Locality Name (city, district)
stateOrProvinceName   = State or Province Name (full name)
countryName       = Country Name (2 letter code)
countryName_min       = 2
countryName_max       = 2
commonName        = Common Name (hostname, IP, or your name)
commonName_max        = 64

# Default values for the above, for consistency and less typing.
# Variable name             Value
#------------------------------     ------------------------------
0.organizationName_default    = The Sample Company
localityName_default      = Metropolis
stateOrProvinceName_default   = New York
countryName_default       = US

[ v3_ca ]
basicConstraints  = CA:TRUE
subjectKeyIdentifier  = hash
authorityKeyIdentifier    = keyid:always,issuer:always

[ v3_req ]
basicConstraints    = CA:FALSE
subjectKeyIdentifier    = hash


7. CA]$ openssl req -new -x509 -extensions v3_ca -keyout private/cakey.pem -out cacert.pem -days 3650 -config ./openssl.cnf 
Generating a 1024 bit RSA private key
....................++++++
................++++++
writing new private key to 'private/cakey.pem'
Enter PEM pass phrase:demo
Verifying - Enter PEM pass phrase:demo
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Organization Name (company) [The Sample Company]:My Company
Organizational Unit Name (department, division) []:CA Division
Email Address []:ca@sample.com
Locality Name (city, district) [Metropolis]:Santa Clara
State or Province Name (full name) [New York]:California
Country Name (2 letter code) [US]:  
Common Name (hostname, IP, or your name) []:TSC Root CA

Create a Certificate Signing Request
8. CA]$ openssl req -new -nodes -out req.pem -config ./openssl.cnf
Generating a 1024 bit RSA private key
...++++++
.....................++++++
writing new private key to 'key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Organization Name (company) [The Sample Company]:My Company
Organizational Unit Name (department, division) []:Web Server
Email Address []:ca@test.com
Locality Name (city, district) [Metropolis]:Santa Clara
State or Province Name (full name) [New York]:California
Country Name (2 letter code) [US]:US
Common Name (hostname, IP, or your name) []:hostname.domain.com

Sign a Certificate
9. CA]$ openssl ca -out cert.pem -config ./openssl.cnf -infiles req.pem
Using configuration from ./openssl.cnf
Enter pass phrase for ./private/cakey.pem:demo
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
organizationName      :PRINTABLE:'My Company'
organizationalUnitName:PRINTABLE:'Web Server'
localityName          :PRINTABLE:'Santa Clara'
stateOrProvinceName   :PRINTABLE:'California'
countryName           :PRINTABLE:'US'
commonName            :PRINTABLE:'hostname.domain.com'
Certificate is to be certified until Aug 12 18:22:03 2016 GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated

10. CA]$ cat key.pem cert.pem >key-cert.pem
11. CA]$ ls *.pem
cacert.pem  cert.pem  key-cert.pem  key.pem  req.pem

Deploy Certificate
12. Copy the appropriate files, usually cert.pem and key.pem to the location where the certificates will be used as specified by the application.

Rails configure Webrick to use SSL

Came from this post. Change bin/rails to be...

To generate certificates, follow this post.

#!/usr/bin/env ruby

require 'rails/commands/server'
require 'rack'
require 'webrick'
require 'webrick/https'

if ENV['SSL'] == "true"
  module Rails
      class Server < ::Rack::Server
          def default_options
              super.merge({
                  :Port => 3001,
                  :environment => (ENV['RAILS_ENV'] || "development").dup,
                  :daemonize => false,
                  :debugger => false,
                  :pid => File.expand_path("tmp/pids/server.pid"),
                  :config => File.expand_path("config.ru"),
                  :SSLEnable => true,
                  :SSLVerifyClient => OpenSSL::SSL::VERIFY_NONE,
                  :SSLPrivateKey => OpenSSL::PKey::RSA.new(
                                   File.open("certs/key.pem").read),
                  :SSLCertificate => OpenSSL::X509::Certificate.new(
                                   File.open("certs/cert.pem").read),
                  :SSLCertName => [["CN", WEBrick::Utils::getservername]],
              })
          end
      end
  end
end

APP_PATH = File.expand_path('../../config/application',  __FILE__)
require_relative '../config/boot'
require 'rails/commands'

For self-signed certificates as outlined here.

#!/usr/bin/env ruby

require 'rails/commands/server'
require 'rack'
require 'webrick'
require 'webrick/https'

if ENV['SSL'] == "true"
  module Rails
      class Server < ::Rack::Server
          def default_options
              super.merge({
                  :Port => 3001,
                  :environment => (ENV['RAILS_ENV'] || "development").dup,
                  :daemonize => false,
                  :debugger => false,
                  :pid => File.expand_path("tmp/pids/server.pid"),
                  :config => File.expand_path("config.ru"),
                  :SSLEnable => true,
                  :SSLCertName => [["CN", WEBrick::Utils::getservername]],
              })
          end
      end
  end
end

APP_PATH = File.expand_path('../../config/application',  __FILE__)
require_relative '../config/boot'
require 'rails/commands'

Then run server as...

$ SSL=true rails s

A Simple Ruby DSL

It's very easy to create a new language with ruby, here's a simple DSL

class SleepActivity

  def initialize(name)
    @name = name
  end

  def start(&block)
    sleep 1
    puts "#{@name} goes to bed"
    instance_eval &block if block_given?
  end

  def slumber(&block)
    sleep 1
    puts "#{@name} is starting to sleep"
    instance_eval &block if block_given?
  end

  def cycle(period, &block)
    sleep 1
    period.times do |time|
      puts "Sleep cycle #{time + 1}:"
      instance_eval &block if block_given?
    end
  end

  def awake(duration)
    sleep 1
    puts "Still awake for #{duration} minutes"
  end

  def light_sleep(duration)
    sleep 1
    puts "Sleeping lightly for #{duration} minutes"
  end

  def deep_sleep(duration)
    sleep 1
    puts "Sleeping deeply for #{duration} minutes"
  end

  def rem(duration)
    sleep 1
    puts "REM for #{duration} minutes"
  end

end

if $PROGRAM_NAME == __FILE__
  sleep_activity = SleepActivity.new("Sam")
  sleep_activity.start do
    awake 5
    slumber do
      cycle 5 do
        light_sleep 20
        deep_sleep 60
        rem 10
      end
    end
  end
end

Here's the output

samdc@mango:~/dev/ruby/projects/dsl$ ruby dsl_interpreter.rb 
Sam goes to bed
Still awake for 5 minutes
Sam is starting to sleep
Sleep cycle 1:
Sleeping lightly for 20 minutes
Sleeping deeply for 60 minutes
REM for 10 minutes
Sleep cycle 2:
Sleeping lightly for 20 minutes
Sleeping deeply for 60 minutes
REM for 10 minutes
Sleep cycle 3:
Sleeping lightly for 20 minutes
Sleeping deeply for 60 minutes
REM for 10 minutes
Sleep cycle 4:
Sleeping lightly for 20 minutes
Sleeping deeply for 60 minutes
REM for 10 minutes
Sleep cycle 5:
Sleeping lightly for 20 minutes
Sleeping deeply for 60 minutes
REM for 10 minutes