Django Notes

1. Backup and seed data using fixtures
# Save data
$ python manage.py dumpdata --format=json myapp > myapp/fixtures/initial_data.json
# Load data
$ python manage.py loaddata vlanapi/fixtures/initial_data.json

2. Open python shell - interactive console
$ python manage.py shell
Python 2.7.12 (default, Jul  1 2016, 15:12:24) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>

3. Start a project
$ django-admin startproject mysite

4. Start initial migration
$ python manage.py migrate

5. Start server
$ python manage.py runserver

6. Create an app
$ python manage.py startapp newapp

7. Create and run a migration
$ python manage.py makemigrations
$ python manage.py migrate

Python Notes

1. Booleans
The numbers 0, 0.0, and 0+0j are all False; any other number is True.
The empty string "" is False; any other string is True.
The empty list [] is False; any other list is True.
The empty dictionary {} is False; any other dictionary is True.
The empty set set() is False; any other set is True.
The special Python value None is always False.

2. Working on REPL
Import library
>>> from autoinfra.resource import Client, Ddr, Resource
List all available methods
>>> dir(Client)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_connect_ssh', 'bring_up_eth', 'configure_ip', 'eth_filter', 'get_eths_details', 'get_eths_via_ifconfig', 'get_eths_via_ip_a', 'get_switch_port_details', 'is_connected', 'is_eth_link_detected', 'send_cmd']
Access docstring on a method
>>> help(Client.get_switch_port_details)

3. Testing
From this post.

$ pip install nose

From root of project run nosetests, this would look for any test under this directory and run it
$ pwd
/auto/home13/delacs/Documents/projects/nextgen/libs/autoinfra
$ nosetests
.....
----------------------------------------------------------------------
Ran 5 tests in 2.659s
OK

To run a single file
$ nosetests -v autoinfra/resources/tests/test_ddr.py
test_get_10g_eths (test_ddr.TestDdr) ... ok
test_se_cmd (test_ddr.TestDdr) ... ok
test_send_cmd (test_ddr.TestDdr) ... ok
----------------------------------------------------------------------
Ran 3 tests in 2.602s

To run a particular test in a file
$ nosetests -v autoinfra/switches/tests/test_cisco.py:TestCisco.test_move_to_vlan_no_nxapi_single
test_move_to_vlan_no_nxapi_single (test_cisco.TestCisco) ... ok
----------------------------------------------------------------------
Ran 1 test in 12.423s
OK

This works when test files have imports like this...
from autoinfra.resources.client import Client

4. Install python 2.7.12 on another directory (/opt/python)
# wget https://www.python.org/ftp/python/2.7.12/Python-2.7.12.tgz
# tar zxf Python-2.7.12.tgz
# cd Python-2.7.12/
# ./configure --prefix=/opt/python
# make
# make install

5. Create a virtual environment
$ sudo yum install python-virtualenv
$ mkdir sample_project
$ cd sample_project
$ virtualenv -p /opt/python/bin/python2.7 .venv

6. Data dumper
>>> from pprint import pprint
>>> pprint(vars(Host.objects.get(name="w3-perfsds-0001")))

Install TP-Link AC1200 T4U in Ubuntu 16.04

$ sudo apt-get install synaptic
search for rtl8812au-dkms and install
$ sudo service network-manager stop
$ sudo modprobe -rfv 8812au
$ sudo modprobe -v 8812au
$ sudo service network-manager start
$ sudo dkms status
rtl8812au, 4.3.8.12175.20140902+dfsg, 4.4.0-31-generic, x86_64: installed

Everytime kernel is updated, do these steps to recompile:
$ dkms status
$ sudo dkms remove rtl8812au/4.3.8.12175.20140902+dfsg -k $(uname -r)
$ sudo dkms install rtl8812au/4.3.8.12175.20140902+dfsg
$ sudo modprobe 8812au

Cron Like Program in Ruby with Clockwork

I like this gem I discovered called clockwork. A good replacement for cron plus more because you can do extra programming when scheduling jobs.

require 'clockwork'
module Clockwork
  handler do |job|
    puts "Running #{job}"
  end 

  # handler receives the time when job is prepared to run in the 2nd argument
  # handler do |job, time|
  #   puts "Running #{job}, at #{time}"
  # end

  every(1.hour, 'hourly.ms_status') do
    `cd $HOME/Documents/projects/monweb_management && $HOME/.rvm/wrappers/ruby-2.3.0@RailsDev/rake file:get_ms_status`
  end 

  every(1.day, 'daily.pbba_query', :at => '07:00') do
    `cd $HOME/Documents/projects/monweb_management && $HOME/.rvm/wrappers/ruby-2.3.0@RailsDev/rake file:get_pbba_sizing`
  end 
end

Then run it like this...

$ clockwork job_scheduler.rb
I, [2016-05-24T09:18:21.215569 #771]  INFO -- : Starting clock for 2 events: [ hourly.ms_status daily.pbba_query ]
I, [2016-05-24T09:18:21.215729 #771]  INFO -- : Triggering 'hourly.ms_status'

RVM Implode

My rvm installation and ruby got messed up after a change in home directory mounts at work. Looks like rvm had referenced everything back in my old home directory full path. The only way I found to easily fix this issue is to re-install rvm, ruby and all gems I need including rails.

1. Remove every trace of rvm
$ rvm implode
Are you SURE you wish for rvm to implode?
This will recursively remove /auto/home13/delacs/.rvm and other rvm traces?
(anything other than 'yes' will cancel) > yes
Removing rvm-shipped binaries (rvm-prompt, rvm, rvm-sudo rvm-shell and rvm-auto-ruby)
Removing rvm wrappers in /auto/home13/delacs/.rvm/bin
Hai! Removing /auto/home13/delacs/.rvm

Note you may need to manually remove /etc/rvmrc and ~/.rvmrc if they exist still.
Please check all .bashrc .bash_profile .profile and .zshrc for RVM source lines and delete or comment out if this was a Per-User installation.
Also make sure to remove `rvm` group if this was a system installation.
Finally it might help to relogin / restart if you want to have fresh environment (like for installing RVM again).

2. Now follow this post to reinstall everything.

Capstone Cloud Computing References

1. Hadoop cluster setup on AWS - http://insightdataengineering.com/blog/hadoopdevops/
2. Cassandra Cluster setup on AWS - http://ealfonso.com/setting-up-a-cassandra-cluster-on-awsubuntu14-04/
3. Python with Hadoop Streaming
- http://www.glennklockwood.com/data-intensive/hadoop/streaming.html
- http://www.michael-noll.com/tutorials/writing-an-hadoop-mapreduce-program-in-python/
4. Python with Cassandra - https://academy.datastax.com/demos/getting-started-apache-cassandra-and-python-part-i 5. Cassandra Tutorial - http://wiki.apache.org/cassandra/GettingStarted

namenode:~$ cd $HADOOP_HOME
namenode:/usr/local/hadoop$ bin/hadoop jar share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar -file /home/ubuntu/mapper.py -mapper /home/ubuntu/mapper.py -file /home/ubuntu/reducer.py -reducer /home/ubuntu/reducer.py -input /user/mobydick.txt -output /user/gutenberg-output
namenode:/usr/local/hadoop$ bin/hadoop jar share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar -D mapreduce.job.maps=4 -D mapreduce.job.reduces=1 -file /home/ubuntu/mapper.py    -mapper /home/ubuntu/mapper.py -file /home/ubuntu/reducer.py   -reducer /home/ubuntu/reducer.py -input /user/mobydick.txt -output /user/gutenberg-output

$ hdfs dfs -ls /user/gutenberg-output/
Found 5 items
-rw-r--r--   3 ubuntu supergroup          0 2016-01-26 07:53 /user/gutenberg-output/_SUCCESS
-rw-r--r--   3 ubuntu supergroup      91541 2016-01-26 07:53 /user/gutenberg-output/part-00000
-rw-r--r--   3 ubuntu supergroup      91157 2016-01-26 07:53 /user/gutenberg-output/part-00001
-rw-r--r--   3 ubuntu supergroup      91940 2016-01-26 07:53 /user/gutenberg-output/part-00002
-rw-r--r--   3 ubuntu supergroup      92025 2016-01-26 07:53 /user/gutenberg-output/part-00003

Delete output before starting a new job:
$ hdfs dfs -rm /user/gutenberg-output/*
$ hdfs dfs -rmdir /user/gutenberg-output/

tmux Terminal Multiplexer

Good tutorial from here and here.

Managing tmux sessions:
$ tmux      # start tmux server
$ tmux at   # attach running sessions to a terminal
$ tmux ls   # list running tmux sessions
$ exit      # close tmux session

Sharing sessions between terminals:
$ tmux new -s session_name # make new named session
$ tmux at -t session_name  # attach to exist session (allowing shared sessions)
$ tmux kill-session -t session_name  # kill named session

Commands (used within a running tmux session):

NOTE: All commands need to be prefixed with the action key.
      By default, this is CTRL-b

 c  - create new window
n/p - move to next/previous window
0-9 - move to window number 0-9
 f  - find window by name
 w  - menu with all windows
 &  - kill current window
 ,  - rename window

 %  - split window, adding a vertical pane to the right
 "  - split window, adding an horizontal pane below
←/→ - move focus to left/right pane
↑/↓ - move focus to upper/lower pane

 !  - Break current pane into new window
 x  - Kill the current pane.
 z  - toggle maximize the current pane.
 d  - detach the current client

 [  - enter copy mode (then use emacs select/yank keys)
      * press CTRL-SPACE or CTRL-@ to start selecting text
      * move cursor to end of desired text
      * press ALT-w to copy selected text

 ]  - paste copied text

 ?  - show tmux key bindings