Django Data Seeding

This is a more flexible way of seeding data

1. Create required directories and files
$ mkdir testapp/management
$ mkdir testapp/management/commands
$ touch testapp/management/__init__.py
$ touch testapp/management/commands/__init__.py

2. Write code that populates the db
$ vi testapp/management/commands/populate_db.py
from django.core.management.base import BaseCommand
from testapp.models import Region
from django.db import connection

class Command(BaseCommand):
    args = ''
    help = 'Management script that populates the database with initial seed'

    def _truncate_region(self):
        cursor = connection.cursor()
        cursor.execute("TRUNCATE TABLE `testapp_region`")

    def _create_regions(self):
        regions = (
        ('REG001', [
                {"name": "vmk-4027","vlanId":4027,"usefor":["vsan","nfs"]},
                {"name": "vmk-4028","vlanId":4028,"usefor":["traffic"]},
                {"name": "vmk-102","vlanId":102,"usefor":["pub"]},
            ]),
        ('REG002', [
                {"name": "vmk-4029","vlanId":4029,"usefor":["vsan","nfs"]},
                {"name": "vmk-4030","vlanId":4030,"usefor":["traffic"]},
                {"name": "vmk-102","vlanId":103,"usefor":["pub"]},
            ]),
        )
        for item in regions:
            region = Region(name=item[0], vlans=item[1])
            region.save()

    def handle(self, *args, **options):
        self._truncate_region()
        self._create_regions()

3. Execute the program
$ python manage.py populate_db

Async Background Tasks with Celery on Django

From this post and this post and this post

1. Install Redis - got from here
$ brew install redis
$ ln -sfv /usr/local/opt/redis/*.plist ~/Library/LaunchAgents
$ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.redis.plist
$ redis-cli PING
PONG

2. Install celery with redis support bundle
$ pip3 install -U "celery[redis]"

3. Create celery.py in django project
$ vi sample_project/sample_project/celery.py
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'sample_project.settings')

app = Celery('sample_project')

# Using a string here means the worker don't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

4. Import celery app in __init__.py
from __future__ import absolute_import, unicode_literals

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app

__all__ = ['celery_app']

5. Add config in settings.py
$ vi sample_project/sample_project/settings.py
# celery
CELERY_BROKER_URL = 'redis://localhost:6379/0'

6. Create app
$ python3 manage.py startapp testapp
$ vi sample_project/sample_project/settings.py
INSTALLED_APPS = (
    # (...)
    'testapp',
)

7. Create task
$ vi sample_project/sample_project/testapp/tasks.py
from __future__ import absolute_import

from celery import shared_task

@shared_task
def test(param):
    return 'The test task executed with argument "%s" ' % param

8. Testing

Run server
$ python3 manage.py runserver

Run worker
$ celery -A sample_project worker --loglevel=info

From another terminal...
$ python3 manage.py shell
Python 3.6.2 (default, Jul 17 2017, 16:44:45)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from testapp.tasks import test
>>> test.delay('This is a test')


From the worker terminal
2017-07-26 00:50:54,529: INFO/MainProcess] Received task: testapp.tasks.test[4f1ab1d4-645d-480b-a1b5-6e2887e34b52]
[2017-07-26 00:50:54,533: INFO/ForkPoolWorker-2] Task testapp.tasks.test[4f1ab1d4-645d-480b-a1b5-6e2887e34b52] succeeded in 0.0010309600038453937s: 'The test task executed with argument "This is a test" '

9. For results backend using Django ORM

a. $ pip3 install django-celery-results

b. Install app in settings.py
INSTALLED_APPS = (
    ...,
    'django_celery_results',
)

c. Create celery database tables
$ python3 manage.py migrate django_celery_results

d. Configure Celery to use the django-celery-results backend.
In settings.py...
CELERY_RESULT_BACKEND = 'django-db'

Django with Postgres on Mac

1. Create virtual env following this post
2. Install django
(my_env) $ pip3 install psycopg2
(my_env) $ pip3 install django
3. Create django project
(my_env) $ django-admin startproject sample_project
(my_env) $ python3 manage.py runserver
4. Install postgres following this post
5. Create database
$ createdb -U postgres testdb
6. Modify settings in django
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'testdb',
        'USER': 'postgres',
        'PASSWORD': '',
        'HOST': 'localhost',
        'PORT': '5432',
    }
}
7. Migrate
$ python3 manage.py migrate
8. Create superuser
$ python3 manage.py createsuperuser

Mac Cheat Sheet

1. Install sshpass
$ brew install http://git.io/sshpass.rb

2. Using iTerm2, configure profiles and use sshpass so there's no need to enter passwords for ssh, follow this post for configuring iTerm2. What worked for me is in Command, select "Login shell", then send text at start:
sshpass -p 'password' ssh -o StrictHostKeyChecking=no user@server

Postgres Cheat Sheet

1. Postgres version
$ psql --version

2. Connect to server locally using default postgres acct
$ psql -U postgres

3. Connect with a password
$ psql -U postgres -W

4. Common commands
\h - help
\q - quit
\l - list databases
\d - list tables
\d table - describe table

5. Create db
postgres=# CREATE DATABASE testdb;

6. Use the db
postgres=# \c testdb;
testdb=#

7. List of users
postgres=# \du

8. List postgres versions running
$ pg_lsclusters
Ver Cluster Port Status Owner    Data directory               Log file
9.1 main    5433 down   postgres /var/lib/postgresql/9.1/main /var/log/postgresql/postgresql-9.1-main.log
9.4 main    5432 down   postgres /var/lib/postgresql/9.4/main /var/log/postgresql/postgresql-9.4-main.log
9.6 main    5434 online postgres /var/lib/postgresql/9.6/main /var/log/postgresql/postgresql-9.6-main.log

9. Start/Stop/Restart/Status of postgresql
$ sudo systemctl start postgresql
$ sudo systemctl stop postgresql
$ sudo systemctl restart postgresql
$ sudo systemctl status postgresql

10. Backup/Restore
$ pg_dump -Fc -h hostname -p 5432 -d dbname -f /path/backupfilename
$ pg_restore -d dbname -v -1 /path/backupfilename

11. Cron daily backup
$ crontab -e
0 5 * * * export PGPASSWORD=MyPass && pg_dump -Fc -h hostname -p 5432 -d dbname -f /path/backupfilename_`date +20\%y\%m\%d` >> /path/pg_dump_dbname.log 2>&1

Install Postgres on Mac

Followed this post

1. Go to http://postgresapp.com/ then download and install
2. Add Postgres to PATH
$ vi ~/.bash_profile
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/latest/bin
$ source ~/.bash_profile
3. Start Postgres by double clicking the app, then on the top icon, start "Initialize"
4. Try connecting using default postgres role
$ psql -U postgres
psql (9.6.3)
Type "help" for help.

postgres=#

Install Python 3.6 on MacOS

Generally followed this post

I. Install brew
1. Install Xcode from the App Store
2. Install Xcode’s separate Command Line Tools app
$ xcode-select --install
3. Install and setup homebrew
$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
4. Make sure brew is in PATH
$ vi ~/.bash_profile
export PATH=/usr/local/bin:$PATH
5. Source it
$ source ~/.bash_profile
6. Try brew
$ brew doctor
Your system is ready to brew.

II. Install Python
$ brew search python
$ brew install python3
$ python3 --version
Python 3.6.2
To install a python package
$ pip3 install package_name
To upgrade python
$ brew update
$ brew upgrade python3

III. Creating Virtual environments
$ mkdir sample_project
$ cd sample_project
$ python3.6 -m venv my_env
$ source my_env/bin/activate

Change bash command prompt in Debian

Followed this post

$ vi /etc/bash.bashrc
Add this at the end:

# If id command returns zero, you have root access.
if [ $(id -u) -eq 0 ];
then # you are root, set red colour prompt
   PS1="\\[$(tput setaf 1)\\][\\u@\\h:\\w] (apps)# \\[$(tput sgr0)\\]"
else # normal
   PS1="[\\u@\\h:\\w] (apps)$ "
fi