vimrc setting for Python
" configure expanding of tabs for various file types au BufRead,BufNewFile *.py set expandtab au BufRead,BufNewFile Makefile* set noexpandtab " -------------------------------------------------------------------------------- " configure editor with tabs and nice stuff... " -------------------------------------------------------------------------------- set expandtab " enter spaces when tab is pressed set textwidth=120 " break lines when line length increases set tabstop=4 " use 4 spaces to represent tab set softtabstop=4 set shiftwidth=4 " number of spaces to use for auto indent set autoindent " copy indent from current line when starting a set nowrapscan " Don't search from the top if end-of-file is reached set smartcase " Smartcase: ignore case except when upper case is typed set ignorecase " Ignore case during searches new line " make backspaces more powerfull set backspace=indent,eol,start set ruler " show line and column number syntax on " syntax highlighting set showcmd " show (partial) command in status line
Schedule cronjob every second time it runs
I needed a way to run a cronjob every other Saturday. You could alter this for other scenarios as long as you set the correct schedule in your crontab.
0 3 * * 6 /root/scripts/MYSCRIPT.sh
The first time cron runs, it should detect that no file is present (in this case “/tmp/cron_control”), which in turn creates the file and exits. The next time cron is scheduled, it will see that the file exists, removes it and proceeds to run the rest of the code. If you want your script to run at the very next scheduled timeslot, you could reverse this and create the “/tmp/cron_control” beforehand.
#!/bin/bash
CRON_CONTROL="/tmp/cron_control"
if [ -f ${CRON_CONTROL} ]
then
rm -f ${CRON_CONTROL}
else
touch ${CRON_CONTROL}
exit 0
fi
... RUN THE REST OF CODE ...
exit 0
Git command
Track remote branch
bash# git fetch bash# git branch --track previously_created_branch origin/previously_created_branch
How to backup all MySQL databases using Python
#!/usr/bin/env python
import ConfigParser
import os
import time
username = 'dbuser'
password = 'dbpassword'
hostname = 'localhost'
filestamp = time.strftime('%Y-%m-%d')
# Get a list of databases with :
database_list_command="mysql -u%s -p%s -h %s --silent -N -e 'show databases'" % (username, password, hostname)
for database in os.popen(database_list_command).readlines():
database = database.strip()
if database == 'information_schema':
continue
filename = "backups/%s-%s.sql" % (database, filestamp)
os.popen("mysqldump -u%s -p%s -h %s -e --opt -c %s | gzip -c > %s.gz" % (username, password, hostname, database, filename))
How we use Puppet to inventory our servers.
I think every sysadmin can relate to this, you’re in the kitchen getting a cup of coffee and your boss happens to walk in. You spend a couple of minutes having small talk but as soon as you’re done he drops the bomb. “Hey, could you send me a list of all the servers we have in our datacenter? Finance wants a list for insurance purposes, but I have an old copy … think you could update it and get it to me by the end of the day?” If you’re on top of your game you probably have it in a wiki or a fancy excel spreadsheet with bright color coded cells. Nope, not I, I totally forgot to add the last 20 servers I provisioned. I moved on to the next task, and that spreadsheet he was referring to has a timestamp that’s two months old.
Honestly, we’ve tried using both methods but it quickly gets outdated. Lack of time, lack of interest, you name it and our list includes servers that are no longer in production. It wouldn’t be an issue if we have 20 servers but our lab has over 80 hosts, both physical and virtual machines. Maintaining a list could get out of hand if our spreadsheet (or wiki) isn’t updated routinely when a new host is added or remove from our lab. The next time my boss asks me for an updated inventory list, I’ll send him the URL to our newly built dashboard. Here’s how I made it happen …
Gremlins, I found a family of gremlins who wanted to work as long as I didn’t call the exterminator on them. Nah, seriously I decided to re-do our Puppet environment in order to take advantage of “stored configuration.” With stored configuration, Puppet is able to save data into a database of your choice (SQLite, PostgreSQL or MySQL). “Inventory-ing” the server is possible by using a Ruby library called “Facter.” Since Facter is required by Puppet, I decided to take advantage of the information it obtains from the host after each run. Facter returns valuable information such as hostname, amount of memory, IP address and whether the host is a virtual or physical. All of this information and then some is stored inside a MySQL database.
Ok, so Puppet/Facter does the heavy lifting but how do I get this into a spreadsheet? I dont, I hate spreadsheets and I refuse to update cells and play with columns and rows. Instead, I used my PHP skills and created a customize dashboard that displays everything from hostname, IP, memory usage and rack location. If a server received an additional 2G of RAM, Puppet will update the value the next time it’s scheduled to run. It’s a win-win situation for everyone, I no longer have to worry about updating a wiki page and my boss doesn’t have to nag me for an updated list.
exec of init (/sbin/init) failed
One fine day I get an email from an engineer. Apparently he removed glibc, no idea what he was doing but I was asked to somehow rebuild the box and preserve data. The problem was, we weren’t able to log into the system. Single user mode didn’t work either so now I’m stumped.
Here’s the message I kept getting on the console:
exec of init (/sbin/init) failed !!!: 2
umount /initrd/dev failed: 2
kernel panic – not syncing. Attempted to kill init!
Ok, no biggie, let’s slap another drive on the server, install linux and mount the other drives in order to copy the data off. Then it occurred to me, the server has 4 drive bays and has no room for an additional drive. Now what the hell do I do? Google search resulted in various “fixes,” but every one of them suggested that I copy the required folders/files from an identical box elsewhere, but I wouldn’t even know where to start. Ok, fair enough if he removed glibc, let’s try to reinstall glibc shall we? Before I reply and tell everyone the bad news, I attempted to boot off a rescue CD and see if I could figure out why it’s complaining. I attempted to chroot the /mnt/sysimage folder but nope I wasn’t able to – /bin/sh was missing. YOU HAVE TO BE KIDDING?!
Well, there goes my plan of reinstalling glibc. Then it hit me, all “rpm -ivh” does is extract the compress files into its appropriate directory. Let’s just extract them myself without having to run “rpm -ivh
Note, I saved the RPMS under /mnt/sysimage (which is essentially your / folder under normal circumstances).
#shell>cd /mnt/sysimage
#shell>wget http://servername/packagename
#shell>pwd
/mnt/sysimage/
#shell>rpm2cpio PACKAGENAME | cpio -idmv
I had to run the last command for every dependencies for glibc. How did I figure that out? Once I downloaded glibc’s RPM, I ran “rpm -ivh,” which then gave me a list of the required dependencies. (Tip: run “rpm -ivh *.rpm –test” to make sure you satisfy all the dependecies) After I extracted all the RPMs, I rebooted and crossed my finger. Four minutes later I had a working system! I was able to SSH into the server and my engineer is happy once again.
How to modify and repackage an RPM
Having to install any software and then configuring it so that it fits into your environment can be a pain in the butt. Especially if you have to roll it out to hundreds of servers. One such software is nagios’ nrpe client. After every install, I had to modify the nrpe.conf file to allow access from my master server along with a few nrpe checks on the client side (i.e. disk space utilization and cpu usage threshold). I could’ve used puppet to modify the file but hey let’s play with SPEC files and rpmbuild.
The example below rebuilds nrpe, substitute the filename with the package you want to work with. Note, if you’re rebuilding a 64 bit package, you need a 64bit machine (with 64bit OS obviously).
1 – Download the RPM source file
Usually in the following format “nrpe-2.12-14.el5.src.rpm” (notice the “src” within the file name).
2 – Install the source file
[php]rpm -ivh nrpe-2.12-14.el5.src.rpm[/php]
You’ll see a bunch of files scattered within /usr/src/redhat. Depending on what you’re looking to modify, you’ll have to dig through the folders (mainly SOURCES). There should be a tar.gz file under /usr/src/redhat/SOURCES. I’m interested in modifying a couple of lines within nrpe.conf (the main conf file used by nrpe). I uncompressed the tar.gz file, then proceeded to modify nrpe.conf and re-tar it back up (overwrite original file). Make sure you delete the uncompress folder prior to rebuilding. Once that’s done, I rebuild and create a “new” rpm file with the modified nrpe.conf.
3 – Rebuild RPM package
[php]cd /usr/src/redhat/SPECS
rpmbuild -ba nrpe.spec
[/php]
This should create an RPM file under /usr/src/redhat/RPMS
Using tcpdump
To view incoming and outgoing packets on port 80
[code]tcpdump -i eth0 tcp port 80[/code]
To view outbound traffic over port 80
[code]tcpdump -i eth1 tcp dst port 80 and src host 192.168.0.10[/code]
To suppress verbose output, you just want to view source/destination
[code]tcpdump -q -i eth1 tcp dst port 80 and src host 192.168.0.10[/code]
Large file transfer using netcat
I didn’t feel like setting up a NFS share and ftp access within the organization is just plain stupid if you ask me. scp is cool but also lots of overhead.
netcat is the shit.
client -> serverA
target -> serverB
On my client (where I want to send files from), I connect to serverB via port 11223
[php]
tar -cz SOMEFILES | nc serverB 11223
[/php]
On target (where the file copies to), I open up port 11223 and extract my files to current working directory.
[php]
nc -l 11223 | tar -xz
[/php]
OS and browser detection with javascript
[php]
var BrowserDetect = {
init: function () {
this.browser = this.searchString(this.dataBrowser) || “An unknown browser”;
this.version = this.searchVersion(navigator.userAgent)
|| this.searchVersion(navigator.appVersion)
|| “an unknown version”;
this.OS = this.searchString(this.dataOS) || “an unknown OS”;
},
searchString: function (data) {
for (var i=0;i
var dataProp = data[i].prop;
this.versionSearchString = data[i].versionSearch || data[i].identity;
if (dataString) {
if (dataString.indexOf(data[i].subString) != -1)
return data[i].identity;
}
else if (dataProp)
return data[i].identity;
}
},
searchVersion: function (dataString) {
var index = dataString.indexOf(this.versionSearchString);
if (index == -1) return;
return parseFloat(dataString.substring(index+this.versionSearchString.length+1));
},
dataBrowser: [
{
string: navigator.userAgent,
subString: "Chrome",
identity: "Chrome"
},
{ string: navigator.userAgent,
subString: "OmniWeb",
versionSearch: "OmniWeb/",
identity: "OmniWeb"
},
{
string: navigator.vendor,
subString: "Apple",
identity: "Safari",
versionSearch: "Version"
},
{
prop: window.opera,
identity: "Opera"
},
{
string: navigator.vendor,
subString: "iCab",
identity: "iCab"
},
{
string: navigator.vendor,
subString: "KDE",
identity: "Konqueror"
},
{
string: navigator.userAgent,
subString: "Firefox",
identity: "Firefox"
},
{
string: navigator.vendor,
subString: "Camino",
identity: "Camino"
},
{ // for newer Netscapes (6+)
string: navigator.userAgent,
subString: "Netscape",
identity: "Netscape"
},
{
string: navigator.userAgent,
subString: "MSIE",
identity: "Explorer",
versionSearch: "MSIE"
},
{
string: navigator.userAgent,
subString: "Gecko",
identity: "Mozilla",
versionSearch: "rv"
},
{ // for older Netscapes (4-)
string: navigator.userAgent,
subString: "Mozilla",
identity: "Netscape",
versionSearch: "Mozilla"
}
],
dataOS : [
{
string: navigator.platform,
subString: "Win",
identity: "Windows"
},
{
string: navigator.platform,
subString: "Mac",
identity: "Mac"
},
{
string: navigator.userAgent,
subString: "iPhone",
identity: "iPhone/iPod"
},
{
string: navigator.platform,
subString: "Linux",
identity: "Linux"
}
]
};
BrowserDetect.init();
[/php]
[php]
if (Firefox on Mac)
{
document.write("
“);
}
else if (Firefox on PC)
{
document.write(“
“);
}
[/php]




