Skip to main content

Best scripting/programming language to learn from an Ops perspective

Knowing some sort of programming is an absolute must for any systems administrator. Hell, even in the Microsoft world, the necessity can't be anymore apparent with the creation of PowerShell. So what is the best language to learn for a beginner? This is a question that many junior sysadmins or students asked themselves when starting of.

Having been a systems administrator for well over eight years now, and worked with Bash, Perl, PHP, Python, and Ruby extensively within those eight years. I think this is a two part answer. In my opinion the best language to start is Bash/shell. Bash/shell is everywhere. Having an advanced Bash knowledge automatically makes you a better sysadmin. This is because Bash/shell scripting is practically glued to other external *nix userland tools (grep, sort, tail, awk, sed, etc..). Knowing Bash extensively, makes you a better command line user. The second of the answer to which is the best scripting/programming language to learn, depends on what environment you work in, or want to work in. If you work or want to work in a Ruby on Rails shop, then obviously you would want to opt learn Ruby, likewise if you work in a WordPress shop, you should opt for learning PHP.

Programming vs Scripting
A systems administrator is NOT a software engineer. There is a huge difference between scripting some sort of convoluted process (automation) and in architecting a complex piece of software. So it is important to acknowledge that the skill set between the both is not the same. Most of the religious wars between a programming language preference people love to display on the internet, is mainly from a developers perspective. Whether the arguments are valid or not, in my opinion they are irrelevant from a sysadmin's point of view. Basically we must know how to use an API and not necessarily how to create one. While in fact it helps to know how to construct your objects by knowing how to create elegant classes, methods, and attributes. It's not vital for a sysadmin to know the exact details on how the API/library was created.

Programming Experience
Once you know a scripting language reasonably well. By knowing either Ruby, Perl, Python, or PHP, you practically can't avoid being already familiar with any of the other scripting languages mentioned. These languages are so similar, it's fairly easy to pick up book and learn another scripting language in a short time span.

It's no secret Ruby is my favorite language, however I'm not overly religious about it. It's not the end of the world if I don't get to use Ruby everyday. In fact, if I need to write a script really fast. Usually my first choices are either Bash or Perl. This is mainly because these are the languages I've known the longest, and for reason I'm able to program in much faster. For me it all boils down to which language will take me less time hack around with and which tool is best for the job. This usually means, is there an existing program, API/library out there that does much of the work I need to solve my problem.

Thus said, when I first started my career, Bash and Perl the defacto scripting languages for a *nix admin to know, now in days it seems Python and Ruby are what companies prefer. Even now with the popularity of NodeJS, soon JavaScript will come into the front along with the other traditional major scripting languages.

Happy hacking.

Packt Publishing Free E-Books crawler

I'm a big fan of Packt Publishing, and have purchased quite a few books from them. So when I first heard a couple of months back that they were going to give out free e-books everyday, my jaw literally dropped. https://www.packtpub.com/packt/offers/free-learning

I've normally been manually checking the site everyday for books that I might be interested on reading. The problem with this, is that their have been days that I missed out getting some free books that I would've love to read. So I wrote a short script that would notify me if there's a free book available that I might be interested in reading. I would've love if Packt Publishing provided an rss feed so I can easily get notifications of their free books. Thus said, I really can't complain since they're already kind enough to give the world free e-books to spread knowledge.

https://github.com/alpha01/Packt-Publishing-Free-Learning

Programming: 

Locking Down WordPress Access with Varnish 3.x

I have Varnish in front of all my WordPress sites and configured all /wp-admin traffic use https via Nginx. See https://www.rubysecurity.org/wordpress_admin-ssl

So to lock down access to my WordPress site's requires both Varnish and Nginx configs to be modified.

Block at the http Varnish level:

sub vcl_recv {
    if ((req.url ~ "wp-(login|admin)") && (client.ip !~ MY-IP-ADDRESS)) {
                error 403 "Fuck off";
        }
}

Block at the https Nginx level (using shit.alpha01.org as an example):

                location /wp-admin {
                        allow   MY-IP-ADDRESS;
                        deny all;
                        proxy_pass https://shit.alpha01.org/wp-admin;
                        proxy_set_header X-Real-IP $remote_addr;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header Host $http_host;
                }
                location /wp-login.php {
                        allow MY-IP-ADDRESS;
                        deny all;
                        proxy_pass https://shit.alpha01.org/wp-login.php;
                        proxy_set_header X-Real-IP $remote_addr;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header Host $http_host;
                }

Linux: 

Awesome Applications: 

Locking Down Drupal Access with Nginx

This site is powered by Drupal. Drupal and WordPress for that matter, are well targeted platforms, mainly because of their large install base on the internet. Quite frankly the reason I bother using both Drupal and WordPress instead of a flat-file based CMS is because I have to deal with these web applications at work on a daily basis, so it's a great way to keep myself current with the technology that's paying my bills.

I have Nginx acting as an SSL proxy for www.rubysecurity.org, which is hosted on an Apache back-end. So I have a few configs that I've enabled to lock down access to my Drupal site. The configs are made at the Nginx proxy level, so they can never reach Apache.

Firstly, I have all of Drupal's /admin locked out from outside access:

        location = /admin {
                allow MY-HOME-IP-ADDRESS;
                deny all;
                return 403;
        }

Next, I only allow login access from my home ip address:

        location = /user {
                allow MY-HOME-IP-ADDRESS;
                deny all;

                proxy_pass https://www.rubysecurity.org/user;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
        }

Lastly, since Nginx is unable to process query strings at the location block level, I've setup an additional config to drop all user login query requests.

       if ($args ~* "q=user") {
                set $blockme  M;
        }
        if ($remote_addr != MY-HOME-IP-ADDRESS) {
                set $blockme  "${blockme}E";
        }
        if ($blockme = ME) {
                return 403;
       }

Linux: 

Awesome Applications: 

My Home Lab Network Diagram

It's Friday night, I don't have anything interesting to do, so I created a diagram of my current home lab setup.

Home Lab Network Diagram

Grepping for PHP system level command functions

grep --color -r -E -e '(escapeshellarg|escapeshellcmd|exec|passthru|proc_close|proc_get_status|proc_nice|proc_open|proc_terminate|shell_exec|system)(\s+)?\(' ./

Programming: 

Setting up Graphite on CentOS 6.x gotcha

I installed graphite-web via the EPEL repo, and I was getting an 500 error when accessing the Graphite web interface.
Error log:

[Sat Sep 12 00:56:27 2015] [error] [client 192.168.1.21] mod_wsgi (pid=17318): Exception occurred processing WSGI script '/usr/share/graphite/graphite-web.wsgi'.
[...]
[Sat Sep 12 00:56:27 2015] [error] [client 192.168.1.21] File "/usr/lib/python2.6/site-packages/django/db/backends/sqlite3/base.py", line 344, in execute
[Sat Sep 12 00:56:27 2015] [error] [client 192.168.1.21] return Database.Cursor.execute(self, query, params)
[Sat Sep 12 00:56:27 2015] [error] [client 192.168.1.21] DatabaseError: attempt to write a readonly database

Fix: It turns out the sqlite3 database file Graphite write's too, was own by root. So it was simply a matter of updating the ownership to what ever user Apache is running under, in my case it's apache.

chown -R apache.apache /var/lib/graphite-web/

Linux: 

Awesome Applications: 

VirtualBox: Increase guest VM CPU count

Syntax
VBoxManage modifyvm <VMNAME> --cpus <CPUcount>

[email protected]:~$ VBoxManage showvminfo monitor | grep "Number of CPUs"
Number of CPUs: 1
[email protected]:~$ VBoxManage modifyvm monitor --cpus 3
[email protected]:~$ VBoxManage showvminfo monitor | grep "Number of CPUs"
Number of CPUs: 3

Awesome Applications: 

BIND - Slave zone transfer stopped working

I was surprised to see that a typo fucked up caused all slave transfers to shit themselves. I came across a situation where a new slave zone was specified to a non-existing location in the file system and that caused the rest of the slave zones to get permission denied errors when trying to update.

Logs

Jul 12 03:23:27 ns2 named[1184]: dumping master file: etc/zones/tmp-Zbk9acg9uv: open: permission denied
Jul 12 03:27:50 ns2 named[1184]: dumping master file: etc/zenos/tmp-4yxBXaUMTq: open: file not found
Jul 12 03:29:46 ns2 named[1184]: dumping master file: etc/zones/tmp-KPqzHa9ev9: open: permission denied
Jul 12 03:38:02 ns2 named[1184]: dumping master file: etc/zones/tmp-kuhtUPjcAi: open: permission denied

Awesome Applications: 

Running my own Git server: GitList

For the longest time I've been wanting to streamline updates to my sites, ie. implement good software deployment technique and procedures. To be specific, start using Git for source code management, and Jenkins to deploy. No, I'm not drinking the whole Agile Kool-Aid. After all we're in 2015, and people who still continue to use FTP/SFTP to push out changes to their sites should really need to be practicing more long term sustainable procedures. Setting up a git server is really simple. See https://www.rubysecurity.org/ansible-git

Git workflow:
I prefer to only communicate with Git over ssh and not https. Since I don't use the default ssh port, the initial repository clone looks like this:

git clone ssh://[email protected]$GIT-SERVER:$SSH-PORT/home/git/$REPO

GitHub has become the defacto Git hosting provider. I think much of it's success, aside from the fact that Git is an amazing piece of software, is GitHub's polished web user interface. While Git ships with a daemon that provides a visual look at the repositories, it's definitely not pretty. I wanted to have a local GitHub like interface on my private git repos, so I decided to use GitList. GitList is fairly minimalistic. Requiring just PHP and mod_rewrite, it allows you to browse your repositories, view files under different revisions, commit history and diffs. Configuring GitList is really easy.

git clone https://github.com/klaussilveira/gitlist.git
cd gitlist
chmod 777 cache
mv config.ini-example config.ini

Then update config.ini to point to the location where the Git repositories are stored in the server. On my server, they're located in /home/git.

repositories[] = '/home/git/' ;

Lastly, is just configuring the web server's virtual host. Since I use Apache mine looks like this.

<VirtualHost 192.168.1.16:443>
        ServerName git.rubyninja.org
        ServerAlias git.rubyninja.org

        DocumentRoot /var/www/gitlist

        <Directory "/var/www/gitlist">
                AllowOverride All
                AuthType Basic
                AuthName "Git Repos"
                AuthUserFile /home/svn/.htpasswd
                Require valid-user
        </Directory>

        SSLEngine on
        SSLCertificateFile /etc/httpd/certs/svn.rubyninja.org.crt
        SSLCertificateKeyFile /etc/httpd/certs/svn.rubyninja.org.key
        SSLCACertificateFile /etc/httpd/certs/rubyninjaCA.crt

        ErrorLog logs/git_ssl_error_log
        CustomLog logs/git_ssl_request_log \
                "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
</VirtualHost>

GitList
GitList

Programming: 

Awesome Applications: 

Pages

Premium Drupal Themes by Adaptivethemes