Skip to main content

Redirect to www domain in Varnish

I’m a domain name hoarder, over twenty domain names and counting. I’ve done plenty of 301 domain name redirects in the past at the Apache level using mod_rewrite. Hell, just about every registrar now in days gives you the ability to this really easily through them.

In my case, I opted to simply point DNS directly to my server. Since I’m using Varnish (3.x), it’s more efficient to do the actual domain name redirect in Varnish itself rather than Apache. Also not to mention, the actual configuration is easier to understand in Varnish than using mod_rewrite in my opinion.

Example Varnish 3.x config:

# Tony's shitty portfolio site
sub vcl_recv {
  if (req.http.host ~ "(?i)^(www.)?(tony|antonio)baltazar\.(com|org|net)") {
       set req.http.host = "www.antoniobaltazar.com";
       error 750 "http://" + req.http.host + req.url;
   }
}

sub vcl_error {
  if (obj.status == 750) {
    set obj.http.Location = obj.response;
    set obj.status = 301;
    return(deliver);
  }
}

On this example, any incoming request matching (?i)^(www.)?(tony|antonio)baltazar\.(com|org|net) will have the host request header set to www.antoniobaltazar.com. The error function call with the status method of 750 is simply used internally by Varnish, so we can track it at the vcl_error subroutine. I can see why in Varnish 4.x this is handled differently since, this is basically a synthetic internal Varnish request that we're working on and should not be handled in vcl_error.

Example Varnish 4 config

sub vcl_recv {
  if (req.http.host ~ "(?i)^(www.)?(tony|antonio)baltazar\.(com|org|net)") {
    return (synth(750, ""));
   }
}

sub vcl_synth {
    if (resp.status == 750) {
        set resp.status = 301;
        set resp.http.Location = "http://www.antoniobaltazar.com" + req.url;
        return(deliver);
    }
}

So in Varnish 4, the configuration is even easier to understand.

An additional benefit of using this configuration is that now I’m consolidating and increasing Varnish cache hit rate. This means that Varnish won’t have two different caches for antoniobaltazar.com and www.antoniobaltazar.com.

Awesome Applications: 

Python if __name__ == '__main__' Ruby equivalent

Python is by no chance my favorite language to work in, however I always loved the way you can beautifully write your modules and easily test them via if __name__ == '__main__' statement. I've been doing a lot of Ruby programing these past few weeks, and I came across a situation were I needed this exact feature in Ruby.

My problem:
I needed to run some unit tests to a Ruby based TCP server that gets spawn as daemon. The program is completely command line, and once spawned their isn't any code to communicate with its child process. The units weren't exactly to complicated, I simply needed to make sure that the TCP server started, verified its PID, and be able to kill the process. Lucky for me, in the Ruby world we can accomplish the awesome if __name__ == '__main__' Python statement via if __FILE__ == $0

Example:
Here is a test module called test-module.rb

#!/usr/bin/env ruby

if __FILE__ == $0
  puts "Executed via command line."
else
  puts "Included."
end

Now, if we run this test-module.rb from the command line, the if __FILE__ == $0 block will evaluate to true.

alpha03:tests $ ./test-module.rb
Executed via command line.

If the module gets included the if __FILE__ == $0 block will evaluate to false. Example script called test.rb

#!/usr/bin/env ruby

require './test-module'

Running the test.rb script that required test-module.rb

alpha03:tests tony$ ./test.rb
Included.

Conclusion:
Ruby rocks!

Programming: 

Time Stamping Bash's command history

It seems sharing your dot (.) config files is an act that all cool kids do these days. I won't be sharing my Bash configs, however I will share one cool Bash shell trick of time stamping your command history, I use in all of my systems and servers. This is accomplished using the HISTTIMEFORMAT environment variable. Using standard date format control ouput syntax, it's fairly easy to customize the command history time stamp to whatever time format output you prefer. Finally, in addition to using a customized HISTTIMEFORMAT value, I also add the HISTSIZE environment variable. This environment variable lets you override the default command history count to a much larger history count.

HISTSIZE=10000
HISTTIMEFORMAT="%d/%m/%Y %T "
export HISTTIMEFORMAT

Sample output:

3318 21/04/2016 04:42:20 ls
3318 21/04/2016 04:43:29 vim .bashrc
3319 21/04/2016 04:43:37 vim .bash_profile
3320 21/04/2016 05:03:58 man date
3321 21/04/2016 05:07:43 history |tail
3322 21/04/2016 05:07:47 history |tail -5

Programming: 

Best scripting/programming language to learn from an Ops perspective

Knowing some sort of programming is an absolute must for any systems administrator. Hell, even in the Microsoft world, the necessity can't be anymore apparent with the creation of PowerShell. So what is the best language to learn for a beginner? This is a question that many junior sysadmins or students asked themselves when starting of.

Having been a systems administrator for well over eight years now, and worked with Bash, Perl, PHP, Python, and Ruby extensively within those eight years. I think this is a two part answer. In my opinion the best language to start is Bash/shell. Bash/shell is everywhere. Having an advanced Bash knowledge automatically makes you a better sysadmin. This is because Bash/shell scripting is practically glued to other external *nix userland tools (grep, sort, tail, awk, sed, etc..). Knowing Bash extensively, makes you a better command line user. The second of the answer to which is the best scripting/programming language to learn, depends on what environment you work in, or want to work in. If you work or want to work in a Ruby on Rails shop, then obviously you would want to opt learn Ruby, likewise if you work in a WordPress shop, you should opt for learning PHP.

Programming vs Scripting
A systems administrator is NOT a software engineer. There is a huge difference between scripting some sort of convoluted process (automation) and in architecting a complex piece of software. So it is important to acknowledge that the skill set between the both is not the same. Most of the religious wars between a programming language preference people love to display on the internet, is mainly from a developers perspective. Whether the arguments are valid or not, in my opinion they are irrelevant from a sysadmin's point of view. Basically we must know how to use an API and not necessarily how to create one. While in fact it helps to know how to construct your objects by knowing how to create elegant classes, methods, and attributes. It's not vital for a sysadmin to know the exact details on how the API/library was created.

Programming Experience
Once you know a scripting language reasonably well. By knowing either Ruby, Perl, Python, or PHP, you practically can't avoid being already familiar with any of the other scripting languages mentioned. These languages are so similar, it's fairly easy to pick up book and learn another scripting language in a short time span.

It's no secret Ruby is my favorite language, however I'm not overly religious about it. It's not the end of the world if I don't get to use Ruby everyday. In fact, if I need to write a script really fast. Usually my first choices are either Bash or Perl. This is mainly because these are the languages I've known the longest, and for reason I'm able to program in much faster. For me it all boils down to which language will take me less time hack around with and which tool is best for the job. This usually means, is there an existing program, API/library out there that does much of the work I need to solve my problem.

Thus said, when I first started my career, Bash and Perl the defacto scripting languages for a *nix admin to know, now in days it seems Python and Ruby are what companies prefer. Even now with the popularity of NodeJS, soon JavaScript will come into the front along with the other traditional major scripting languages.

Happy hacking.

Packt Publishing Free E-Books crawler

I'm a big fan of Packt Publishing, and have purchased quite a few books from them. So when I first heard a couple of months back that they were going to give out free e-books everyday, my jaw literally dropped. https://www.packtpub.com/packt/offers/free-learning

I've normally been manually checking the site everyday for books that I might be interested on reading. The problem with this, is that their have been days that I missed out getting some free books that I would've love to read. So I wrote a short script that would notify me if there's a free book available that I might be interested in reading. I would've love if Packt Publishing provided an rss feed so I can easily get notifications of their free books. Thus said, I really can't complain since they're already kind enough to give the world free e-books to spread knowledge.

https://github.com/alpha01/Packt-Publishing-Free-Learning

Programming: 

Locking Down WordPress Access with Varnish 3.x

I have Varnish in front of all my WordPress sites and configured all /wp-admin traffic use https via Nginx. See https://www.rubysecurity.org/wordpress_admin-ssl

So to lock down access to my WordPress site's requires both Varnish and Nginx configs to be modified.

Block at the http Varnish level:

sub vcl_recv {
    if ((req.url ~ "wp-(login|admin)") && (client.ip !~ MY-IP-ADDRESS)) {
                error 403 "Fuck off";
        }
}

Block at the https Nginx level (using shit.alpha01.org as an example):

                location /wp-admin {
                        allow   MY-IP-ADDRESS;
                        deny all;
                        proxy_pass https://shit.alpha01.org/wp-admin;
                        proxy_set_header X-Real-IP $remote_addr;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header Host $http_host;
                }
                location /wp-login.php {
                        allow MY-IP-ADDRESS;
                        deny all;
                        proxy_pass https://shit.alpha01.org/wp-login.php;
                        proxy_set_header X-Real-IP $remote_addr;
                        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header Host $http_host;
                }

Linux: 

Awesome Applications: 

Locking Down Drupal Access with Nginx

This site is powered by Drupal. Drupal and WordPress for that matter, are well targeted platforms, mainly because of their large install base on the internet. Quite frankly the reason I bother using both Drupal and WordPress instead of a flat-file based CMS is because I have to deal with these web applications at work on a daily basis, so it's a great way to keep myself current with the technology that's paying my bills.

I have Nginx acting as an SSL proxy for www.rubysecurity.org, which is hosted on an Apache back-end. So I have a few configs that I've enabled to lock down access to my Drupal site. The configs are made at the Nginx proxy level, so they can never reach Apache.

Firstly, I have all of Drupal's /admin locked out from outside access:

        location = /admin {
                allow MY-HOME-IP-ADDRESS;
                deny all;
                return 403;
        }

Next, I only allow login access from my home ip address:

        location = /user {
                allow MY-HOME-IP-ADDRESS;
                deny all;

                proxy_pass https://www.rubysecurity.org/user;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
        }

Lastly, since Nginx is unable to process query strings at the location block level, I've setup an additional config to drop all user login query requests.

       if ($args ~* "q=user") {
                set $blockme  M;
        }
        if ($remote_addr != MY-HOME-IP-ADDRESS) {
                set $blockme  "${blockme}E";
        }
        if ($blockme = ME) {
                return 403;
       }

Linux: 

Awesome Applications: 

My Home Lab Network Diagram

It's Friday night, I don't have anything interesting to do, so I created a diagram of my current home lab setup.

Home Lab Network Diagram

Grepping for PHP system level command functions

grep --color -r -E -e '(escapeshellarg|escapeshellcmd|exec|passthru|proc_close|proc_get_status|proc_nice|proc_open|proc_terminate|shell_exec|system)(\s+)?\(' ./

Programming: 

Setting up Graphite on CentOS 6.x gotcha

I installed graphite-web via the EPEL repo, and I was getting an 500 error when accessing the Graphite web interface.
Error log:

[Sat Sep 12 00:56:27 2015] [error] [client 192.168.1.21] mod_wsgi (pid=17318): Exception occurred processing WSGI script '/usr/share/graphite/graphite-web.wsgi'.
[...]
[Sat Sep 12 00:56:27 2015] [error] [client 192.168.1.21] File "/usr/lib/python2.6/site-packages/django/db/backends/sqlite3/base.py", line 344, in execute
[Sat Sep 12 00:56:27 2015] [error] [client 192.168.1.21] return Database.Cursor.execute(self, query, params)
[Sat Sep 12 00:56:27 2015] [error] [client 192.168.1.21] DatabaseError: attempt to write a readonly database

Fix: It turns out the sqlite3 database file Graphite write's too, was own by root. So it was simply a matter of updating the ownership to what ever user Apache is running under, in my case it's apache.

chown -R apache.apache /var/lib/graphite-web/

Linux: 

Awesome Applications: 

Pages

Premium Drupal Themes by Adaptivethemes