Simple Gallery Site with Picasa Web Albums as the CMS (using Tornado and Google Data API)

July 21, 2012

Requirement: Artist portfolio site with content management capabilities. The site wasn’t not to be a blog, and therefore solutions such as wordpress were out of the question.
In addition, wordpress doesn’t really work very well for items of the gallery type.
On the other hand, a flickr/picasaweb album link isn’t suitable for printing on your business card.
Thus I created “Quickfolio”.
The source is shared on github.
I made it Tornado based, simply because I love the Tornado’s ease of use and the fact that it is in Python.

The nginx config file sample provided in the repo, enables nginx to serve the static files directly as well as refer all dynamic request back to Tornado.

I used Google’s developer guide for Picasa Web Albums extensively.

So, no database, no interface, no nothing. Just a quick setup and you have a full blown gallery with Google acting as the CMS and CDN!

I even added two examples of sites I built using QuickFolio in the readme on github.

TODO: make init script for Tornado (I’ll probably base it on my unicorn solution)

My Solution to Google Code Jam Round 1B 2010: ‘File Fix-it’

April 12, 2012

The problem is stated here.

I will first attempt explain the challenge simply, since from my recent experience, it seems that a lot of software engineering exercises have as much to do with reading comprehension as with actual programming skills.

Then I will describe my approach and also present an alternative quick solution.

Finally I will paste the annotated code and make the script available for download.

Continue Reading »

HBase migration across major versions of HBase (0.2x -> 0.9x)

September 30, 2011

Background:

I inherited a legacy version of the hadoop stack (0.20.2) running hbase v. 0.20.3 on top.
As this is quite an old and unsupported version, we obviously wanted to upgrade.
The main problem with this is that this system is already in use in production, thus increasing the upgrade risks. In addition, the instance installed was compiled and patched manually, so an in-place upgrade was out of the question.

For the new cluster, installed side by side the old one, I chose the “Cloudera Distribution including Apache Hadoop (CDH3)” which is very easy to install and set up via aptitude, and prepares all the standardized stuff like config file locations and init scripts.
This should also hopefully enable relatively painless updates in the future.

Sematext already has a nice blog post on the subject of various HBase backup options, so I won’t go over them in too much detail.

All I will say is that the built-in CopyTable MR job is out of the question since it only works with two clusters of the same major version of HBase. Otherwise life would be much easier…
Continue Reading »

Identifying main traffic sources with netstat and awk (one-liner explained)

March 27, 2011

This is just a short guide on how to find the main offenders in case of web server hammering.

Sample of eventual output:

netstat -natp | grep :80 | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n | tail

     25 195.150.23.130
     25 67.222.164.140
     28 95.34.20.117
     31 72.45.232.204
     34 209.56.4.6
     36 64.27.200.208
    106 50.17.245.114
    112 209.234.226.230
    247 216.242.75.236
    283 184.106.21.219

Continue Reading »

The Power of reddit.

March 22, 2011

Holy Crap

Nothing technically significant there – but since this is the first time I’ve been exposed to such traffic – here is a shout out to people who referred here.

Thanks Ruby Weekly and thanks to the Ruby Show.

Another special thanks to the commenters here and on reddit – provided some very interesting technical insights. It is always humbling and informative to be peer reviewed by such experienced people.

The funny thing is that I’m not even a Ruby guy – I guess several people found the system side of things interesting as well.

Glad to be of help in any case, and will do my best to add interesting content to the site.

Ruby on Rails served by unicorn with nginx for static files and reverse proxy on Ubuntu – detailed guide.

March 10, 2011

Ruby, the ever-so-popular scripting language and Rails – one of the leading web application frameworks are prone to serious performance issues if not served efficiently.
Enter unicorn, a high performance replacement for mongrels and such.
And of course, my goto tool for all things http: nginx.

This is a short step-by-step detailed guide to achieving the following:

  • RoR application listening on 2007 via unicorn
  • nginx listening on 80 forwarding all dynamic requests to the unicorn, and serving static files.

The nginx and the ruby are compiled from source, to provide the latest versions (1.9.2-p136 and 0.8.54 respectively) rather than the ones available via apt (for Ubuntu 10.10 that means: nginx 0.7.67 and ruby 1.8).
Continue Reading »

Dell servers power management settings in BIOS and performance impact

March 1, 2011

Ok so it all started like this:

Bunch of web servers, all the same model (Dell R710), all functioning properly – however two of them are faster then the others (40% lower response time AND lower load average).
Average connections to all the machines was identical.

  • Hardware: Identical
  • Software: Identical (to eliminate this as an issue, a disk was removed from the “fast” server RAID and placed in the “slow” server as a primary drive, and the array was rebuilt with the exact same data)

So it wasn’t a hardware issue (as far as the specs and components could show) and it wasn’t a software issue (same applications and code were the same on both machines).

Just a 40% performance discrepancy…
Continue Reading »

Owner/Group/Permissions Synchronization

February 11, 2010

Image the following scenario:

cd /etc
chown root.root * -R

OOPS
You have just destroyed the server.
This, and other similar mishaps (i.e. chmod 777 / -R) occur more often than one would imagine.

If you are in luck like me, you have access to more than one linux server, and in most cases these servers are similar enough in order to enable a quick restoration of the many file permissions you just destroyed.
Continue Reading »

Apache Cluster Config Sync

January 11, 2010

It has become quite commonplace today, for high traffic sites to require more than one web server.
Here is a small script in charge of synchronizing the server configuration files (/etc/httpd in this example).

In this case, I will be using apache as the web server of choice, simply because of its prevalence and popularity, however the idea is the same, and the script could probably be easily edited to fit your web server of choice (nginx, lighttpd or what have you).
Continue Reading »

MySQL Static Cache Daemon usage example

January 9, 2010

As promised, here is short example of the Daemon in operation.
Continue Reading »

 
Powered by Wordpress. Theme by Shlomi Noach, openark.org
Hosted by Evolution On-line