Owner/Group/Permissions Synchronization

11 February 2010 // Filed under scripts

Image the following scenario:

OOPS
You have just destroyed the server.
This, and other similar mishaps (i.e. chmod 777 / -R) occur more often than one would imagine.

If you are in luck like me, you have access to more than one linux server, and in most cases these servers are similar enough in order to enable a quick restoration of the many file permissions you just destroyed.
(more…)

 ::  Share or discuss  ::  2010-02-11  ::  tom

Apache Cluster Config Sync

11 January 2010 // Filed under scripts

It has become quite commonplace today, for high traffic sites to require more than one web server.
Here is a small script in charge of synchronizing the server configuration files (/etc/httpd in this example).

In this case, I will be using apache as the web server of choice, simply because of its prevalence and popularity, however the idea is the same, and the script could probably be easily edited to fit your web server of choice (nginx, lighttpd or what have you).
(more…)

 ::  Share or discuss  ::  2010-01-11  ::  tom

MySQL Static Cache Daemon usage example

9 January 2010 // Filed under scripts

As promised, here is short example of the Daemon in operation.
(more…)

 ::  Share or discuss  ::  2010-01-09  ::  tom

MySQL Static Cache Daemon

5 January 2010 // Filed under scripts

The problem that this program was designed to solve is a rapidly changing MySQL table in use by a high-traffic website.
For instance, stock quotes on the front page of a bank’s or investments firm’s site.
The data is constantly updated by a service on the backend, and is referred to by some ajax widget or by impatient client, repeatedly refreshing the page (F5-F5-F5-F5-F5).
In order to prevent hammering of the database with SELECT queries, the scripts creates a static page every few seconds, so that users get a relatively updated version of the data (near-live data = good enough).

I chose python for the application, with bash as a wrapper for the daemon init script.
The MySQL data is retrieved with the help of the MySQLdb library (known as python-mysqldb in aptitude repos), also used in a previous script of mine.

The init script is a chkconfig compatible script, although it could be easily modified to a debian/ubuntu styled script as well.

For your convenience, here are all the source files in a tarball in addition to posting the actual code.

(more…)

 ::  Share or discuss  ::  2010-01-05  ::  tom

MySQL on Windows Batch Backup Script

30 November 2009 // Filed under scripts

Don’t ask me why scripting in batch is still necessary on the verge of the year 2010, however, what can you do when some clients still use Windows to host their MySQL servers…

As this was a very quick and dirty script, I won’t go into detail, as it should be quite self explanatory – I think the only interesting part here is the stderr redirection to the logfile, that actually works in windows as should be expected!

See you soon!

 ::  Share or discuss  ::  2009-11-30  ::  tom

MySQL replication monitoring with Python

28 July 2009 // Filed under scripts

Setting up a MySQL fail over Master-Master replication system is something one can find many guides for.

However, monitoring and recovering from a failure of the system is not as well documented, according to a quick Google search.

The top items from that search were:

• Something called: MYSQL master-master replication monitor shell script that mainly included the following piece of code:
  #!/bin/bash
  mysqladmin -u root -h dbserver1.mycorp.com -p 'MyPASSWORD' ping
  if [ $? -ne 0 ]; then
    echo "Send email, mysql not running"
  else
   echo "Do nothing everything is working"
  fi
  done

  Which, while succinct and informative, not exactly thorough.

• Next on the list is some sort of crazy java-based real-time monitor, that is totally overkill and doesn’t answer the exact specifications we had in mind. It also happens to be not that far in its idea from my our (I did not figure out this one by myself – I was greatly assisted by my GNU/Linux mentor at work!) eventual solution.
• The third and final thing on the list remotely resembling what we needed was a pure MySQL solution, which while elegant, and providing some interesting insights and tips, did not answer the requirements stated below.

(more…)

 ::  Share or discuss  ::  2009-07-28  ::  tom

RAID controller *NIX agent

21 July 2009 // Filed under scripts

When managing a large number of servers, it is most likely that you will come across a server that makes use of a RAID controller, and as such you will need to know the status of the disks and the disk array.
As these are usually proprietary controllers, normal tools like ipmiutil, lshw, lspci, etc. aren’t of much use.
Obviously we are talking about Unix-like Systems , and the agent must run as a cron job to send us up-to-date information about our hardware.

It seems that a lot of the major vendors’ (Dell, HP, IBM) controllers are either Adaptec-based or HP-based and thus it is possible to use the CLI utils: arcconf or hrconf (see the end of the post for download links).

As my own experience was mainly with the arcconf utility, the script included below, was written based on it. An example of arcconf’s output can be viewed here.

Small disclaimer, even though this blog is new, a few of these scripts are not so new… This for example is one of my earlier works and is very quirky, and not especially robust, however, it gets the job done!

I will also go into some detail over some basic stuff used in the script like awk, handling command output as variables, and sending mail with bash.
(more…)

1 comment  ::  Share or discuss  ::  2009-07-21  ::  tom

Convert MySQL Tables: MyISAM to InnoDB

20 July 2009 // Filed under scripts

Not getting into the whole which engine is the best debate, since obviously, like most important questions in life, the answer is: “It depends.”

In addition, most of what is written here can be used to convert from any engine to any engine, and thus answering any specific needs your might have.
I needed to convert MyISAM to InnoDB so here it is.

Without further ado…

As far as I know, there are two basic ways to accomplish this:

1.  ALTER TABLE 'tablename' ENGINE = InnoDB

   For each table in the database.

2. Dump the table, edit the dump file where it says “CREATE TABLE” and fix to the preferred engine, and then reload the data back into the DB

Obviously, these methods are ok when you are dealing with a few tables, however a while ago we had a client with a huge database with over 50 tables, so of course, a script was in place.
(more…)

2 comments  ::  Share or discuss  ::  2009-07-20  ::  tom

First Post – Hello Everyone!

20 July 2009 // Filed under Uncategorized

The point of this blog is to share all kinds of scripts and little programs I wrote on various occasions, to solve all kinds of issues.

Hopefully they will be of use to others, as I know many a time I wished for a simple something to Google->Copy->Paste and get on with my life.

 ::  Share or discuss  ::  2009-07-20  ::  tom