It's alive!

Finally managed to get the two new servers for www.faktiskt.se installed and up and running.
Quad CPU database server "Emma" and dual CPU web server "Lina".
A good friend was kind enough to donate a stack of 15krpm 18Gb drives to the project so there is now plenty of disk space as well.

Must say it feels much better to be on MySQL 4.1 instead of the ancient 3.23 version I had on the old RedHat 9 box (yes yes, beat me up, I should have upgraded that box years ago).

The couriers that shipped the servers to Sweden where even more evil than the ones I used in the UK. Luckly a friend helped me to hammer and solder the servers together before installing them. The power socket on one of the boxes looked nasty.
Surgery pics can be seen here.

And a huge thanks to exeo for allowing me to host the machiens at a very good price.

Xmas holiday!

Yeay! Two and a half week holiday, going back to Sweden to celebrate and play with my new Faktiskt.Se servers.



Mary christmas people! :-)

Follow the white rabbit

Ok, so all admins fear network problems. Routing problems and other annoyances, specially if they happened to occour when there is another problem on to solve. I’ve seen this a few times in past years, like the LAN-people are reconfiguring switches just as I reboot a Sun server from the LOM-interface and for some reason the server doesn’t come up cleanly and I can’t access that mgmt-segment where the LOM is since someone forgot my VLAN in the new switch config. Same goes if you are accessing a remote site and the WAN connection goes down or the Internet pipe drops in the middle of a NIC reconfiguration.
So what should people do about this, I’ve started using a concept I call "a rabbit" in most datacenters we have. Its can be a huge security hole if not done right. I want one machine in the network that can access all network segments, and here is the important bit; without passing any L3 switches, any routers and as few tagged VLANs as possible, simply to keep the network access as simple and straight forward as possible. This box should be your entry-point to these segments. The rabbit only works for a single location of course. WAN access without routers is most often a bit of a pain :-)

• Take a decent machine like a U5 with a QFE-card or P3 with a pack of old 100Meg NICS. Oh, and you probably need a serial port as well.
  
• Install a good secure OS on it, preferably OpenBSD. Install as little as possible. Remember this is a rabbit, not a "server".
  
• Configure pf to block all incoming traffic (on ALL interfaces) except SSH (you should probably move SSH to an obscure port like 54088 instead of the default 22).
  
• Connect a GRPS modem, can be an old Nokia phone or a proper modem like the Siemens MC35 I use and either configure PPPd dial-in to the server or simply place a simple getty on that tty. Allowing PPPd to the box is probably better since you can then forward SSH ports from all corners of the network, but in a way opens up a possible security hole (do people still use modems for hacking?).
  
• Connect ethernet cables to switches in the actual segments you want to access. Worst case, use tagged VLANS in the machine, but try to avoid it.

Having the GPRS modem hooked up gives you the possibility to access the box from anywhere in the world, great to be able to plumb NIC’s from a beach in Tahiti. :-)
One additional possibility would be to have a DSL connection to this box to better connection speed to the machine or even VPN, that’s to risky in my opinion. I probably forgot a few steps in but at least you get the picture what the rabbit is supposed to achieve.

Happy Easter.

PS. Creds to my fellow admin Wector for explaining this concept to me with his now well known matburk.

Software wants to be free (and hosted on a Sun cluster)

Sun made a quite interesting announcement earlier this week, this fall actually has been full of interesting announcements from Sun.
This time they announced that more or less the entire Sun software suite will be open sourced and released free to use. Right on! I'd say! :)

Linux made it in through the backdoor
Why do so many small/medium sized companies use Linux on more or less all their servers? I believe one strong factor is simply because that's what the sysadmins know. Young admins, say 22-27 years old, grew up using Linux. Linux was the cool thing and if they wanted to play with a "UNIX-like" (jeje) OS Linux was the obvious choice. So when they get their first positions in small/medium companies with good growth potential they go for what they know and the people they report to doesn't complain since they see how "cheap" (free) Linux is, no need to care about support contracts.

Why both customers and Sun will benefit
Now Sun wants all these admins to look again, to be able to download and test all the awesome features and scalability JES provides. Oracle did partly the same a few years ago when they released most Oracle products free to use for testing and development. My very unscientific guess is that they have sold thousands of licenses based on testing and development made by curious admins and programmers that would never even have thought of using Oracle if they had to fork up the cash without testing it first.
Lets hope Sun can see the same benefits and that people will realize the benefits of running Sun software on Sun servers.
Deploy your application on a pair of X4100 Opteron servers today, cluster them using Sun Cluster server for free. A year down the line your business goes through the roof and you need to expand, now move your application to the a rack full of 64-core Niagra servers and get the support contracts your customers require... all this without modifying one single line of code. Now we see the benefits for both you and for Sun (hint hint you narrow minded financial analysts out there). And of course, if you do need to change code, the compilersa and development environments are also, yes you guessed it, free.

Read Scott McNealys interview in BusinessWeek.

Now why are you still reading this? You should be over here downloading all the enterprise class software you can take.

PostgreSQL 8.1 on ZFS in Solaris

Benchmark time!

Got some free time with my laptop so I decided to do some basic (very unscientific) benchmarks with Postgres to compare Sun's new filesystem ZFS and with the older UFS filesystem.

Test system
Dell Inspiron 8600
Intel Centrino 1.4GHz/1Mb.
768Mb RAM
30Gb 4200RPM 2.5" HDD (Hehe, did you say bottleneck?)

Postgres version is 8.1 and I used more or less the default config. Increased shared_buffers and the fsm parameters, the fsync parameter was toggled on and off during the test as indicated below in the chart.

SQL schema

create table i1 (
        id      serial,
        test    char(50)
);

-- Insert 1 million rows with the letter x 50 times in each row.
insert into i1(test) select lpad('', 50,'x') from 
  generate_series(1,1000000);

I also did a test using the pgbench tool (20 simultaneous clients) that comes in the postgres contrib tree.

Here are the results

FS      FSYNC   Compre. Ins time. Pgbench TPS
------- ------- ------- --------- -----------
UFS     On      -       25718 ms  312
UFS     Off     -       26347 ms  380
ZFS     On      Off     98544 ms  52
ZFS     Off     Off     31850 ms  380
ZFS     On      On      69753 ms  62
ZFS     Off     On      12114 ms  375

Some results are quite intersting.
First, the automatic fsync in postgres kills zfs write performance.
Second, compression gives higher performance on my system.

Conclusion, this benchmark would have been more intersting if it had been done on a Sun Opteron server hooked up to a proper storage system. But the basic IDE drive benchmark may give some pointers. ZFS would probably have performed much better on a more suitable setup and probably proved faster than UFS due to its more direct approach to disk striping.

Portable Solaris

I installed Solaris build 27 on my Dell Inspiron 8600 laptop today. And, of course, it works like a charm. Solaris 10 and OpenSolaris has really come a long way on x86 (looking back at the 2.6 days).

root@halidell$ uname -a
SunOS halidell 5.11 snv_27 i86pc i386 i86pc

Had to install drivers for the onboard bc440 NIC and the ipw2100 WLAN card. Simple installs, pkgadd and followed the instructions. WIFI worked perfectly out of the packages, wificonfig found my access point(s) right away and it took less than 30 seconds to get everything working and browsing slashdot.
Second hurdle was to configure the 1680x1050 widescreen LCD display, seemless as well. Used the same xorg.conf file as I used for Fedora Linux.

Spend a few minutes playing with ZFS and it really rocks.
So much power and so easy to use. Plain and simple.
Veritas should be shaking their balls. Both of their cash-cows, Oracle and Sun both require less and less middleware to be powerfull.

Evil couriers

The damn couriers the delivered my second DL580 server managed to smash the power-cable sockets. Only one of the powersupplies powered up. Crap!
But I think I can fix it with a small hammer and some patiens. Works case I'll "hotwire" it straight through with a kettle lead.
But the four 700MHz/2Mb Xeon processors really kick ass. I installed a minimal centos install and it starts GNOME faster than my 3.0GHz desktop does. Not bad.

Intel Xeon 700MHz slot-2 CPU


Some good news
I have today and tomorrow of from work.Yeay.
When shopping today and picked up a nice blazer and some shoes!
Not even sure what my plan is for tomorrow.