Monday, November 28

PostgreSQL 8.1 on ZFS in Solaris

Benchmark time!

Got some free time with my laptop so I decided to do some basic (very unscientific) benchmarks with Postgres to compare Sun's new filesystem ZFS and with the older UFS filesystem.

Test system
Dell Inspiron 8600
Intel Centrino 1.4GHz/1Mb.
768Mb RAM
30Gb 4200RPM 2.5" HDD (Hehe, did you say bottleneck?)

Postgres version is 8.1 and I used more or less the default config. Increased shared_buffers and the fsm parameters, the fsync parameter was toggled on and off during the test as indicated below in the chart.

SQL schema
create table i1 (
id serial,
test char(50)
);

-- Insert 1 million rows with the letter x 50 times in each row.
insert into i1(test) select lpad('', 50,'x') from
generate_series(1,1000000);
I also did a test using the pgbench tool (20 simultaneous clients) that comes in the postgres contrib tree.

Here are the results
FS      FSYNC   Compre. Ins time. Pgbench TPS
------- ------- ------- --------- -----------
UFS On - 25718 ms 312
UFS Off - 26347 ms 380
ZFS On Off 98544 ms 52
ZFS Off Off 31850 ms 380
ZFS On On 69753 ms 62
ZFS Off On 12114 ms 375
Some results are quite intersting.
First, the automatic fsync in postgres kills zfs write performance.
Second, compression gives higher performance on my system.

Conclusion, this benchmark would have been more intersting if it had been done on a Sun Opteron server hooked up to a proper storage system. But the basic IDE drive benchmark may give some pointers. ZFS would probably have performed much better on a more suitable setup and probably proved faster than UFS due to its more direct approach to disk striping.

Tuesday, November 22

Portable Solaris

I installed Solaris build 27 on my Dell Inspiron 8600 laptop today. And, of course, it works like a charm. Solaris 10 and OpenSolaris has really come a long way on x86 (looking back at the 2.6 days).
root@halidell$ uname -a
SunOS halidell 5.11 snv_27 i86pc i386 i86pc

Had to install drivers for the onboard bc440 NIC and the ipw2100 WLAN card. Simple installs, pkgadd and followed the instructions. WIFI worked perfectly out of the packages, wificonfig found my access point(s) right away and it took less than 30 seconds to get everything working and browsing slashdot.
Second hurdle was to configure the 1680x1050 widescreen LCD display, seemless as well. Used the same xorg.conf file as I used for Fedora Linux.

Spend a few minutes playing with ZFS and it really rocks.
So much power and so easy to use. Plain and simple.
Veritas should be shaking their balls. Both of their cash-cows, Oracle and Sun both require less and less middleware to be powerfull.

Monday, November 21

Evil couriers

The damn couriers the delivered my second DL580 server managed to smash the power-cable sockets. Only one of the powersupplies powered up. Crap!
But I think I can fix it with a small hammer and some patiens. Works case I'll "hotwire" it straight through with a kettle lead.
But the four 700MHz/2Mb Xeon processors really kick ass. I installed a minimal centos install and it starts GNOME faster than my 3.0GHz desktop does. Not bad.

Intel Xeon 700MHz slot-2 CPU
Xeon processor

Some good news
I have today and tomorrow of from work.Yeay.
When shopping today and picked up a nice blazer and some shoes!
Not even sure what my plan is for tomorrow.

Monday, November 14

Second DL580 server

And we have lift-off! :-)

At least quite soon.
Won another Compaq DL580 auction today, used the "buy-it-now" feature actually.
£205 for a Quad 700Mhz/2Mb CPU machine with 2Gb RAM. Unfortunatly no HDDs' in it.
Now I have both the DB back-end and the Apache front-end machine.
Need to find a few drives now, one more 36Gb drive and at least three 18Gb drives would be sweet.
I'll post some geek pr0n later :)

Sunday, November 13

eBay snipers and Compaq servers

Ok, so I've this web site project going. A bit secret so far but I'm coding some things for a very cool site. And for this site I of course need a few servers to run it on. I'm not expecting insane amounts of hits bit still want to cater for quite some traffic. One dedicated DB server and one Apache server. I have high hopes :-)
After scanning eBay for a few weeks I decided that two good old Compaq DL580s was the way to go. Nice quad CPU machines with lots of redundancy and expandability. And best of all, the price on eBay is quite good. They usually go for about £250GBP in a decent spec. I would have preferred to get Sun or IBM machines but after doing the price/performance maths on refurbished machines HP/Compaq just made more sense. The alternatives where either IBM x350/6000r's or Sun E420r's I guess, perhaps if I can find a real bargin.

I actually managed to snag a DL580 for just £150GBP last week. Quite good spec, 2x700MHz Xeons, 4Gb RAM (yes four!), 3x36Gb SCSI drives and a nice SmartRaid 6400 controller. The drives alone usually run for £35 on eBay.
Haven't much time to play with it yet, re-configured the drives to a RAID5 array, flashed the BIOS to the latest version and installed CentOS on it, feels extremly fast and responsive for being a 4 year old 700MHz machine.
Also bought a few Intel gigabit server NICs at £14 a pop.

Then today I found another nice DL580, good price and all four CPUs installed. So I place my bid, someone bids over, ok I increase my bid by 30 pounds. Then 1 second before the auction ends some bastard snipes it right infront of my nose. DAMN!
eBay should really try to stop people from using these 3rd party softwares to fuck things over.
Looks like I have to get a machine from Germany now, some company there has like 30 DL580s in stock and seem to be quite alright to negotiate with. Hmm, perhaps I should get two more machine so I can get two max speced ones and also have a spair chassis in case things go face up.

PS. Anyone wanna trade 5x 36Gb FC-AL drives (and the bulky MetaStor array the come in) for HP/Compaq SCSI drives? :-)

Wednesday, November 9

PostgreSQL 8.1 released!

Yeay!

Finally ready for production, or at least intense testing.

Biggest news:
  • Role-based access control, similar to Oracle
  • Bitmap scans. Faster index usage, fewer tuples to check.
  • min() and max() can now come in on an index
  • Table partioning. Could be quite nice to have if you have huge tables. Think billions of rows.
See the full list here.

Go test check it out!
Download here.

Oh yes, PgAdmin 1.4 was released. Download that here.

Damn, I need more RAM for all my VMware instances.

Saturday, November 5

Oracle XE 10g beta public release

Oracle is launching step 2 (InnoDB was 1) in messing with MySQL with the launch of Oracle XE.
XE is free to use and free to distribute. Perfect to bundle with small software packages or when developing a website that needs scalability. Oracle guarantees application compatibility with SE/EE versions of Oracle.

I also think that the release of XE will bring more light to the cute little tool HTMLDB, for those of you who are not familiar with HTMLDB it is more or less an application server with a built in IDE. You can create a small web based application in the matter of minutes. Perhaps a small reporting interface or a timesheet system. A good example of a larger HTMLDB application is Ask Tom. Tom Kyte is by the way the main man endorsing XE. He is happily explaining how much better Oracle XE is than MySQL in this nice podcast.

Installing Oracle XE is a piece of cake on most systems, Oracle even acknowledge that it will work on Debian (spoke to benr on IRC and he mention that there where a few issues installing on Gentoo). A quick rpm -i and small config script sorted everything out on my Fedora Core 4 VM. Just connect. Well almost, the installer still has a few tiny bugs but those will probably be sorted before final release.
One annoyance is that there is no oraenv script in a decent path. The default env script is located in /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/bin/oracle_env.sh. Perhaps not the best choice if you ask me. The whole idea of installing under /usr/lib seems a bit silly.

Main limits
  • Will only utilize 1 CPU
  • Can only use 1 Gb RAM
  • Maximum USER tablespace size 4Gb.
  • All high availability and online juggling features are disabled
  • Only available for Linux and Windows on i386 so far
  • Only one instance per machine/container
  • No official support available

Main features
  • Free (doh!)
  • Easy to install using RPM in Linux. No more OUI.
  • Easy to use with HTMLDB, minimal settings to play with.
  • Almost full Oracle SQL feature set available.
  • Application compatibility with big brother guaranteed
  • Support community will probably be great


Read the FAQ [pdf].
Download Oracle XE beta release here and don't forget to register and visit the XE forums.