Not so sure about that scalability [LWN.net]

Not so sure about that scalability

Posted Jul 1, 2004 0:59 UTC (Thu) by roblatham (subscriber, #1579)
Parent article: The Global File System goes full circle

You make the claim "GFS is highly scalable, which means that hundreds of systems can share a filesystem on a SAN", and I'm curious where you got that information.

I have not seen or heard anyone report deploying GFS with more than 32 nodes. The jazz cluster at Argonne National Laboratory, for example, has 8 NFS servers export a GFS file system, but that is only 8 nodes accessing GFS at any time.

While you no doubt *could* have 100s of systems sharing a GFS file system, the expensive SAN infrasturcute you would require would make that a very rare configuration. The file-based locking mechanism makes GFS a poor choice for the large high performance clusters linux is so famous for.

Kudos to Redhat for releasing more code into the wild. People desiring a way to share a file system between two high-availabilty web or email server will rejoyce, no doubt.

(Log in to post comments)

Not so sure about that scalability

Posted Jul 1, 2004 7:30 UTC (Thu) by AnswerGuy (subscriber, #1256) [Link]

I would think that a GFS and perhaps NFSv4 or AFS could be combined into a SAN/NAS cluster product.

You'd have several nodes acting as file servers (to the network) and using a on the SAN that they share.

Not so sure about that scalability

Posted Jul 1, 2004 22:00 UTC (Thu) by seanegan (guest, #15672) [Link]

I work for a large three letter company with a RedHat support contract. RedHat came out to our campus to give a talk about GFS. I learned a couple things.

First, they said there are current deployments of GFS with more than 32 block device nodes. The limiting factor is the lock manager. They now have a load balancing and redundent lock manager servers. So rather than having a single lock manager server, you now have a small cluster of them.

Second, the locks are per-block not as you said per-file. And the locks are revokable if a host with a GFS FS mounted becomes uncommunicative the lock can be recovered from that host.

Third, you do not need a SAN (I'm thinging you meant Fibre Channel SAN) to use GFS. You can cost effectively set up a Gb ethernet LAN and use the GNDB server to serve blocks over TCP. I pressed them for an estimate of the speed up compared to NFS. The response was reluctant, but they said one customer measured speedups around 25 times faster than NFS on the same LAN. The presenter continued that it could be slower or faster than that number depending on usage but that it would never be slower than NFS.