LWN: Comments on "Ext3 for large filesystems"

Ext3 for large filesystems

yaojingguo — Fri, 05 Jun 2015 03:54:52 +0000

Wikipedia says that if the block size is 4KiB, the max ext3 file size is 2TiB not 4TiB. And a reading of ext3_max_size function in linux/fs/ext3/super.c confirms Wikipedia's opinion.

Ext3 for large filesystems

jzbiciak — Tue, 20 Mar 2007 16:35:30 +0000

It still amazes me that a single desktop icon on a modern computer takes up about as much RAM as an entire video game console had back in the 80s.

64MB in the early 80s? Those must have cost a pretty penny. I remember 5MB and 10MB disks costing several hundred dollars.

Ext3 for large filesystems

BrucePerens — Tue, 20 Mar 2007 16:32:30 +0000

Not too long ago, Unix systems replaced seek() with the lseek(), and were able to handle longer offsets even though they were 16-bit machines. The word width of your CPU in no way limits the width of words upon which you can perform arithmetic. The new 48-bit field is not using compiler types wider than 32 bits for its implementation. They could use "long long", but I suppose that's less portable.

Bruce

Ext3 for large filesystems

BrucePerens — Tue, 20 Mar 2007 16:24:31 +0000

I think I've thrown out the 64 Megabyte 3.5 inch hard disks I bought in the early 80's. It doesn't feel that long ago.

Ext3 for large filesystems

sandeen — Mon, 12 Mar 2007 15:38:51 +0000

> My understanding is that even XFS has a maximum filesystem size of only 16TB on 32 bit, even though it goes to 9 EB on 64 bit systems.

Though xfs is fully 64-bit capable on 32-bit machines, the 16T limit is OS-imposed. There is a 32-bit index into the (4k) page cache on x86, so 2^32 * 4096 is 16T is the maximum offset that can be addressed.

Petabytes

dlang — Sun, 15 Oct 2006 21:32:24 +0000

actually, you can get a petabyte of storage for ~$2m off-the-shelf that fits in ~4 racks consuming ~30KW of power

buy 40 Sun x4500 servers ( http://store.sun.com/CMTemplate/CEServlet?process=SunStor... ) each with 24TB storage, 2 dual-core opterons, and 16G ram

so 2PB of storage are $4m in 8 racks (80 servers) consuming ~60KW. that's not too scary (although it is quite a bit)

backups are an issue, my guess is that the backup is mirroring to another similar set.

Petabytes

knan — Sun, 15 Oct 2006 15:55:53 +0000

"For ther former you would need something like 2048 teraservers" ... or a few modern SAN storage arrays. There are 500TB off-the-shelf solutions available, and I'm sure the manufacturers will be happy to sell you bigger custom variants.

Extents

klog — Fri, 13 Oct 2006 20:28:35 +0000

Woohoo!
IBM's "Old Iron" has worked using extents for 40 years...

Ext3 for large filesystems

sitaram — Fri, 07 Jul 2006 04:00:59 +0000

well I waited long enough for someone else to remark on this...

"4 TB - big, but perhaps limiting for today's hot applications (such as comprehensive, nationwide telephone call archival)"

Once again, our editor's sense of humour hits home :-)

ext3 block size

csnook — Sat, 01 Jul 2006 18:26:51 +0000

In the current block layer, the block size must be no larger than the page size. Since page size is 4k on nearly all architectures, 4k blocks are the sane way to go. I suppose if you gave Itanium boxes to a bunch of kernel developers to test on, they might be inclined to work on larger block sizes.

ext3 block size

riel — Sat, 01 Jul 2006 14:13:55 +0000

Raising the block size has a number of consequences for the memory management subsystem in the kernel, as well as the I/O subsystem.

Furthermore, it would mean that even small files would take up an unreasonable amount of disk space...

Petabytes

man_ls — Sat, 24 Jun 2006 17:39:47 +0000

If you could expand on your post, I'm very curious about it. 2 Petabytes? 32 Petabytes? Where do you store that kind of information? For ther former you would need something like 2048 teraservers, just the power requirements are scary. How do you back it all up?

ext3 block size

dmuino — Mon, 19 Jun 2006 21:47:13 +0000

Why is the ext3 max block size == 4k?

Wouldn't just raising EXT3_MAX_BLOCK_SIZE at least temporarily alleviate this problem?

Obviously the performance issues dealing with large files are still there, and extents is a feature that becomes more and more important, but if you need something now that gives you filesystems larger than 8TB this might be something to explore.

Ext3 for large filesystems

sbergman27 — Fri, 16 Jun 2006 00:33:01 +0000

Does this all mean that ext3/4 with these patches would handle 1024 PB on a 32 bit system? Or only on a 64 bit system?

My understanding is that even XFS has a maximum filesystem size of only 16TB on 32 bit, even though it goes to 9 EB on 64 bit systems.

Ext3 for large filesystems

Stephen_Beynon — Thu, 15 Jun 2006 22:04:11 +0000

This is from memory so I may be wrong ...

In the beginning Linux used the minix file system. This had many
limitations including as I recall 14 character filename limits.

The EXTended filestem was developed to address these problems. This was
relatively rapidly replased by the second extended filestem (ext2) (I do
not recall what the problems with ext1 were).

Ext3 extended Ext2 with new capabilities - most notably a journel. This
considerably improves the resilience of the filesystem to unexpected power
failures !

Ext3 for large filesystems

xorbe — Thu, 15 Jun 2006 21:45:53 +0000

Wait a sec... what's the "ext" in ext2 and ext3 if they are just adding
extents now??

Ext3 for large filesystems

jzbiciak — Thu, 15 Jun 2006 16:58:00 +0000

Well, if storage requirements double every 18 months, then 1024 PB will be enough to hold them for just shy of a decade, then. :-)

Ext3 for large filesystems

smoogen — Thu, 15 Jun 2006 15:29:24 +0000

Actually large amounts of cluster systems seem to run into the 4TB file size and 8TB file limit these days. We had a project that needed approximately of 32 Petabytes for its raw data. I thought it was outlandish, but then got a couple of requests for 2 Petabyte systems for genome research.