Not logged in
Log in now
Create an account
Subscribe to LWN
LWN.net Weekly Edition for May 16, 2013
A look at the PyPy 2.0 release
PostgreSQL 9.3 beta: Federated databases and more
LWN.net Weekly Edition for May 9, 2013
(Nearly) full tickless operation in 3.10
for your (1) point... the good news is that hardlinks are relatively rare....
The 2006 Linux Filesystems Workshop (Part III)
Posted Jul 9, 2006 20:12 UTC (Sun) by PaulMcKenney (subscriber, #9624)
I suppose that one way to do this would be to hold some chunks aside, and to periodically re-chunk into these chunks, sort of like generational garbage collection.
But perhaps see how it does in tests and benchmarks?
Posted Jul 11, 2006 23:13 UTC (Tue) by dlang (✭ supporter ✭, #313)
the discussion was to split the disk into 1G chunks, but that can result in a LOT of chunks (3000+ on my home server :-). changing the chunk size can drasticly reduce the number of chunks needed, and therfor the number of potential places for the directories to get copied.
In addition, it would be possible to make a reasonable sized chunk that holds the beginning of every file (say the first few K, potentially with a blocksize less then 4K) and have all directories exist on that chunk, then only files that are larger would exist in the other chunks (this would also do wonders for things like updatedb that want to scan all files)
this master chunk would be absolutly critical, so it would need to be backed up or mirrored (but it's easy enough to make the first chunk be on a raid, even if it's just mirroing to another spot on the same drive)
this sounds like something that will spawn endless variations and tinkering once the basic capabilities are in place.
Posted Jul 13, 2006 18:15 UTC (Thu) by PaulMcKenney (subscriber, #9624)
But, yes, 3,000 chunks does seem a bit of a pain to manage -- at least unless there is some way of automatically taking care of them. But do the chunks really need to be the same size? Could the filesystem allocate and free them, sort of like super-extents that contain metadata?
If we explore all the options, will it get done before 2020? Even if we don't explore all the options, will it get done before 2020? ;-)
Posted Jul 23, 2006 19:12 UTC (Sun) by rapsys (guest, #39313)
I think that two idea are interesting as I readed some article about
independant read/write head in hard disk for future to improve the
I read that there is 3-4year about Video Hard Disc Recorder for TV (but
hdtv and drm may have killed such improvement ?)
The point is that if we are able to have 3-5 head on a hard disk, why not
mark one of that head surface to be reserved to backup control sum ?
(this would not kill performance if head are independant as disk bandwith
is far bigger than the mechanic part)
Posted Jul 23, 2006 19:40 UTC (Sun) by rapsys (guest, #39313)
The interest of the previous idea is if your data has been physicaly
corrupted (write overpowered, etc) in the other chunk, of the (group
of )block checksum will be wrong.
If the chunk of control sum is avaible and valid (see below), the kernel
issue a EAGAIN/ERESTORING to the application and regenerate the data from
the control sums' chunk.
It will allow to not loose a movie/music file because a stupid block in
the middle have been corrupted :'(
The more interesting things is that the hard disk will remap the
problematic magnic part to somewhere else because you re-write on the same
place and it's marked as trashed by hd.
(it's an assumption that need to be true every time and hd manufacturer
NEED to respect, after few write/read cycle test scheduled by firmware on
that place maybe)
The only problem I see if an overpowered writes cross the chunk
The control sum will be corrupted and will lead kernel in trouble in case
of birthday paradox.
So we will need to have (group of )block checksum for the control sums'
(Will it fit in space with equation : x(data)+1(control sums) ?)
I made two assumption :
- read/write to that chunk is not too costly (independant head, etc)
- control sum are not cpu killer (maybe have a special feature in hd to do
- control sums of chunk are not cpu killer too (but we only increase the
consumed cpu time for each write on disk)
Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds