User: Password:
Subscribe / Log in / New account

MeeGo and Btrfs

MeeGo and Btrfs

Posted May 11, 2010 22:37 UTC (Tue) by lbt (subscriber, #29672)
Parent article: MeeGo and Btrfs

FWIW I tried btrfs on a 1Gb mmc on a device running meego

I was using 2.6.33

I got disk full messages and yet df showed:
/dev/root 920M 651M 269M 71% /

So I went and asked about it ... some chat in #btrfs got me:
"I was told my fs was full. I just got a chance to do a du on it...585M"
"thats probably right"
"269m in use for metadata sounds correct"
"you will want to update your kernel to something 2.6.34 ish"

"OK, so btrfs is really not a good choice for small disks/devices not at ~50% overhead"
"yeah basically"

At the time I estimated 50% I wasn't clear on the metadata measurement bug in my kernel. I guess it's more like 25% used by metadata. I'd be interested to know if this is a percentage overhead or a lump-sum.

I'm really interested in many of the btrfs features - and on the desktop I may have volumes I'd make this tradeoff for in a hearbeat... not so sure about on my phone.

(Log in to post comments)

MeeGo and Btrfs

Posted May 11, 2010 23:07 UTC (Tue) by elanthis (guest, #6227) [Link]

Even on my phone, it's probably not a big deal. I've got 16G on my phone and I'm using all of 300M for personal files and such, plus the core OS and some apps. The worst thing I can possibly see about btrfs in that regard is if manufacturers go around claiming the phone has 16G of storage when in practice it can only hold 8G, but in all honesty that's a problem today; the device manufacturers don't spell out how much of the storage is used up by the OS and core applications, so you're not getting anywhere near the full storage capacity for personal use no matter what.

MeeGo and Btrfs

Posted May 12, 2010 0:23 UTC (Wed) by Fowl (subscriber, #65667) [Link]

An interesting point, but I know I love to fill my phone to within a few hundred MB of full with music and videos. If that stopped the phone bit working, I'd be most annoyed.

MeeGo and Btrfs

Posted May 12, 2010 7:13 UTC (Wed) by eru (subscriber, #2753) [Link]

Me too... One major use for mobile phones and similar is snapping photos and videos, and these take space, especially as the quality is going up these days (latest models can film at HD resolution). I could live with a 16G flash giving me 14G, but not if it really allows only 8G.

MeeGo and Btrfs

Posted May 12, 2010 13:29 UTC (Wed) by nix (subscriber, #2304) [Link]

What on earth are they using that much space *for*? It can't really be metadata, not at half the size of the data. Is it old not-yet-GCed COW copies or something?

MeeGo and Btrfs

Posted May 12, 2010 17:07 UTC (Wed) by dlang (subscriber, #313) [Link]

this seems like a significant problem

while there are a couple spots where capacity is significantly above usage (personal desktops using TB size drives, phones holding audio files), there are MANY other situations where people routinely run up against size limits

think of laptops running SSDs

think servers with multi-TB raid arrays.

in both cases using significant amounts of space for metadata will significantly hurt you.

I have quite a few systems, and many of them do routinly run up against drive size limits. These tend to be personal machines with <150G (SSDs, laptops, high-speed SCSI drives), or servers (where I have multi-TB raid arrays, but they are sized for what I am storing n them)

loosing 30% of the space to metadata would be a problem.

MeeGo and Btrfs

Posted May 13, 2010 1:56 UTC (Thu) by Fowl (subscriber, #65667) [Link]

Another thing I just thought of is that having that much metadata makes it that much harder to keep it all in memory.

That would be fairly key to keep latency down I would think.

MeeGo and Btrfs

Posted May 20, 2010 8:23 UTC (Thu) by mfedyk (guest, #55303) [Link]

I remember reading this in #btrfs a while back. It seems you missed the full explanation of what is happening here.

(Btw, #btrfs can be very quiet at times and quite active at other times. Many times people join the channel and ask a question but leave before someone who can answer has seen it. Stick around in-channel for at least 12 hours, there are people in many different time zones.)

Btrfs sections areas of block storage like spinning disks, SSDs, etc. into chunks. You can think of them like block groups in ffs and ext*, but with more capabilities. There are meta-data chunks and data chunks among other types of chunks. The size of the chunks are fixed size once allocated until a rebalance (it can be run while the volume is online, no need to unmount) operation is invoked. I suspect your meta-data chunk was 256MB.

The size of btrfs meta-data can be a bit misleading. Btrfs does tail-packing on the end of files, and if the file is small enough it will be stored entirely in the meta-data chunk.

The numbers reported to df have been a bit misleading until recently. It used to be that used only counted data blocks and free counted all types of free space including free space in the meta-data chunks which couldn't be used for full sized data blocks (only small files and tails). Now used is counted as <used data blocks> + <used meta-data blocks> and free is counted as <free data blocks> + <block space not allocated to any chunk>. This cleared up most of the ambiguity where df was reporting free space, but apps would see disk full errors. (I'm not sure if that made it into 2.6.33.) Though if df reports zero free you can still write small files that fit into tails because that goes into the meta-data chunk.

A patch to mkfs.btrfs and the kernel module that scales the size of meta-data chunks based on blockdev size would be a welcome contribution from the meego developers I suspect.

The Fedora 13 kernel has a more recent version of btrfs than stock 2.6.33 has and the current advice is to run the latest btrfs from git so that duplicate bug reports aren't reported for issues already fixed.

You can find more details of your btrfs volume with these commands:

(or "btrfs fi sh" in latest git btrfs-progs)

btrfs fi df <path to btrfs volume>
(in latest git btrfs-progs and recent btrfs kernel module)
"fi" is short for "filesystem" and can be shortened as long as it is unambiguous.

In short, it looks like meego will only be using the features that work well in btrfs right now and I suspect with their somewhat narrow use cases it has a good chance of working well for them right now. On that note the btrfs project can always use more patches, testing and documentation.

(I have been testing btrfs for the last few months and I run as my root filesystem on my desktop and laptop. I do not represent the project and only explain what I think I understand of the project and filesystem. Please let me know if you have found any errors.)

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds