Our bloat problem
Our bloat problem
Posted Aug 4, 2005 13:35 UTC (Thu) by elanthis (guest, #6227)In reply to: Our bloat problem by eru
Parent article: Our bloat problem
That isn't generally how a terminal scrollback buffer works, however. You generally work with blocks of memory, so even blank areas on lines are filled in with data. That's required due to how terminals work in regarding to character attributes. Which also brings up the point that you have more than just character data per cell, you also have attribute data. And then let's get to the fact that in 2005, people use more than just ASCII, and you actually can't use only a byte per character, but have to use something like 4 bytes per character in order to store UNICODE characters.
So if you have an 80 character wide display with 100 lines of scrollback, and we assume something like 8 bytes per character (4 for character data, 4 for attributes and padding) we get 8*80*100 = 640000. And that's just 100 lines. Assuming you get rid of any extraneous padding (using one of several tricks), you might be able to cut down to 6 bytes per character, resulting in 6*80*100 = 480000. Almost half a megabyte for 100 lines of scrollback.
More features requires more memory. If you want a terminal that supports features that many people *need* these days, you just have to suck it up and accept the fact that it'll take more memory. If you can't handle that, go find a really old version of xterm limited to ASCII characters without 256-color support and then you might see a nice reduction in memory usage. The default will never revert to such a terminal, however, because it flat out can't support the workload of many people today, if for no other reason than the requirement for UNICODE display.
Posted Aug 4, 2005 13:44 UTC (Thu)
by tialaramex (subscriber, #21167)
[Link]
Posted Aug 4, 2005 16:21 UTC (Thu)
by eru (subscriber, #2753)
[Link]
But that is a very wasteful implementation choice. There are several other
ways of doing it (like the linked list I proposed) that are not much more
complex to program. I forgot about attributes in my original post, but they,
too can easily be represented in ways that average much less than 4 bytes
per character. And as another poster pointed out, you can store Unicode
with less than 4 bytes per character. In today's computers the CPU is
so much faster than the memory that it may not pay to optimize data
structures for fast access at the cost of increased size.
I think this difference illustrates a major reason for the bloat problem:
using naive data structures and code without sufficient thought for
efficiency. Maybe OK for prototypes, but not after that. I am not advocating
cramming data into few bits in complex ways (as used to be common in the
days of 8-bit microcomputers), but simply avoid wasting storage
whenever it can be easily done. Like, don't store boolean flags
or known-to-be small numbers in full-size ints, allocate space
for just the useful data (like in the scroll-back case), don't
replicate data redundantly.
I wonder if the well-known GNU coding guidelines (see Info node "standards" in Emacs
installations) may be partly to blame for bloat problems in free software... To quote:
If a program typically uses just a few meg of memory, don't bother
making any effort to reduce memory usage. For example, if it is
impractical for other reasons to operate on files more than a few meg
long, it is reasonable to read entire input files into core to operate
on them.
Right, but what when you have lots of programs open at the same
time, each using "just a few meg of memory"? (I recognize Stallman
wrote that before GUI's became common on *nix systems).
Posted Aug 4, 2005 19:29 UTC (Thu)
by berntsen (guest, #4650)
[Link]
/\/
You don't need 4 bytes per character for Unicode in most places. A brief examination of the unicode xterm shows that, as expected it doesn't actually store everything as 32-bit ultra-wide characters. Most strings can be stored as UTF-8, a few places might deal with the actual code point and have a 32-bit integer temporarily, but certainly not huge strings of them.Our bloat problem
That isn't generally how a terminal scrollback buffer works, however. You generally work with blocks of memory, so even blank areas on lines are filled in with data.
Effect of Implementation Choices
Memory Usage
============
If people malloc like you multiply, I see where the bloat is comming from, you have a factor of 10 wrong ;-)Our bloat problem