LWN: Comments on "Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)"

Tool detects library bundling?

cmccabe — Tue, 29 Nov 2011 01:17:24 +0000

Most of Chrome's userbase is running Microsoft Windows or Apple iOS. These operating systems don't have dependency management. Google had to build their own updater from scratch and bundle copies of all of their own libraries. In a lot of ways, what they created is similar to an OS within the OS.

They could have done things a lot differently on Linux, but it was easier just to reuse the same code on all three operating systems.

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

jrn — Fri, 25 Nov 2011 22:18:28 +0000

> However, git's packfile delta-detection mechanism doesn't scale well when asked to look at very large files, and if you chop it into chunks you have the problem that an insertion near the start of the file throws off all the chunk boundaries.

... which is why bup uses a rolling hash to decide where to cut.

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

nix — Fri, 25 Nov 2011 18:36:14 +0000

Yeah, figuring out an algorithm that has acceptable performance has been fun.

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

nix — Fri, 25 Nov 2011 18:35:27 +0000

Yep, I'm stealing ideas from bup as well. However, git's packfile delta-detection mechanism doesn't scale well when asked to look at very large files, and if you chop it into chunks you have the problem that an insertion near the start of the file throws off all the chunk boundaries. These problems *are* soluble, but you can't keep using git if you do that...

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

mjthayer — Fri, 25 Nov 2011 14:15:03 +0000

> There is an academic paper on the site where you can download the code. No secret sauce here.

Is the hash/signature database that he is matching against also available somewhere? I couldn't seem to find a link.

I think APNG was a good move

MaxSt — Thu, 24 Nov 2011 15:44:49 +0000

PNG is an extensible format, it allows third-parties to create PNG extensions by inventing their own chunks. So who can blame Mozilla for creating "acTL", "fcTL" and "fdAT" chunks... But it makes little sense to try to include every possible extension into PNG standard and libpng. Mozilla should have asked libpng developers to provide easy-to-use callback interfaces for handling third-party chunks outside of libpng.

I think APNG was a good move

CChittleborough — Thu, 24 Nov 2011 12:15:34 +0000

As I understand it, APNG was a quick-and-somewhat-dirty hack to solve an aesthetic problem: Mozilla used animated GIFs for their throbbers, but GIFs don't support transparency. (If you accept that Mozilla "needs" to compete with IE/Safari/etc on appearance, you could even say that Mozilla needed an animation format that supported transparency.)

Mozilla could have brought libmng back, but the 5 reasons they gave for dropping it in 2003 (see the first comment here) are all still valid, plus there are security risks in supporting a large library which reads and interprets complicated files from the web. I think that they were wise to design a no-frills variant of PNG and hack support for it into libpng, and that they were also wise to support APNG images in web content, not just in chrome. OTOH, I'm glad the PNG guys did not accept APNG into the PNG standard, because it is fairly crude. YMMV.

Here's hoping Mozilla does a better job of following any future bug fixes to libpng.

Tool detects library bundling?

MaxSt — Thu, 24 Nov 2011 09:55:05 +0000

Mozilla never suffered any fallout, and you'll be surprised how many people downloaded APNG plugin for Chrome.

MNG never took off and SVG will kill it

CChittleborough — Thu, 24 Nov 2011 03:31:27 +0000

MNG is a full-featured 2D raster graphics system. It allows you to create sprites, which can be static images or animations (sequences of images), then make them move around the viewport, with layer-based compositing. It uses two image formats in addition to good old PNG: Delta-PNG for compressing a sequence of PNG images, and "JNG" (JPEG Network Graphics), for JPEG compressed images in a PNG-style chunk format with an optional alpha channel. It has "commands to move, copy and paste images (rather than replicate them as in GIF)", plus "nested loops for complex animations" (quoting from www.libpng.org/pub/mng/).

In short, MNG has pretty much everything you could ask for in a raster-based 2D animation system. But it lacks something important: artist-friendly MNG-oriented editing software. (There are popular video editors that can output MNG, but they have their own native format. There are tools for creating/editing MNG content, but none of them ever became popular.) The result is a marked shortage of MNG content of interest to the wider populace. As a result, there is not much demand for MNG-viewers ... and the vicious cycle goes on.

Another factor is that SVG+javascript can do everything MNG can do (except JPG+alpha images) and a lot more besides, and all you need is a text editor. (It may be called Scalar Vector Graphics, but it is good for raster images as well.) For non-programmers, SVG has purely declarative animations that are as powerful as MNG (AFAIK), but IE and Webkit don't support SVG Animations (yet?). (MNG does have one advantage: one file contains the whole thing, whereas SVG practically requires the raster images to be delivered separately.)

So those of us who liked MNG are going to be disappointed. Sigh.

Tool detects library bundling?

Lennie — Wed, 23 Nov 2011 23:22:52 +0000

"This is why you shouldn't bundle libraries, isn't it?"

Tell that to the Chromium/Chrome guys, they bundle everything.

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

khc — Wed, 23 Nov 2011 21:27:08 +0000

I haven't looked into the details of ssdeep, but from the abstract of the linked publication (http://www.sciencedirect.com/science/article/pii/S1742287...) it looks like something that I am familiar with, which definitely predates the publication date. Using the algorithm as the basis of a deduplication backup system is a good idea, but I think if/when you implement it you will find making it perform well maybe harder than it seems.

(I work on a deduplication storage product)

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

martinfick — Wed, 23 Nov 2011 21:06:10 +0000

Have you looked into BUP? It already does inner file level deduplication in backups.

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

gmaxwell — Wed, 23 Nov 2011 19:57:39 +0000

It's just fuzzy source code matching, nothing as advanced as coverity/clang-scan-build. The idea is that the same vulnerabilities get copied into lots of code, so if you find it once you can fuzzy search to find it other places.

There is an academic paper on the site where you can download the code. No secret sauce here.

Developers and dancing pigs

dmarti — Wed, 23 Nov 2011 18:10:23 +0000

Yes, the original dancing pigs rule was about end users, but you can apply it to developers, too. "Given a choice between coding for security and dancing pigs, developers will implement the dancing pigs every time."

Tool detects library bundling?

mathstuf — Wed, 23 Nov 2011 14:15:44 +0000

AFAIK, APNG is just a string of png files with a delay and a loop chunk (there's probably more, but the point is that it's simple). MNG is not and instead has loops, logic and other constructs.

http://www.libpng.org/pub/mng/spec/

Tool detects library bundling?

clugstj — Wed, 23 Nov 2011 13:39:57 +0000

Well, no. There was never a "need" for APNG. They chose to fork libpng (instead of using MNG), so they are responsible for the fallout from this choice.

Tool detects library bundling?

sammythesnake — Wed, 23 Nov 2011 13:37:43 +0000

What happened to MNG? I thought that was going to be the animated PNG equivalent, and I'm sure I heard of that many years ago (8+?)

http://www.libpng.org/pub/mng/

Cheers & God bless
Sam "SammyTheSnake" Penny

Tool detects library bundling?

tialaramex — Wed, 23 Nov 2011 09:42:50 +0000

Sure, but you can see why it's hard to avoid the joke, right? I mean, APNG is exactly suitable for displaying dancing pigs. And dancing pigs are closely associated with bad decisions made in favour of features no-one really needed over security.

Tool detects library bundling?

renox — Wed, 23 Nov 2011 08:31:43 +0000

Well not really, "dancing pigs" is about users' behaviour, here this is the developers who chose to fork a library for APNG support and don't monitor closely the source library which is a shame.
I think that we can safely say that 99% of users don't care about APNG and they never even heard of APNG..

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

linusw — Wed, 23 Nov 2011 06:41:54 +0000

This thing seems to be a static code analysis tool similar to Coverity, which indeed also helps to fix a few bugs out there. But - and this is the crucial question - do these static analysis tool companies pool their knowledge or are they all doing proprietary secret sauce in their code repositories and fuzzy databases? Imagine what they could do with an open community exchanging ideas.

It feels to me like right now doing tools like this seems like a pretty easy way to attract venture capital.

Is there some independent scientific scrutiny on the effectiveness of each tool of this type? Such as if a researcher would input a corpus of code with known bugs and see which tools identified which bugs? With such "benchmarks" they could gain a higher level of trust.

Tool detects library bundling?

dmarti — Wed, 23 Nov 2011 00:08:49 +0000

Hooray! dancing pigs!

Tool detects library bundling?

pabs — Tue, 22 Nov 2011 23:53:18 +0000

They need it because they forked libpng to add APNG support, which the libpng authors refuse to accept:

http://en.wikipedia.org/wiki/APNG

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

pabs — Tue, 22 Nov 2011 23:06:42 +0000

As someone who has been helping maintain Debian's massive embedded-code-copies file, using this tool would be a major step up from existing practices.

http://wiki.debian.org/EmbeddedCodeCopies

We're already checking packages and binary files for a limited number of different embedded code/data situations, but using this ssdeep-based tool would be be a much-welcome complement to these efforts.

http://lintian.debian.org/tags/embedded-feedparser-librar...
http://lintian.debian.org/tags/embedded-javascript-librar...
http://lintian.debian.org/tags/embedded-library.html
http://lintian.debian.org/tags/embedded-pear-module.html
http://lintian.debian.org/tags/embedded-php-library.html
http://lintian.debian.org/tags/duplicate-font-file.html

As a result I've mailed the Debian security team and also the researcher to see if we can get his work ported to Debian and run over the archive on a regular basis.

http://lists.debian.org/debian-security/2011/11/msg00044....

Tool detects library bundling?

dmarti — Tue, 22 Nov 2011 22:55:13 +0000

"Mozilla Firefox had embedded libpng and was vulnerable for more than three months after the flaw was discovered."

This is why you shouldn't bundle libraries, isn't it?

Tool kills hidden Linux bugs, vulnerabilities (SC Magazine)

nix — Tue, 22 Nov 2011 21:11:14 +0000

ssdeep is potentially useful for all sorts of other fuzzy matching stuff, too, even though its authors were focused on vulnerability scanning. (I'm planning to reuse the algorithm in a deduplicating backup system, if and when I ever actually write it rather than just tossing designs about.)