Recently posted comments

What I find interesting is that the traditional BSD sysctl interface uses integer constants. And while most distributions have disabled the sysctl syscall, AFAIU /proc/sys was and continues to be just another interface to the same underlying mechanism. Are newer knobs no longer exposed via the old syscall interface? I ask because the new BPF interface exposes string paths, which seems not just unwise but possibly unnecessary if there already exist fixed integer identifiers.

BPF is already running amok and it's basically impossible to debug when it inevitably goes wrong. You want to limit containers? Work with existing LSM folks. If the existing LSMs are not enough then extend THEM.

Oh, I see. People think that LSM equals unusable SELinux and LSM multiplexing is still not a thing.

I'm a privacy advocate, but I don't see anything wrong with encrypting the url in response to user agents not honoring the ping attribute and stripping redirects.

/me glares in pulseaudio's general direction...

The choice of directory location is unfortunately vague and didn't age well, but at least it's an attempt.

A hard SSE2 requirement in glibc would be the fault of whoever built it. The library can still work fine on CPUs as low as i586 (Geode and similar) — I don't think there are any plans to break Linux userspace on a whole line of fairly popular coreboot-supporting routers.

And if that day actually comes, musl is in pretty good shape already…

That said, rmayr above is right, this is not a technical discussion, so I will stop here.

On the other hand, sml, I do not consider myself being a troll - the above statements are not intendent to provoke a flame war or anything, these are my genunie opinions. I am sorry if they do not match your narrow view of the universe.

No offense meant against you personally, but the fact that people/companies feel the need to do such shady things is an indicator of a very sorry state of affairs.

I have zero qualms about breaking programs that are so obviously broken themselves. You don't free resources you didn't create: that's a rule for any system, not just unix. Just no. Let's no enable programs that break this rule.

You say that other cases "have to be dealt with"...but we have widely used examples showing that to not actually be the case.

If you spoof a chrome user-agent though, then don't send you the evil google redirect, which means you can copy urls no problem.

However, with a chrome user-agent youtube won't work anymore, because it will use chrome-specific extensions:

https://fossbytes.com/google-accused-of-sabotaging-micros...

(If pidfds also wake you up when non-termination waitpid() results would be returned iff you did a waitpid(.., WNOHANG), then they do seem like a complete replacement for waitfd, which is great because it means I can drop another annoying invasive patch!)

btw, this does mean that you'd need to be able to get a pidfd not only at clone time but also at PTRACE_SEIZE time, or tracers could never get hold of a useful pidfd at all...

"A general purpose library", IOW, some random, binary only code with undefined behaviour (in the sense that no specific behaviour is ever defined when this 'argument' shows up) is a situation which cannot be handled.

With Linux, one can even build programs which are entirely signal-driven by using realtime signals for I/O readiness notification (see fcntl(2)) and sigwaitinfo instead of one of the file descriptor based I/O multiplexing calls. I've used that once for a moderately complex Perl program (about 25 kloc) because it was the easiest to use in this environment.

[1] It's supposed to be a matter of just installing specific packages. But maybe not? I don't know!

These functions will typically be called under memory pressure. Then reallocation cost can be quite high. Also, it should be easy to change the implementation in the future. It does not belong to the userspace interface. If one wants to implement a complete defrag at some time in the future, this should probably be changed. However, I do not really see a reason for a full defrag here. All objects in the dentry cache have the same size, so newly allocated entries can always go into existing holes.

Reminder that comment filtering is available from your LWN account page.

[0] https://lwn.net/Articles/766080/

Unfortunately, the change is to make privacy worse for everyone in the same way Microsoft sending DNT by default did.

In what way are the Unicode case-folding rules rather complex? They are for the most part fairly simple, one to one matchings of characters, with a few exceptions that you just have to deal with. The German ß and the various titlecase characters in Unicode are there and are going to have to be dealt with.

Sometimes it's a problem to delay all signal handling until you return to the event loop; sometimes you really want signal handlers to be able to run during some long synchronous operation.

It doesn't solve the signal coalescing problem at all, making the data in siginfo impossible to use reliably in many situations.

It doesn't solve the problem that there are a finite and rather small set of signals available and you basically have to allocate them statically with no protocol for avoiding or handling conflicting uses of a signal.

Signals are basically impossible to use correctly.

There's only one case where the behaviour of POSIX calls is undefined wrt signals: If a signal handler which interruped an unsafe function calls an unsafe function. The I/O-multiplexing calls are async signal safe, hence, the usually sensible way to marry signal handling and an I/O multiplexing loop is to block all handled signals, install signal handlers for them and then use pselect/ ppoll/ epoll_pwait to unblock handled signals only while the code is waiting for I/O. Signal handlers are then free to call whatever function they want as the sole interrupted function will be async signal safe. And the only function which can terminate with an EINTR condition will be the multiplexing call. Problem solved[*].

[*] Signals mask are inherited, hence, code creating new process has to deal with that. Just like it has to deal with all other inheritable things. BFD.

For example, a general-purpose library can't even do something as simple as create a process and wait for it to finish. This is just ridiculous.

It is arrogant to demand that Mozilla set Firefox default preferences to suit the needs of the incredibly rare user who blocks redirects.

> Mozilla does some things with the design of Firefox that do not maximalize privacy. There may be other reasons they do that, but it is reasonable to conclude that it is likely Google's money influences those decisions.

That's not reasonable at all, given that there are many things Google would like (or would have liked) Mozilla to do that Mozilla doesn't do (e.g. adopt Chromium, abandon Tracking Protection, support PNaCl). You have no evidence that something nefarious is going on and apparently expect Mozilla to somehow prove the complete absence of such influence. If you want Mozilla to turn down Google money on principle and go broke "for the good of the Web", then you are misguided because that wouldn't be good for the Web at all.

Honestly I don't think your position is rational at all. If Firefox shipped ping by default you'd disable it, sites would use it, you could easily see which sites and links are trying to use it, and you'd deal with less Web breakage because you would have to do less blocking of redirects. What's not to like?

So what would change with ping in mails or forums is precisely...nothing.

Redirects in mails are a "best-practice" for quite some time, for various reasons. Professional ESP add them, because interaction with those links is a valuable indicator if the sending customer is clean (low interactions indicated that the mails are unwanted). Additionally it is a marketing analytics tool they can sell.

Phishing uses redirects as well, because the indirection can delay take-downs or slow down recognition by spam filters if multiple domains are used.

It does seem rather incongruous to me to justify the feature via by pointing to samba's emulation of NTFS case folding, and Android's emulation of FAT file name lookup rules, but then implementing unicode normalization and correct unicode case folding...which those don't do.

Lot's of sites work with redirects blocked. Whether it's too annoying to have on by default depends on the mix of sites one typically visits.

Google gives Mozilla money. Google is very interested in being able to track what people do on the web. Mozilla does some things with the design of Firefox that do not maximalize privacy. There may be other reasons they do that, but it is reasonable to conclude that it is likely Google's money influences those decisions.

I usually don't send a UA (it's officially an optional header), but pretending to be using a browser that doesn't allow disabling ping while actually disabling ping might be an easy way to avoid most link tracking. But that assumes the web site cares more about how the user preceives performance than being able to be able to track people, since the clients can't be trusted to honor ping, no matter what browser they claim to be.

That may be, but FAT, exFAT, and NTFS don't use the unicode case folding rules. If the justification is to make something compatible with those systems, do we actually need the (rather complex) unicode rules?

lynx presumably doesn't do js or images unless specifically extended and configured to do so,
and likewise mutt and emacs M-x eww, but who is keeping an eye out so it doesn't get snuck in somehow?

If it did get snuck in, how much damage could happen before it became a CVE and got corrected?

The question is what can <your idea of bad guys> do if they can insert involuntary action triggers
like ping= into email streams or html streams passing through their computers.

Ransom notes that you can't help acknowledging that your browser or gui email reader has seen? Nice ;-/

To get bad stuff to your html renderer, maybe more simply than owning the computers, buying an ad slot
and preparing html ad material for automatic insertion -- I have seen lots of ads injected locally by isps even
as I browse a foreign web site, and guess the local isp is getting paid by the advertiser to insert their stuff.
(idk how all that is arranged, and who sanitizes what, and who determines the final hrml my browser sees).

BTW, what would LWN do if I posted this post in HTML mode and included a ping= ? Could I do my own analytics
on the reading of my post? ;-)

Any such extension could also be written for the ping attribute and would be complete site-agnostic, simpler and robuster.

The cost to the site operator is actually identical for redirects and ping. However, for the user, pings are cheaper, because the ping and the request to the link target can be done simultaneously while for redirects the client needs to wait for the first response to know the real target. Add to that the cost of the Javascript. Add to that the issue of broken link copying.

I doubt disabling redirects in general is actually feasible. Redirects are a core concept of HTTP and disabling them would break countless legitimate use cases (asking for permission would be quite annoying).

That's not very problematic (from an API perspective it is very weird, but not internally), but my idea was pidfd_open could take some wait flags as waitfd did (and check if you're one of parent or real_parent, and only return a readable descriptor), among other things, and that intent cannot be described easily when opening a directory in /proc. This means everyone can poll but perhaps only parents (real or tracing) would be able to open a readable instance.

I can only hope this is taken into consideration (I raised this exact problem as I ran into it as well), and /proc descriptor stuff be removed, as it brings more problems for future extensions.

You could ofcourse resurrect a waitfd too, again, but I see no point when you could return a pollable/readable descriptor from clone and pidfd_open (and possibly even disable termination signals and autoreap them - the whole act of waiting is asynchronous). It also has a nice touch of consistency to it (and the fact that resources in a different namespace suddenly aren't addressable and can be taken a reference to from a different namespace - the filesystem), in that sense pidfd_open isn't very different from open, but it works like open only on PIDs you can *see* (unlike /proc, which is leaky across shared mount namespaces but separate PID namespaces).

It's actually the opposite situation. It's impossible for users to distinguish tracking redirects from other kinds of redirects, if they notice the redirect at all. On the other hand, with "ping" a single line of user CSS (e.g. delivered by a browser extension) will show you which links are tracked --- before you click on them:

a[ping]::before { content: "(tracked) "; }

> Forcing redirects on the service, makes them pay more for tracking (to handle more requests or to proxy requests) which discourages tracking.

Using redirects or "ping", the site has to handle the same number of requests. There is almost no difference in server work required. There is no evidence disabling "ping" discourages tracking in practice.

> It also makes it more obvious to the user if they have redirects blocked or needing approval.

Essentially nobody blocks redirects. Redirects are used for many things, many of which are essential to the function of sites (e.g. OAuth), so blocking them breaks the Web. "ping" on the other hand is a much narrower feature which makes far more sense to block.

> Mozilla doesn't do the right things because they get a lot of money from a company that wants tracking to be easier.

As someone who worked there for a long time but doesn't now: this is just not true. You don't have any evidence for such a connection, and you are just making stuff up.

> Arguably ping shouldn't have been implemented at all, because almost all sites that want that information don't want to let people opt out of it, so need to do it server side instead of client side anyway, leaving very few users of the feature.

That may have been true up to now, but now that Safari and Chrome prevent disabling "ping" I expect sites to use UA-sniffing to use "ping" instead of redirects for those browsers ... leaving Firefox users with a worse user experience for no privacy benefit.

Otherwise they will just continue to do tracking via redirects (and then, optionally, hide the redirects again via Javascript) and get *exactly* the same information.

As a demonstration of this, visit google.com and search for "lwn".

Then admire the nice, first link seemingly pointing to https://lwn.net.

Then use the "copy link to clipboard" function and paste it in the address bar.

Then realize, that the link actually is https://www.google.com/url?sa=t&rct=j&q=&esrc... (will likely vary).

Then ask yourself what "ping" would make worse (I can tell you what it would improve: Copying links would work again).

Mozilla doesn't do the right things because they get a lot of money from a company that wants tracking to be easier.

Arguably ping shouldn't have been implemented at all, because almost all sites that want that information don't want to let people opt out of it, so need to do it server side instead of client side anyway, leaving very few users of the feature.

I revived and unbroke them as part of DTrace for Linux (see e.g. https://oss.oracle.com/git/gitweb.cgi?p=dtrace-linux-kern...) because this argument only applies to process-directed waitpid() results: ptrace() is thread-directed, so if you kick off a new thread to handle the requests from other threads it cannot see any of the waitpid() results, and the thread that can see the waitpid() results isn't able to poll on the requests because it's too busy doing a waitpid() :( you'd have to busywait, or poll with pauses, and thus add huge latencies to signal handling in all your traced children: no thanks.

Though there are some other nasty areas of kernel API that might be worse than process management. Signals themselves, for example. https://ldpreload.com/blog/signalfd-is-useless

In practice, having it disabled does not improve anyone's privacy. Web sites still track link clicks; they just change the link to point to a URL on their own server which redirects to the actual destination. This is slower and more user-unfriendly than "ping" (e.g. it breaks "copy link URL to clipboard"). We rightly decry "security theatre"; disabling ping is privacy theatre.

In fact, enabling "ping" by default (but some browsers still allowing users to disable it) would probably be a net increase in privacy. If all major browsers enable ping by default, then some sites would stop implementing the redirect workaround, and the small set of users who manually disable the feature would get increased privacy.

This is a classic case of the Web being a complex ecosystem and users being unable or unwilling to understand the second-order effects of browser decisions. Unfortunately there are enough vocal critics of "ping" pressuring Firefox to do the wrong thing that Mozilla probably can't afford to do the right thing here.

Very complimentary assumption. Thanks to OpenBSD's tireless efforts in this respect, "security" is usually considered equivalent to "block known exploits". Hence "our software can be attacked with curl" => "block curl" is a completely natural reaction. After all, there's really no reason to spend a lot of time on this unless someone demonstrably comes up with an exploit which doesn't use curl.

This looks dubious. This hardcodes the decision that the cost of relocation multiplied by the probability that the node is going to be read later is smaller than a re-read of the entry from disk. I doubt that this is the place do make this sort of decision. Furthermore, this just looks semantically incorrect - if what is asked is relocation, than that is what should be done. Otherwise this could backfire in other possible use cases. For instance, suppose that for some reason the whole allocated space gets defragmented. Then, instead of the actual defragmentation, the whole dentry cache would be flushed, which is a completely different operation.

Process management has been the most fucked-up part of Unix since forever. Adding FD-based interface will go a long way towards unfucking it. Adding more ad-hoc kludges won't.

Our potential customers' traffic pattern was: read-mostly, with numerous clients trying to download the bulk content from the servers. I.e., imagine a hundred HTTP server threads, each performing a sequential download of a large file - but in time, their individual read requests get interleaved, leading to a pretty good approximation of random access to the LBA address space of the disk (or RAID volume).

BFQ seems to promise to solve something similar: apparently it tries to keep some "flow stats" per "control group" (?) and if the summary pattern for the group amounts to sequential access, it can do transaction combining to achieve performace benefits. Similar but not the same, I suspect.

The "parallel readers" pattern is probably not what Facebook are facing - by the description they have a more varied mix of different load patterns, and basically consider the access paterns to be mostly random.

I also tested a scenario with many parallel *writing* threads. Imagine a storage back end for surveilance video recording from many cameras. Here the Linux kernel can offer a configurable and huge dirty cache, suitable for write-combining during write-back caching. Especially promising with the "deadline" scheduler. But, as you increase the load, the write-back pending times start growing, and unfortunately the "deadline" scheduler employs a strict timeout, after which the pending request is scheduled for "immediate" writeback = into a FIFO queue with no more ordering :-( So the promise of elevator-based optimization (albeit geologically slow) collapses into a pure FIFO and the show basically stops.

In the days when I was playing with this, the cheap bulky SATA drives could do about 100-130 MBps (nowadays north of 200 MBps) but during the years, the truly random IOps remains at just about 75. That's 75 random seeks per second. And to squeeze "close to sequential" MBps performance out of a cheap disk drive, you would need to combine the traffic into transactions no smaller than say 4 MB (10 MB would be even better). Here is a graph based on my simple measurements:
http://support.fccps.cz/download/adv/frr/hdd/perf_vs_tsiz...

And also the elevator-based ordering only has a limited room to scale the IOps thoughput - possibly bounded by the disk drive electronics processing horsepower for IOps:
http://support.fccps.cz/download/adv/frr/hdd/IOps_vs_qdep...

Once we get into SSD's, at a first glance there's no point in trying to order and combine the transactions, as there's no mechanical arm with magnetic heads to be moved about, there's no track seeking inertia, and the elevator sorting algorithms and "stream tracking stats" are too expensive (in terms of latency / crunching horsepower) to be any use.
The basic "page" or "row" for NAND Flash reading / writing is about 4 kB, i.e. just about one memory page of the host CPU.
Still, beware of writing. The erase block size (a binary number on the order of megabytes) may again be a good clue.
Even sequential writing may hit a hidden boundary after a couple hundred megabytes (or maybe gigabytes in the bigger drives), long before the drive's payload capacity is fully overwritten - allegedly because modern drives first start using the MLC/TLC chips in "SLC mode" (which is faster) and only when that space runs out, they need to fall back to the native "symbol depth".
Random writing of "much less than an erase block per transaction" is generally not a good idea, because it requires more "flash janitoring" to take place (allocation of pages from different erase blocks, wear leveling must "shuffle pages out of the way" etc) and can really slow things down...

It's a heap of factors to consider, between the application-level abstraction of a file (or database, or whatever) and the physical sectors on spinning rust or pages in NAND Flash chips. I've already hinted at the fun available in striped RAID, and also note that even sequential files need to store metadata, which may add some "out of sequence" seeks as well. Modulo barrier operations. Admittedly difficult to optimize for at the block level or just slightly above, while the apps not necessarily even care to use "hinting" about their future intentions ( fadvise() ) which is readily available from the kernel. Does the httpd even have some useful clues about the traffic pattern it is serving? Well at least it could know the size of a file being read from disk...

IMO it's really the user space apps that should be aware of their respective buffering needs, and should do the optimal stream buffering internally, keeping a suitably large buffer in the user space. Makes me wonder if mmapping the source file of a stream (rather than an explicit read() = copy into a user-space buffer) would give the kernel more opportunities for merging multiple parallel sessions, that are trying to "stream" the same popular file down a myriad independent TCP sessions running at different transfer rates... Not sure, maybe someone has already written an optimized "file-serving back-end httpd" during the decade that has passed, I'm not keeping an eye on it. Not sure though, people seem to be focusing more on building clouds of Java VM's, using JavaScript on the server side etc... and the most popular HTTP daemons apparently still have buckets or buffers in kilobyte sizes. And the "file download services" are tackling the issue using automatic Flash-SSD-based cache that caters for the popular downloads, so that the spinning rust does not need to be accessed so intensively.

This debate about storage IO bandwidth reservations feels like the "kernel and sysadmin team" fighting the "user-space apps team". Sounds odd to me. There should be an architect who would know better than play chess with himself on both sides of the checkerboard (on each side frantically trying to beat the opponent).

Looks like there is only x86_64 aka. AMD64 for now.

1) mentioning India in that context is not _racist_ (it might carry some presumptions about countries or whatever, but nothing about races per se)

2) there are indeed _some_ (not all, not even majority, nothing like that) barely competent just-out-of-college IT wokers in India.

As such, I fail to see a problem with the original post by mebrown (https://lwn.net/Articles/784998/).

Sadly, neither Red Hat nor SUSE are helping to bring KDE into the Wayland world. The only reason GNOME has been able to make it is because Red Hat has developers pushing it forward for their commercial Linux desktops.

Even SUSE, who is a sponsor of KDE, does basically nothing with the desktop. :(

SUSE even ships GNOME with Wayland by default in SUSE Linux Enterprise 15.

We also should all know that a lot of the most competent people in the industry come from India. The statement which was objected to is easily understood as making an assumption that only the less qualified group exists. That, I believe, is not fair.