Emacs 29.1 released

Posted Jul 31, 2023 18:03 UTC (Mon) by atai (subscriber, #10977)
In reply to: Emacs 29.1 released by sahas
Parent article: Emacs 29.1 released

guess you don't run any other app but emacs

Emacs 29.1 released

Posted Jul 31, 2023 18:16 UTC (Mon) by jem (subscriber, #24231) [Link] (4 responses)

>guess you don't run any other app but emacs

Or Firefox, Chromium, Blender, Inkscape, Libreoffice, FreeCAD, the whole suite of KDE software. Just to name a few.

Emacs 29.1 released

Posted Jul 31, 2023 21:49 UTC (Mon) by jrtc27 (subscriber, #107748) [Link] (2 responses)

Firefox has run under Wayland for several years now, and KDE works too. Not perfectly, but good enough for day-to-day use. I have a system where that's the only configuration KDE can be run in, even.

Emacs 29.1 released

Posted Aug 1, 2023 0:36 UTC (Tue) by rsidd (subscriber, #2582) [Link] (1 responses)

I think OP meant, all of the above work without xwayland. Personally I see no harm in having xwayland around.

Emacs 29.1 released

Posted Aug 1, 2023 6:22 UTC (Tue) by jrtc27 (subscriber, #107748) [Link]

That would make more sense

Emacs 29.1 released

Posted Aug 1, 2023 5:59 UTC (Tue) by ibukanov (subscriber, #3942) [Link]

Chromium and many apps based on it like VSCode runs Ok under pure Wayland if one passes the flag --ozone-platform-hint=auto, see https://wiki.archlinux.org/title/chromium.

Emacs 29.1 released

Posted Jul 31, 2023 18:16 UTC (Mon) by corbet (editor, #1) [Link] (23 responses)

I run a fairly wide variety of applications on my machine, and none of them are using the X11 protocol at this point. From my point of view, at least, that transition has been done for a while.

Emacs 29.1 released

Posted Jul 31, 2023 18:49 UTC (Mon) by lunaryorn (subscriber, #111088) [Link] (22 responses)

As you say, it's your point of view… it quite depends on what applications you use.

Electron applications still require some non-default flags to run on Wayland, proprietary Qt applications often still don't ship Qt's wayland library in their bundled copy of Qt, and Java still doesn't support Wayland at all. From my point of view we still have a long way to go for this transition.

Say what you want about Electron, but Jetbrains' IDEs arguably keep at lot more people stuck on xwayland than Emacs used to do ;)

Emacs 29.1 released

Posted Jul 31, 2023 20:34 UTC (Mon) by NYKevin (subscriber, #129325) [Link] (21 responses)

Another example is that quite a few terminal emulators do not have OSC-52 support, which means (in practice) that:

1. If you are the author or maintainer of some sort of terminal app...
2. ...that wants to copy or paste things to the system clipboard...
3. ...and have it work over SSH...
4. ...and not have to tell your users "sorry, it's broken in gnome-terminal"...
5. ...then the only way to do that is via X11 forwarding...
6. ...which (as far as I'm aware) is not supported by Wayland.

The gnome-terminal bug is from 2018 and discussion consists almost entirely of security bikeshedding, despite the fact that multiple mature FOSS implementations exist in the form of other terminal emulators: https://gitlab.gnome.org/GNOME/vte/-/issues/2495

A list of bugs in other emulators can be found here (no idea if this is up-to-date, just found it on Google): https://github.com/ojroques/vim-oscyank#vim-oscyank

Emacs 29.1 released

Posted Jul 31, 2023 22:57 UTC (Mon) by Sesse (subscriber, #53779) [Link] (10 responses)

I don't think I understand. Could you give an example of such an app where this is relevant? Not supporting forwarding of graphical apps over SSH is inconvenient, but I don't understand what it has to do with cut and paste. Not the least since the normal way of cutting and pasting in a terminal is through selecting text, not OSC-52.

Emacs 29.1 released

Posted Aug 1, 2023 0:05 UTC (Tue) by NYKevin (subscriber, #129325) [Link] (8 responses)

Ironically for a story about Emacs, Vim is the most obvious example. When you yank into the + and * registers, it copies to the system clipboard. It can also paste from them. See https://vimhelp.org/gui_x11.txt.html#quoteplus for documentation. This has been supported for decades, even on platforms that do not use X11 (such as Windows), and there is no adequately explained reason why it should suddenly break just because we're all moving to Wayland.

Proper support for this is necessary, because both Vim and (e.g.) Tmux can be used to split the screen into smaller windows or panes (respectively) and so if you try to copy raw characters with your mouse, you'll get a lot of nonsense whitespace that shouldn't be there (at least, for longer selections, anyway). In order to fix that, you either need Vim (or Tmux etc.) to have native clipboard support, or you need it to somehow tell the terminal emulator exactly which characters should be considered for mouse selection (which I don't believe is covered by any well-known standard, but I could be mistaken).

Emacs 29.1 released

Posted Aug 1, 2023 8:44 UTC (Tue) by Sesse (subscriber, #53779) [Link] (7 responses)

OK, so the point is that terminals support this when running under X11, but not the equivalent protocol under Wayland? Or that there exists a semi-reasonable workaround (xclip with X forwarding) for X11 that does not exist in Wayland?

(FWIW, I use vim as my only editor, but I never use "+ or "*. Obviously your workflow is different.)

Emacs 29.1 released

Posted Aug 1, 2023 15:22 UTC (Tue) by NYKevin (subscriber, #129325) [Link] (5 responses)

I don't know that Wayland *has* an equivalent protocol (to X11 forwarding). If it does, I've never seen any SSH flags etc. for it.

OSC-52 is a generic terminal/serial protocol that can be implemented by anybody on any kind of windowing system, but it just handles clipboard access (whereas X11 forwarding is a far more complex beast that lets you run a whole X client over SSH). The extremely simple version is, the program outputs some control characters, which describe either a copy or a paste, and the terminal emulator is responsible for doing the copy/paste operation as requested, so the program doesn't need to know anything about $DISPLAY etc. in order to access the system clipboard.

Most of the gnome-terminal bikeshedding is of the "but what if somebody has something sensitive in their clipboard?" variety. The standard solution to this problem appears to be write-only access, i.e. you implement copy but not paste (or have paste just output an empty string), so that at worst a remote host can clobber your clipboard, but it can't read secrets from your clipboard.

Emacs 29.1 released

Posted Aug 1, 2023 15:56 UTC (Tue) by Sesse (subscriber, #53779) [Link] (4 responses)

I wasn't asking about a gnome-terminal rant, I was trying to figure out whether my understanding of your use case was correct. :-) Because if the point really only is to have a workaround for copying clipboard contents from terminal applications over SSH, that would seem rather marginal for a windowing system's functions.

Now, it sure would be nice to have some sort of network transparency for graphics even as we phase out X11. But that feels like a different issue to me.

Emacs 29.1 released

Posted Aug 1, 2023 18:35 UTC (Tue) by NYKevin (subscriber, #129325) [Link] (2 responses)

> Because if the point really only is to have a workaround for copying clipboard contents from terminal applications over SSH, that would seem rather marginal for a windowing system's functions.

I agree. That's why I wrote a "rant" about gnome-terminal: This should be the terminal emulator's problem, not the windowing system's problem.

Emacs 29.1 released

Posted Aug 2, 2023 16:44 UTC (Wed) by Per_Bothner (subscriber, #7375) [Link] (1 responses)

"This should be the terminal emulator's problem, not the windowing system's problem."

Right - but unfortunately there is no simple portable API for accessioning the clipboard. Even for JavaScript-based terminal emulators (such as DomTerm and xterm.js) there are security policies that limit access. The simplest is to invoke a helper application - but these aren't portable either. If curious you can look at the code for get_clipboard_command in lws-term/utils.cc in the DomTerm sources.

Emacs 29.1 released

Posted Aug 3, 2023 17:27 UTC (Thu) by farnz (subscriber, #17727) [Link]

In this context, OSC 52 is that API - it's an API between an application and the terminal emulator, allowing the application running in the terminal to ask the terminal emulator to perform clipboard accesses for it.

The problem is that VTE, one of the common implementations of a terminal emulator (used by GNOME Terminal, among others), deliberately doesn't support this API. As a result, instead of terminal applications like vim using OSC 52, they use a remote X11 connection to access the clipboard using X11 APIs, while rendering using terminal APIs like OSC. This is a hack around the lack of OSC 52 support in common terminal emulators.

Emacs 29.1 released

Posted Sep 18, 2023 6:55 UTC (Mon) by daenzer (subscriber, #7050) [Link]

> Now, it sure would be nice to have some sort of network transparency for graphics even as we phase out X11.

FWIW, there's Waypipe, which works more or less like X11 over SSH from a user PoV: https://gitlab.freedesktop.org/mstoeckl/waypipe

> But that feels like a different issue to me.

Yeah, since this discussion is about a feature of the terminal emulator, which should normally run locally in the user session, I'm not sure how SSH tunnelling of the display protocol is relevant.

Emacs 29.1 released

Posted Aug 1, 2023 16:01 UTC (Tue) by intelfx (subscriber, #130118) [Link]

> Or that there exists a semi-reasonable workaround (xclip with X forwarding) for X11 that does not exist in Wayland?

This, yes.

The problem: users want terminal applications to interact with the system clipboard. By extension, users also want _remote_ terminal applications to interact with the _local_ system clipboard (think Vim over SSH).

The solution: an escape sequence protocol (OSC-52) was invented to do exactly that, and it trivially works over any nested configuration of remote shells (application emits an escape sequence -> remote shells relay that sequence -> user's terminal emulator parses the escape sequence and acts on it).

HOWEVER, VTE developers think that OSC 52 is a security threat and refuse to implement it. The closest possible workaround is for the terminal applications to "piggyback" on the X11 clipboard directly, because everyone who cares about this feature is ultimately sitting in an X11 environment. This workaround is "local" in nature, but it can be extended to remote sessions by way of SSH X11 forwarding.

The problem #2 is that Wayland has no equivalent of X11 forwarding.

Emacs 29.1 released

Posted Aug 1, 2023 6:08 UTC (Tue) by ibukanov (subscriber, #3942) [Link]

Given the broken support for OSC-52 many terminal editors calls tools like xclip to access the system clipboard. But then to make it work over ssh one has to use ssh -Y to forward X11 over ssh. That does not work in pure wayland world.

Emacs 29.1 released

Posted Aug 1, 2023 1:12 UTC (Tue) by flussence (guest, #85566) [Link] (7 responses)

My experience with the vim-ssh-X11 software tower is that copy and paste has only ever worked accidentally, and I've gotten used to the habit of doing a raw terminal text-select gesture and fixing it up by hand (or dropping to `less`).

If it's not *supposed* to suck then that's news to me…

Emacs 29.1 released

Posted Aug 1, 2023 1:20 UTC (Tue) by willy (subscriber, #9762) [Link] (2 responses)

I'm certain that sucking is by design. At some point recently Debian made it so I had to hold down shift while selecting to get the behaviour I'd become used to over the past twenty years of using vim. Just one more turd on the pile of shit.

Emacs 29.1 released

Posted Aug 1, 2023 9:31 UTC (Tue) by Sesse (subscriber, #53779) [Link] (1 responses)

To be fair, that was upstream, not Debian. But I still have “set mouse=” in my .vimrc, indeed.

Emacs 29.1 released

Posted Aug 2, 2023 1:15 UTC (Wed) by salewski (subscriber, #121521) [Link]

> To be fair, that was upstream, not Debian.

The upstream issue with a long discussion:

"Defaulting mouse=a in a terminal is incorrect (new as of 7.4.2111)"
https://github.com/vim/vim/issues/2841

> But I still have “set mouse=” in my .vimrc, indeed.

Same here.

Emacs 29.1 released

Posted Aug 1, 2023 1:50 UTC (Tue) by NYKevin (subscriber, #129325) [Link] (3 responses)

It seems to work reliably for me as long as I do not exit vim before pasting (and as long as I'm content to wait a couple of seconds after pasting).

This is because of a rather unfortunate design decision on the part of X (basically, your clipboard is not a buffer, it is merely a reference to the X client that initiated the copy, so if that X client exits, then the clipboard is a dangling reference to nothing), which can cause problems even for local X clients (unless your DE takes responsibility for the clipboard data, which I know some DEs do). It would be fixed if vim used OSC-52, because then the X client responsible for the copy would be the terminal emulator, and not vim. It would also be fixed if we could somehow establish a protocol for vim to tell ssh to do the copy locally, but that would be rather silly since OSC-52 is a more general solution anyway.

X selection

Posted Aug 2, 2023 12:47 UTC (Wed) by jch (guest, #51929) [Link] (1 responses)

> This is because of a rather unfortunate design decision on the part of X (basically, your clipboard is not a buffer, it is merely a reference to the X client that initiated the copy,

There's a good reason for that: the X11 selection mechanism allows negotiating the format of a selection transfer: for example, a word processor can offer the selection as RTF, HTML and plain text, and the requestor can choose the format that it prefers. This allows evolving the protocol without breaking compatibility with existing applications, for example when we implemented Unicode copy-paste back in the late 20th century:

https://www.irif.fr/~jch/software/UTF8_STRING/UTF8_STRING...

If the selection were stored in the server, the application would need to transfer it in all supported formats every time the user selects something, possibly over a slow network connection.

> so if that X client exits, then the clipboard is a dangling reference to nothing)

No, if the selection owner closes the connection to the X server, the selection owner is set to NULL.

X selection

Posted Aug 3, 2023 19:04 UTC (Thu) by farnz (subscriber, #17727) [Link]

It's interesting to compare the (older) X11 mechanism to the Win32 clipboard mechanism in this regard.

In the X11 mechanism, the server provides the ability to own one or more selections; both cut and copy are "just" claiming ownership of a selection. Paste requires you to use the TARGETS atom to list possible formats, then choose one, then request a conversion to your preferred format; as an optimization, if you only support one possible format, you can ask for data in that format, and simply fail if the owner can't supply it. Once the data is converted to the right format by the owner, you can transfer it through the server.

Win32 is different in some significant ways:

Only one clipboard, rather than multiple selections.
Clipboard data formats are either "standard clipboard formats" (defined by the API), or "registered clipboard formats" (registered by name, multiple applications can register the same format.
There are built-in conversions between certain standard clipboard formats.
The default behaviour is to eagerly transmit the data to the server on cut/copy (via OpenClipboard/SetClipboardData/CloseClipboard), but an application can choose to merely indicate that it can supply the data in this format, and delay supplying it until it's requested.
You can send data to the server in multiple formats, which are expected to be different representations of the same data. Some of these can be delayed, others eager - for example, a word processor could eagerly supply the data as RTF (since that's what it has to hand), and offer delayed conversions to HTML and plain text. Or LibreOffice could eagerly supply in OpenDocument, with delayed conversions to RTF, HTML and plain text, but only where the data is small.
When the client requests the data, it asks the server to enumerate formats in preference order for the source. When it finds a format it's happy with, it asks for the data in that format. If it's a delayed offering, the server does IPC with the owner to get the conversion done, and then returns the data to the client as-if it were an eager offering.

It's worth noting that both mechanisms allow negotiating the format of a transfer; the primary difference is that the Win32 mechanism puts the data ownership in the server, and gives the owner no knowledge of when a paste happens (since it's implementation-defined when a delayed representation is requested by the server).

Emacs 29.1 released

Posted Aug 8, 2023 1:45 UTC (Tue) by repetitivestrain (guest, #165872) [Link]

(Speaking with my Emacs-on-X11-and-Wayland maintainer hat on.)

Revolving around selection ``ownership'' is not a bad design decision, it's a feature of the X inter-client communications convention. There are many different selections, most of which do not justify uploading the selection data to an X server or another form of specialized selection data storage.

For the selections that do warrant being saved after their owner windows are destroyed (CLIPBOARD is the only selection that comes to mind), users have the option of running a CLIPBOARD-managing client, which proactively monitors for changes in selection ownership, converts them to a number of specific targets and subsequently obtains ownership over them, or employing a clipboard manager, a client selection ownership and data will be transferred to when the original owner of the CLIPBOARD selection terminates.

The disadvantage of running a CLIPBOARD managing client is that you limit the flexibility of the protocol itself: for example, if your CLIPBOARD managing client only requests the STRING and COMPOUND_TEXT targets from your selection, then future attempts by other clients to insert the contents of CLIPBOARD won't be capable of receiving UTF-8 or file names from that selection; this is also compounded by how relegating the task of replying to selection requests to a separate client diminishes the ability of selection owners to provide feedback in response to selection requests (for example, there are several Emacs packages that provide visual feedback ranging from deactivating the region to beeping whenever another client attempts to insert the contents of the clipboard.)

Whereas with a clipboard manager, that flexibility is only lost once your client terminates. Lamentably, support for the clipboard manager protocol is rather lackluster, which is its sole disadvantage. I presume Vim doesn't support it; why not use Emacs instead?

Many lessons learnt during the long development of the X Window System appear to have been lost on the Wayland developers, but the selection transfer mechanism is not one of them. Wayland defines a single clipboard selection, but as with X, selection data is not uploaded to the display server. Instead, a token dubbed a data source is provided, and subsequent requests for selection data by other clients supply a FIFO which the creator of the data source is expected to write the requested data to.

In their inexorable quest to pastiche Microsoft Windows, GNOME's display server requests and saves the selection data itself every time a new data source is set, thereby serving as the Wayland analogy of a CLIPBOARD-managing client, making life difficult for the aforementioned Emacs packages. But I have to concede that the lack of a clipboard manager-esque protocol is a glaring omission in Wayland, and GNOME's manifesting the only means to provide users with behavior that they are already accustomed to.

Emacs 29.1 released

Posted Aug 1, 2023 13:10 UTC (Tue) by daenzer (subscriber, #7050) [Link] (1 responses)

> 5. ...then the only way to do that is via X11 forwarding...
> 6. ...which (as far as I'm aware) is not supported by Wayland.

I suspect I might be missing some context here, but FWIW, SSH X11 forwarding works in a Wayland session via Xwayland, same as with any other X server.

Emacs 29.1 released

Posted Aug 1, 2023 15:12 UTC (Tue) by Wol (subscriber, #4433) [Link]

If you'd bothered to read the thread ...

The context you are missing is that XWayland IS NOT THERE.

Cheers,
Wol