The Clever Audio Plugin

Our introduction to Linux audio and MIDI plugin APIs ended with a mention of the Clever Audio Plugin (CLAP) but did not get into the details. CLAP is an MIT-licensed API for developing audio and MIDI plugins that, its developers feel, has the potential to improve the audio-software situation on Linux. The time has now come to get to those details and look at the state of CLAP and where it is headed.

When CLAP resurfaced in late 2021 after years of radio silence, xkcd #927 references were a popular response in all discussions about it. But there are a number of serious questions to ask about this API as well. Does CLAP actually compete with the other audio APIs available on Linux, including VST3, LV2, and others? Is it a viable alternative? Does it address problems that developers have with other APIs?

The backstory

Alexandre Bique started working on CLAP in 2014. In a nutshell, he wasn't happy with the industry dominance of Steinberg's VST API, but he also wasn't happy with some LV2 design decisions. In 2015, Bique was hired by Bitwig as a software engineer, where he's been working ever since. That's pretty much when the project's activity ground to a halt for the first time. Bique then resumed development in early 2016, then stopped again in October 2016.

When Steinberg started aggressively retiring VST2, Urs Heckmann, the creator of the proprietary u-he synths, contacted Bique and told him that he liked the CLAP API's simplicity. He also asked if Bique wanted to finish it, to which Bique said "yes". So in April 2021, Bique resumed work on CLAP, and has been tirelessly hacking on it ever since. There have been multiple releases of the API in the last several months, for the first time since 2015.

Right now, v0.24.1 of the CLAP SDK is available (tagged in Git). The specification is close to being final. It might undergo some further revision, but Bique is mostly working on code examples now and expects to ship v1.0 soon.

What's different about CLAP

The reasoning behind the creation of CLAP is multifaceted, being at the same time ethical, legal, and technical. In a post at KVR, Heckmann claims that the main reason for CLAP's existence is its liberal licensing. He also mentions a strong governance motive:

As plug-in developers we always feel like our products are second-class citizens in the DAW [digital audio workstation] ecosystem, as if the plug-in standard forms a harness of what we can do and what we can't do. As someone expresses it so nicely, "as if the host puts its fingers into the plug-in and directs it". CLAP feels a lot more on equal level, which is already expressed by having a host object and a plug-in object. The host isn't a shapeless god entity, it's just another struct that we communicate with. [...]

Current formats are maintained by host developers with a conservative product philosophy. A very short conclusion here is that some of the people who support CLAP want to switch from the back seat to the steering wheel of what a host/plug-in environment could make possible. And I'm absolutely certain that the hosts and plug-ins which do the switch - not just adoption, but also implementation - will gain market share.

Other reasons listed by the CLAP team are technical: fast plugin scanning for hosts that need to update the list of available plugins, controlled multi-threading, metadata for plugin categorization, etc. All these features are also available in LV2, though.

CLAP also takes into consideration some criticism that LV2 received from developers over the years, and thus it has neither RDF 1.1 Turtle metadata (too lengthy to write by hand, requires build system enhancement when automated), nor versioned extensions (dealing with those is cumbersome). There are even more LV2-specific issues that CLAP does not appear to have, like heavy, under-documented APIs, design limitations that even the LV2 maintainer thinks call for a partial API deprecation, and more.

It's still difficult to make a case for or against CLAP as compared to LV2, because people arrive at new APIs with preconceptions and older APIs' baggage. I've definitely seen developers praising the ease of creating a CLAP plugin from scratch. And indeed, a single header file required to get started sounds like a good thing. Over to Heckmann:

Several JUCE based open source synthesizer plug-ins have been ported to CLAP, almost literally overnight (I have to check chat protocols, but I do think I went to bed with one ported and woke up to yet another ported).

On the other hand, Ardour's simplistic, built-in, LV2 reverb plugin is 462 lines of C code and maybe a hundred lines of Turtle metadata (that you can generate from the source). Which makes it nearly an overnight hack, if you know what you are doing.

One developer I talked to, Vadim Sadovnikov (an LSP developer), summarized CLAP as a "modern API that is nevertheless as easy to get started with as was/is VST2". He summarized some of the benefits of CLAP: its pure C interface, simple header files that you can easily drop into your project and start using, the ability to package multiple plugins into a single library file, and UTF-8 in everything.

Here is William Light's (of LHI audio) take on the subject, expressed in a private conversation:

VST3 is really the only cross-platform widely-supported option, and there remain outstanding technical issues with MIDI (particularly surrounding MIDI Polyphonic Expression). Steinberg has also shown themselves to be legally heavy-handed in a way that has made the plugin developer community uneasy. In my opinion, CLAP does fill a major need here – namely, a plugin API owned and managed by the community rather than a single company.

From a purely technical perspective, I find CLAP to be very thoughtfully architected. It has its idiosyncrasies, but so do all of the other plugin APIs, and, in my opinion, CLAP has fewer oddities than the other APIs, and they're less onerous.

It's important to note that Bique is strongly against this kind of comparison, though. In a private conversation, he stated:

I want CLAP to stand on its own. I don't claim that CLAP is better than VST3, AU, LV2 or anything else. I don't want to go in the direction of CLAP vs XY because I think it is toxic.

Concerns

Navigating discussions on this API and talking to involved parties feels a lot like walking on thin ice. There are three major groups of people I have met: developers who think that CLAP is the next big thing, those who think it's a case of a not-invented-here syndrome, and developers who are in the wait-and-see camp. All of them are extremely outspoken.

Having said that, I have to add that, surprisingly, there isn't much analysis of CLAP available yet. A commonly used technical point I've seen across a few discussions is that CLAP doesn't allow for digital signal processing (DSP) options separated from the user interface. But there is not much agreement between developers about the need for that either.

An example of where this is useful right now are MOD multi-effect boxes: the DSP code is executed inside the box, and while you have physical knobs and encoders to change parameters, you get vastly more control over the effects graph in the web-based editor that runs in a browser on your laptop. For the vast majority of plugin users, though, it's more like an edge use case. Besides, should MOD units come to rely on CLAP rather than LV2, the use case could be covered by adding an extension to the specification.

In most cases, DSP/UI separation is a significant additional complication for both plugin and host developers. One developer I spoke to views the omission of DSP/UI separation in CLAP as a welcome simplification rather than a mistaken omission.

Another concern I've come across is that we are going to end up with Bitwig and u-he as dictators shaping CLAP as they please. However people behind CLAP hint about devising a plan to introduce some sort of a governance body that would make it impossible for one or two companies to have full control over the evolution of the API. Again, we'll have to wait and see what happens there.

But the biggest objection seems to be that the time spent on CLAP could be used to iron out bugs and problematic parts of LV2.

How to see CLAP in action

Today, if you want to see a real CLAP plugin working in a real host application like a digital audio workstation, you'll have to mix free and proprietary software.

Bitwig is currently the only digital audio workstation that has CLAP support in a released version, and it's proprietary. The feature is also hidden from users by default. This will change in the future; once the CLAP v1.0 release is available, Bitwig developers will enable its support by default. There's also some evidence that there is initial, currently unreleased support for CLAP plugins in Reaper, another proprietary digital audio workstation.

On the plugin side, u-he's MFM2 synth (proprietary) is available in CLAP as well but probably won't be be publicly updated until CLAP 1.0 is out. As MFM2 and Bitwig rely on different versions of the CLAP API, running the CLAP version of the synth in Bitwig won't work right now.

That is, however, not a problem with free plugins. The Surge family of GPLv3 plugins has preliminary CLAP support: these include Surge XT synth (and the effects stack), Monique synth, and Shortcircuit XT sampler. Two GPLv3 plugins from Jatin Chowdhury (Analog Tape Model and Build-Your-Own-Distortion) also have initial CLAP support. Several more plugins are available as part of the Robbert van der Helm's NIH-plug framework.

Potential for further adoption

There are several prerequisites for CLAP to become mainstream. In a world where tens of thousands of plugins and only a few dozens of mainstream host applications coexist, the success of CLAP largely depends on how many host applications will support it. So far, it has been mostly developers of a few proprietary digital audio workstations who are actively interested in adding CLAP support.

Will free hosts support CLAP? So far, developers are noncommittal on the topic.

Paul Davis has reservations about the need for CLAP but he also doesn't mind support for it being added to Ardour. Robin Gareus, another Ardour developer, has a good relationship with Bique and, in fact, got him to add optional support for zero-copy processing to the API. So there is no reason why the Ardour team would take a stance against CLAP even though they are heavily invested in LV2 (having participated in core LV2 spec development and created several extensions).

In a private conversation, Alexandros Theodotou didn't object to having support for CLAP in Zrythm in the future either, but he thinks it's too early to bother working on that. Rui Nuno Capela, of Qtractor fame, is familiar with the latest news on CLAP but "has no plans to pursuit yet another plugin API at all" (from private conversation). One of the Meadowlark DAW developers expressed interest in adding CLAP support and actually added some preliminary code to the project's engine. That said, this program is at a very early development stage. It's going to take a while for anybody to benefit from either CLAP or LV2 support in it.

In a nutshell, free host developers are likely to accept patches adding CLAP support, but less likely to work on CLAP themselves. The amount of work involved will vary from one host to another; most hosts out there support multiple plugin APIs, so they have an abstraction layer, and thus adding support for another API is not too much of a burden. But CLAP supports advanced features like polyphonic modulation in synths; if you want these features, that might mean writing entirely new code that touches more parts of the host's code than plugin-related code.

One other way to help CLAP adoption is to wrap CLAP plugins for existing APIs so that plugin developers do not have to care about hosts' support for CLAP. The host application would see something like, for example, a VST3 plugin, but there would be a CLAP plugin inside. There is a new project called claptrap that aims to give developers convenient tools to do just that.

The other prerequisite is support for CLAP in frameworks, like JUCE (GPLv3 + proprietary) and iPlug2 (open source, custom license), where you write code once and generate binary plugins for multiple plugin APIs. Here is a rather sensible observation by user "audiojunkie" on KVR:

Developers want to write once and be able to compile for all architectures, OSes, etc. JUCE is the de-facto standard and overall king in this realm. Without JUCE buy-in, a new format will just muddy up the waters for development and be another thing that users will think they want developers to support, and developers will hate it for that reason. If this format doesn't solve more than the Steinberg licensing problem, it's not likely to help anyone.

Currently, this is solved by CLAP's own JUCE6 extensions. There is, however, the host end of the issue because multiple sequencers and DAWs use JUCE for their user interface. Kjetil Matheussen, for example, refuses to add support for APIs to Radium if these APIs are unsupported by JUCE.

On the iPlug2 side, there is a somewhat active "clap" branch. There is also a new framework targeted at Rust developers specifically, NIH-plug, with support for VST3 and CLAP as API targets.

Overall, the situation is not hopeless at all. On the other hand, despite the project's age, these are still the early days and we are looking at years of development ahead.

How to get started

The project's home page now lists most relevant links for developers: the SDK, example host and plugin implementations, various bindings and helper projects, etc. There is neither a mailing list nor a forum for discussing CLAP yet; the conversation is currently scattered across multiple communication channels. The Surge team has a "clap-chatter" chat on its Discord server and, as far as I can tell from lurking there for the past few months, it seems welcoming to non-Surge developers.

In conclusion

CLAP is a case of a surprising rebirth fostered mainly by developers of proprietary software. The main reason it's on anyone's radar is because it's backed by u-he and, apparently, has support from Bitwig and Reaper — all of these companies and software projects are well known in the overall music industry. So that gets people's attention.

From the technical standpoint, there is a huge overlap between VST3, LV2, and CLAP. However, with regard to VST3, there are also ethical and business concerns at play. In the case of LV2, the disagreement appears to boil down to a difficult learning curve on the LV2 side, as well as on how it is run as a project, how gatekeepers respond to concerns of plugin developers, etc.

Watching CLAP unfold is going to be interesting or, in the worst case scenario, educational. It's not a particularly good health symptom for the industry that, almost 20 years since the beginning of the work on the GMPI plugin, developers still want to displace Steinberg with a truly open-source, community-driven plugin API. LV2 was not the success that had been hoped for, in this regard at least. Whether CLAP will fare better remains to be seen.