Weekly Edition
Return to the Kernel page
| |||||||||||||
The first stable OpenVZ release
The OpenVZ project has announced its existence and its first stable
release. OpenVZ is yet another virtualization approach for Linux, based on
SWsoft's "Virtuozzo" product. The OpenVZ approach differs from others,
however, in that it creates its virtualized environments within a single
kernel; the result, it is claimed, is better performance. Unfortunately,
the released patch is for the ancient 2.6.8 kernel.
(Log in to post comments)
The first stable OpenVZ release Posted Dec 5, 2005 20:59 UTC (Mon) by kolyshkin (subscriber, #34342) [Link] Let me explain LWN readers why openvz patch is against kernel 2.6.8. OpenVZ kernel introduces another abstraction layer, which means a lot of modifications here and there all across the kernel code. It is quite a complex thing. One of our major goals is stability for end users; indeed, if you host one system, your kernel should be stable, but if you host hundred of such systems, which is the case for OpenVZ, your kernel should be rock solid. To improve stability, we do not port OpenVZ kernel stuff to each and every vanilla kernel release, instead improving the version we have. What about mainstream kernel bugfixes? We cherry-pick it from newer kernels? What about recent security fixes? We cherry-pick them. What about support for newer hardware? We do update the drivers in our OpenVZ kernel. So, instead of taking a newer kernel which have both good and (hopefully not too much of, but who knows) bad stuff, we stick to that ancient kernel and backporting that good stuff. Also, porting to a newer mainstream kernels needs some significant time and effort, that we'd better spend improving OpenVZ kernel. More to say, major Linux distro vendors does the very same thing for their kernel. Look at what is the current kernel for RHEL4? It is 2.6.9. But it is quite far from vanilla 2.6.9 - they do also backport mainstream bug fixes, security fixes and merge in newer drivers (aside from a lot of Red Hat own fixes). Will it stay what way for OpenVZ kernel? Surely not - at some point we will rebase to some more recent 2.6 kernel - at that time this might be 2.6.20 or something like that, we don't really know now. Finally, if you feel there is something that OpenVZ kernel is missing but should definitely have, be it a security fix or something else - file a bug, preferably with a patch attached. We are open. :)
The first stable OpenVZ release Posted Dec 5, 2005 21:30 UTC (Mon) by bk (guest, #25617) [Link] That sounds like a maintainance nightmare. Are you planning on submitting OpenVZ for inclusion in the kernel tree? That seems like the only viable long-term solution.
The first stable OpenVZ release Posted Dec 6, 2005 1:10 UTC (Tue) by kolyshkin (subscriber, #34342) [Link] > That sounds like a maintainance nightmare.It is actually not as awfully complex as it looks, and really improves stability. > Are you planning on submitting OpenVZ for inclusion in the kernel tree? That seems like the only viable long-term solution.Yes we do, it's quite natural, we plan to merge our code mainstream, and even have this ambitious goal in our roadmap. Sure this will be a long process, done in many small steps...nobody expects it to happen overnight. :)
Linux VServer Project has done this for years Posted Dec 5, 2005 21:54 UTC (Mon) by NZheretic (guest, #409) [Link] From WikipediaLinux-VServer is a virtual private server implementation for the Linux kernel. It is now maintained by Herbert Pötzl in Austria and is not related to the Linux Virtual Server project, which implements network load balancing.
Linux VServer Project has done this for years Posted Dec 6, 2005 1:18 UTC (Tue) by kolyshkin (subscriber, #34342) [Link] Comparing to Linux-VServer, I could say OpenVZ has a better resource management mechanisms (such as User Beancounters, or UBC for short) and full network stack virtualization (called venet in OpenVZ and ngnet in VServer) - the thing which is still not ready in VServer.
Linux VServer Project has done this for years Posted Dec 6, 2005 5:13 UTC (Tue) by micah (subscriber, #20908) [Link] Can you provide more details about how OpenVZ is different from the Linux-Vserver project? The FAQ entry on the site ("How is OpenVZ different from other technologies?") only equates them as both OS virtualization techniques. You say User Beancounters is one difference, but what is that?
Linux VServer Project has done this for years Posted Dec 6, 2005 6:05 UTC (Tue) by kolyshkin (subscriber, #34342) [Link] UBC is an OpenVZ kernel component which implements VPS' resources accounting and limiting. There is a set of different resources (about 20 of them) for each VPS, and you can set their barrier and limit (can think of it as a soft and hard limit), and see their current usage.
The thing is there are some resources (like kernel memory, or open file descriptors) which are not limited in any other way, so any single VPS can abuse such a resource thus rendering the whole system (which can host hundreds of such VPSs) unusable.
So UBC is used to prevent such abuses (by different means - from returning ENOMEM from a syscall to killing the process that abuses resources), this is an important part of isolation. But this is not its only use.
The other UBC use is to provide guarantees for VPSs.
PS OpenVZ VPS == VServer guest
Linux VServer Project is doing just fine now Posted Dec 6, 2005 7:26 UTC (Tue) by gvy (guest, #11981) [Link] Hm, Kirill, did you compare OVZ and VS1.x or 2.x?
From what I've heard, 2.x is quite capable regarding resource management and provisioning, seems like it quite stacks against Solaris 10 containers.
And it's quite actively forward-ported, I see 2.6.12.4-vs2.0 patch on their site right now. It's critical given such a security bug-ridden kernel as 2.6.x presently is, AFAIR there were at least one local kernel since 2.6.8 (maybe I'm still sleepy though).
Maybe it's worth bothering SWsoft's mgmt regarding some kind of "merger" proposal? I somehow can't deny the thought that the whole OpenVZ thing is some kind of move on their part (perhaps motivated by you, goldhead and friends?) feeling that proprietary technology just loses its relevance here, facing VServer, Xen, and OTOH VMware and maybe even to some degree qemu.
Good luck on doing good things, anyways. Privet vsem :)
Linux VServer Project is doing just fine now Posted Dec 6, 2005 8:33 UTC (Tue) by kolyshkin (subscriber, #34342) [Link] did you compare OVZ and VS1.x or 2.x?We compared with 2.0. VServer mostly uses ulimit, which can handle some cases (e.g. limit the number of processes), but is quite far from what is needed (and what UBC can do). It's critical given such a security bug-ridden kernel as 2.6.x presently is, AFAIR there were at least one local kernel since 2.6.8 (maybe I'm still sleepy though).As I explained above, we do care of security; among the other things, we backport all the relevant security fixes. Say, in 022stab050 release (see changelog) I can count at least couple of security fixed from mainstream (as well a couple of our own ones). Also, Solar Designer is currently performing an extensive security audit for OpenVZ. Maybe it's worth bothering SWsoft's mgmt regarding some kind of "merger" proposal? We were talking about it with Herbert (and generally I think we have good relationship with VServer project). Looks like although the ideas behind OpenVZ and Linux-VServer are similar, the implementation details differs a lot, that makes merge practically impossible. Good luck on doing good things, anyways. Thanks! ;)
Linux VServer Project is doing just fine now Posted Dec 8, 2005 20:30 UTC (Thu) by gvy (guest, #11981) [Link] > Also, Solar Designer is currently performing an extensive security audit for OpenVZ.Sounds interesting. (is it going to get 2.6-ow before usual .20?.. :)
Linux VServer Project is doing just fine now Posted Dec 6, 2005 15:59 UTC (Tue) by micah (subscriber, #20908) [Link] Linux-vserver's 2.6.12.4-vs2.0 is the most recent stable release, there are patches available for 2.6.13 and 2.6.14, but they are not "blessed" as stable (although they contain bugfixes, and are very likely *stable*).
From the mailing list on the subject of the differences:
(will use Z for OpenVZ and S for Linux-VServer)
>> Factors of interest are
Z: the announcement reads "first stable OVZ version"
>> - Debian support,
Z: afaik they are redhat oriented (and recently
Z: no idea
>> - documentation and
Z: no idea
>> - community support,
Z: irc channel and forum/bug tracker
>> - security.
guess both projects are trying to keep high security
OpenVZ is doing fine as well ;) Posted Dec 6, 2005 19:00 UTC (Tue) by kolyshkin (subscriber, #34342) [Link] Let me comment on these claims.
- stability, Although this is indeed the first stable OpenVZ release, OpenVZ is essentially Virtuozzo (without its bells and whistles), and Virtuozzo for Linux is available for more than five years already.
- Debian support, Certainly you can compile OpenVZ from sources and use it on any Linux distro. And yes, we are a part of Gentoo for about two months already (with all the recent releases making their way into Gentoo in a very timely fashion). Debian is one of our goals (see roadmap), although personally I am not a Debian expert, still with some help from the Debian community we will make it. - hardware utilization, We are supporting x86 (i386), x86_64 (AMD64, EM64T) and ia64 platforms. "Supporting" here means we have enough hardware for all the three platforms, and do an extensive quality testing (functionality, performance and stress tests) and security audit on all of them. It's a pity but we can not provide the same level of support for other platforms than those three. Speaking of specific hardware, we are supporting the same set of hardware that RHEL4 does, achieving this by backporting newer drivers from mainstream, vendors and RHEL4 kernels. There is an official Virtuozzo/OpenVZ HCL. - documentation and We have an extensive 100-pages user's guide. Also all utilities has man pages, and there are some short to-the-point howtos on the site and the forum. - community support, There is a bug tracking (Bugzilla) and quite an active support forums, also we have mailing lists and IRC channel (#openvz at freenode). We also provide fee-based support for OpenVZ, done by the same excellent team who supports Virtuozzo. - security. I can definitely say our security is higher than that of vanilla kernel. We achieve that by two means: (1) sticking to older kernel and backporting all the fixes from mainstream and (2) hiring top-rated security specialists to do OpenVZ security audit.
OpenVZ is doing fine as well ;) Posted Dec 7, 2005 12:47 UTC (Wed) by nnnn (guest, #34393) [Link]
Are you saying that when these "top-notch security people" find new
That would be the only way for your offering to be more secure
OpenVZ is doing fine as well ;) Posted Dec 7, 2005 14:37 UTC (Wed) by kolyshkin (subscriber, #34342) [Link] First of all, we do always send fixes we have upstream, and external people who do security audit for us are also sending their relevant findings upstream.
Second, as I have already tried to explain by my first comment here, our security can indeed be better than that of vanilla kernel. We achieve this by sticking to older (=more stable) kernel and backporting all the relevant fixes from vanilla and RHEL kernels (currently there are about 200 such patches, not counting driver updates).
So, we fix bugs and security holes, but do not introduce any new ones, that _might_ be coming with newer kernels. This is essentially the same model used by Red Hat Enterprise Linux.
OpenVZ is doing fine as well ;) Posted Dec 16, 2005 18:02 UTC (Fri) by dev (guest, #34359) [Link] No, we are not hiding security fixes and participate in security@kernel.org.But as you can see both OpenVZ and vserver add additional layer inside the kernel and to some extent actually introduce a new security model. And this security model should be review/checked/thinked over. For example, when you have a kernel bug which allows 'root' user to crash your node it is usually ok, since root is a priviledges and trusted user. In OpenVZ/vserver model root is untrusted user. See the difference? Just an example...
Linux VServer Project has done this for years Posted Dec 6, 2005 16:04 UTC (Tue) by micah (subscriber, #20908) [Link] It sounds like the OpenVZ concept of UBC is basically identical to the token bucket limits that Linux-vserver has with possible more resource limits (at the cost of being more intrusive).
Linux VServer Project has done this for years Posted Dec 7, 2005 10:23 UTC (Wed) by kolyshkin (subscriber, #34342) [Link] Token bucket is just an algorithm that can be used in rate limiting (such as traffic shaping etc.), it has almost nothing to do with resource management. The closest analogue to OpenVZ's User Beancounters is VServer's ulimit/rlimit. Still resource management is a very complex topic and I believe it is better implemented in OpenVZ.
Linux VServer Project has done this for years Posted Dec 9, 2005 11:57 UTC (Fri) by xyphus (guest, #34432) [Link] Does venet allow a VPS to open a secure VPN or tunnel with another host and keep it really private wrt other VPS ? VServer does not seem to allow that yet.
An interesting feature in VServer is what they called unification: secure hard links between identical executables and libs between vservers to save both disk space and memory.
Another question for kolyshkin: what exactly the commercial virtuozzo adds to OpenVZ ?
Thanks,
Linux VServer Project has done this for years Posted Dec 9, 2005 12:49 UTC (Fri) by kolyshkin (subscriber, #34342) [Link] Does venet allow a VPS to open a secure VPN or tunnel with another host and keep it really private wrt other VPS ? VServer does not seem to allow that yet.In OpenVZ each VPS has its own network, so VPSs does not see each other's traffic, so there is no need for a VPN connection between them (still, no problem doing that). An interesting feature in VServer is what they called unification: secure hard links between identical executables and libs between vservers to save both disk space and memory. Has OpenVZ a similar feature?This can be implemented by mounting, say, /lib and /usr read-only from the same directory for all the VPSs. Another solution is available in Virtuozzo (and it not yet available for OpenVZ). what exactly the commercial virtuozzo adds to OpenVZ?Differences between OpenVZ and Virtuozzo, they are outlined here and here.
The first stable OpenVZ release Posted Dec 5, 2005 22:27 UTC (Mon) by madscientist (subscriber, #16861) [Link] This sounds useful if it's got significant performance benefits over other approaches. However, note that as a virtualization tool this model is not as powerful as Xen or UML: those virtualizations allow you to actually run a completely different kernel in your virtual system, which for many applications (including all the uses I currently have for virtualization personally6) is critically important. Unless I'm misunderstanding things, all the virtual environments in OpenVZ use the same kernel.
Of course there's no reason why Linux shouldn't have both capabilities (although it may not be worth it unless OpenVZ provides significant benefits over other approaches).
The first stable OpenVZ release Posted Dec 6, 2005 1:37 UTC (Tue) by kolyshkin (subscriber, #34342) [Link] Yes, OpenVZ uses a single kernel approach. The downsides are inability to for a VPS/guest/domain/whatever_you_call_it owner to load and use some random kernel module and, theoretically, worse stability due because of the single point of failure (in practice this can be mitigated by having a very stable kernel). The benefits of the single-kernel approach are:
With OpenVZ, you have limits and guarantees for things such as various memory-related parameters, and a limit can be substantially greater than a guarantee, which means, if nobody else is using the resource, it can be granted to a VPS. With Xen or UML, you just can not do that. I agree with you that that all three virtualization techniques (the third one presented by products such as VMware or QEmu) have their use cases. For OpenVZ use cases, you can check this small article.
The first stable OpenVZ release Posted Dec 6, 2005 7:28 UTC (Tue) by job (guest, #670) [Link] I would like to know more about how OpenVZ adds security compared to a conventional chroot/chuid solution. It was my understanding that you can't bypass that without a kernel exploit, and that's probably something OpenVZ can't defend against?
The first stable OpenVZ release Posted Dec 6, 2005 9:46 UTC (Tue) by dev (guest, #34359) [Link] chroot/chuid is not a secure solution in many respects:- you can easily make a DoS on memory/CPU/disk and other resources, which are out of control of ulimits. In OpenVZ you control how many CPU/memory and disk space your VPS can consume. - root can break chroot. So you can't give people root access under chroot. This also implies that you can't run many software which require root priviliges, can't listen on some port numbers etc. - root can break chroot. This also implies, that if a hacker has accessed your system being under chroot() and is able to gain root privileges (which is not a hard problem due to SUID apps and security problems in modern software) he will be able to get out of chroot.
2 last points are not applicable to OpenVZ: you have root privileges in VPS, you can run any application you wish and it is imposible to get out of VPS due to enhanced security model and lots of preventive countermeasures which are created just to be sure that no harm can be done.
The first stable OpenVZ release Posted Dec 6, 2005 15:53 UTC (Tue) by PaXTeam (subscriber, #24616) [Link] > it is imposible to get out of VPS due to enhanced security model and lots> of preventive countermeasures which are created just to be sure that no > harm can be done
how do you prevent exploiting kernel bugs (which is what the OP was asking about i think)?
The first stable OpenVZ release Posted Dec 6, 2005 19:58 UTC (Tue) by dev (guest, #34359) [Link] There are probably 2 kinds of kernel bugs:- oopses/bugs which are not exploitable, but probably triggerable. These are just fixed in a timely manner (even faster than in RHEL, you can check update history). - exploitable bugs which allow to gain root privileges. These bugs allow to gain root privileges, but you are still bounded to VPS just in any VM-like solution. - exploitable bugs which allow to write kernel memory. These bugs allow crash the whole system or theoretically to get out of VPS. For this kind of bugs we have some countermeasures (e.g. in VFS).
So the most dangerous category of bugs is the 3rd one, but happily almost all kernel bugs belong to 1st category.
So the answer for your question is: we fix known bugs in a timely manner and insert countermeasures against getting out of VPS boundaries.
The first stable OpenVZ release Posted Dec 6, 2005 20:29 UTC (Tue) by PaXTeam (subscriber, #24616) [Link] it's the 3rd kind of bugs which i am interested in, and in particular your specific countermeasures. is there some documentation on it (or at least some pointers as to what i should be looking for in your kernel patch)?
The first stable OpenVZ release Posted Dec 9, 2005 4:45 UTC (Fri) by riel (subscriber, #3142) [Link] For the truly paranoid, a virtualization model where each virtual machine has its own kernel (ie. Xen) probably makes more sense. On large servers with only a handful of virtual machines, this model works great too.
However, for massive virtualization (think of ISPs, with hundreds of virtual machines on a system) the lower memory overhead and more flexible resource allocation of OpenVZ are probably very much worth it.
Both virtualization models have their place. I hope that we'll end up seeing both in the upstream kernel.
The first stable OpenVZ release Posted Dec 9, 2005 14:17 UTC (Fri) by PaXTeam (subscriber, #24616) [Link] > For the truly paranoid, a virtualization model where each virtual machine has its own kernel (ie. Xen) probably makes more sense.
actually, exactly the opposite is true. think about it, a set of virtual machines is the equivalent of a set of (optionally networked) physical machines. the difference is that when you compromise a machine (virtual or physical, respectively) and want to escalate your access (within the given machine or across machines), under the virtual system you get an extra potential attack vector - the VM supervisor/hypervisor/whatever itself. so the VM model is *always* less secure than the equivalent set of physical machines. and that's why kernel security (and in particular, its resistance to exploits) is even more important than on 'normal' machines.
The first stable OpenVZ release Posted Dec 15, 2005 17:44 UTC (Thu) by boklm (subscriber, #34568) [Link] I think he was talking about Xen vs OpenVZ virtualization techniques security, not VM vs physical machines security.
The first stable OpenVZ release Posted Dec 15, 2005 23:08 UTC (Thu) by PaXTeam (subscriber, #24616) [Link] you misunderstood my comment. it wasn't about Xen vs. OpenVZ but about <any VM solution> vs. physical machines. and the reason for that was the 'truly paranoid' part of his comment, as that (to me at least) implies the 'maximum (kernel) security', which cannot be <any VM solution> by definition, so the 'truly paranoid' is not better off by Xen.
as for whether Xen or OpenVZ has better 'kernel' security, it depends on what privilege escalation you're interested in.
for cross-VM escalation ('getting root in another VM from your current VM'), Xen could be better off since there you'd have to exploit a hypervisor bug (in addition to a per-VM kernel bug) whereas in OpenVZ exploiting a kernel bug is an automatic cross-VM exploit as well (well, modulo their security measures that we have yet to hear from the developers). whether Xen's hypervisor is exploitable or not is an open question.
for in-VM escalation ('getting root in the current VM from a non-privileged account'), there's no difference between the two, exploiting a kernel bug will achieve it in both cases (modulo again the OpenVZ security measures, which can of course be applied under Xen as well, so they'd still be equal).
The first stable OpenVZ release Posted Dec 15, 2005 23:17 UTC (Thu) by riel (subscriber, #3142) [Link] Note that with Xen you could implement some security measures that can not be as easily exploited on a physical computer or with a single-kernel model.
It is possible for the hypervisor to enforce that certain memory in the virtual machine is read-only, and only that memory can be mapped at certain virtual addresses. Using this protection, you can restrict the memory that kernel exploits can mess with, protecting things like kernel text, the system call table, kernel page tables, etc...
This security can not be easily circumvented if there is a kernel bug, since it is enforced by the hypervisor.
The first stable OpenVZ release Posted Dec 16, 2005 15:41 UTC (Fri) by PaXTeam (subscriber, #24616) [Link] part of what you described as a potential feature in Xen has been implemented in PaX for something like 2.5 years now (KERNEXEC is the feature name). the reamining parts can be implemented as well, but that's quite some work i haven't found the time for yet. so single kernel image solutions can be made as resistant as what you said about Xen. this is actually the reason i asked about OpenVZ's hardening features, as i've been working on similar techniques and am obviously interested in other ideas.
The first stable OpenVZ release Posted Dec 9, 2005 14:22 UTC (Fri) by madscientist (subscriber, #16861) [Link] I agree that _IF_ the OpenVZ model allows for much more efficient utilization of resources that it would be a valuable addition to the kernel, and worth having both types of virtualization available.
Just a point, though: "true paranoia" is not the only reason to choose a UML/Xen model. At work, for example, we develop embedded software, and we use UML as a simulation environment. We can run three, four, or even more "virtual machines" on our desktops using UML, and hook them together with networking. Obviously we don't want to be forced to run the same kernel on our desktops that we use in our embedded system.
As another example, at home I'm forced to use a proprietary VPN solution to work from home (that's the only method they support for remote access). This solution has a binary-only kernel module and as with all such modules, they only support a limited set of kernel versions. I can use UML to run a virtual machine with the correct kernel version to connect to work, while still using the kernel I prefer natively.
The first stable OpenVZ release Posted Dec 18, 2005 10:14 UTC (Sun) by dev (guest, #34359) [Link] Sorry for delay.Sure, we can't prevent exploiting kernel bugs at all. All we can do is to add some barriers which would require special knowledge and efforts to overcome them. Some of them are: d_root_check(), check_area_XXX() which try to prevent access to files 100% not belonging to the user. another idea which you brought to me with your message - is to extend vps_dumble flag (which is another security extension, preventing host process to be dumpable when it "entered" the VPS), so that process from host system would always stay undumpable for VPS, not on "enter" only. If you have some other ideas on security, I will be glad to discuss it and hear you opinion on this topic. Probably it's better if we move to devel at openvz dottt org what do you think?
|
|||||||||||||