Building header files into the kernel
Building header files into the kernel
Posted Mar 22, 2019 2:07 UTC (Fri) by neilbrown (subscriber, #359)Parent article: Building header files into the kernel
Store the compressed image in the __init section (which is discarded somewhere in the boot sequence) and have code to copy it into a tmpfs filesystem (which can be paged out).
Maybe put /proc/config.gz there too.
(and maybe a "cowsay" binary too, just it case it is ever needed).
Posted Mar 22, 2019 10:02 UTC (Fri)
by mjthayer (guest, #39183)
[Link] (1 responses)
Glad to see someone else had the same idea. Might it make sense to reduce the in-kernel policy a bit by letting user-space handle the tmpfs? One possible (!) implementation would be to have a device which a) lets you read out the archive and b) lets you free the in-kernel memory holding the archive after you have done with it. (Maybe that could mean putting it into __init first and copying it from there to free-able kernel memory.)
Then again, perhaps the same thing could be achieved without it ever being in kernel memory. Tagged onto the end of the kernel binary? Something similar to initramfs? I was going to write "put into initramfs", which would avoid kernel changes altogether, but the thought of bloating that even more is not appealing.
Posted Mar 23, 2019 22:12 UTC (Sat)
by Mattimo (subscriber, #129903)
[Link]
Posted Mar 22, 2019 12:33 UTC (Fri)
by excors (subscriber, #95769)
[Link] (2 responses)
I think most Android devices don't have swap. They prefer to just kill background apps whenever RAM gets low.
Posted Mar 22, 2019 15:20 UTC (Fri)
by dezgeg (subscriber, #92243)
[Link] (1 responses)
Posted Mar 22, 2019 17:45 UTC (Fri)
by excors (subscriber, #95769)
[Link]
Posted Mar 24, 2019 1:29 UTC (Sun)
by _joel_ (subscriber, #112763)
[Link] (5 responses)
Also note that on Android, we don't use disk-based swap. We have a memory based compressed swap called ZRAM, but the archive is already compressed so the suggested idea would provide no benefit (to us).
One thing I have thought of doing in the future is to make the /proc entry of the archive writeable and write an empty string into it thus freeing the archive's allocated memory and requiring a reboot. However at the moment, I am considering only building this as a module for production Android, and as a built-in when debugging. After the patches can make it, and if others want to free that memory, we can cross that bridge there IMO - such as by writing an empty string into the proc entry.
Posted Mar 24, 2019 1:38 UTC (Sun)
by _joel_ (subscriber, #112763)
[Link]
Posted Mar 24, 2019 4:28 UTC (Sun)
by neilbrown (subscriber, #359)
[Link]
If you don't have real swap, this doesn't help you, but it might be a useful answer to people who complain about a waste of kernel memory, or who emphasize that it is non-swapable memory (the article mentions both).
Posted Mar 24, 2019 5:39 UTC (Sun)
by jsmith45 (guest, #125263)
[Link] (1 responses)
Joel, why even bother remounting? You can just directly use the pre-existing internal kernel mount as if it was a special form of swappable kernel memory. CONFIG_BIG_KEYS actually does that, and the the bpfilter user mode helper module copies data that is baked into its module (as discardable init data) into the main internal tmpfs mount, so there is precedent for most of this. I'll be showing code as if making the file swappable was unconditional, but it should be very obvious how to add a config option that hybridizes what patch V5 has, and the below. One word of warning: This basically requires CONFIG_TMPFS. It will also work if !CONFIG_SHMEM, because then it uses tiny-shmem (which is ramfs). However using CONFIG_SHMEM without CONFIG_TMPFS yields a tmpfs that is excessively limited. Now lets create the file in tmpfs, and copy the data into it: OK. Next up, we need to change the section the baked-in data is stored in to be .init.data so it gets discarded after init. Almost done. All that is left is to make reads of the proc file return data from the in memory file. There are actually several ways to approach this. We could use a magic symlink to the internal file, similar to how the proc file descriptor symlinks work. But to make this act as much like a normal file as possible, the following might be easiest. It is definitely a bit odd to have the read delegate back to vfs_read, but as far as I can tell it should work. I'd bet one of VFS guys can come up with some improvements to this approach. Please note that all this code is untested, and is intended to a show the basic approach, although it probably works as-is, modulo any typos (since most of it was copy-pasted from existing parts of the kernel or your patch). Hope you find this helpful, or at least interesting
Posted Mar 26, 2019 23:53 UTC (Tue)
by quotemstr (subscriber, #45331)
[Link]
It occurs to me that we shouldn't even have these options. The kernel would be simpler to reason about if basic, fundamental, and cheap things like tmpfs and shmem (and procfs!) were hardwired to =y. I've never understood the rationale for extreme configurability. Fundamental things should always be available.
Posted Mar 26, 2019 7:34 UTC (Tue)
by minchan (subscriber, #61813)
[Link]
zRAM supports idle and/or incompressible page writeback once admin configures backing store.
Building header files into the kernel
> Store the compressed image in the __init section (which is discarded somewhere in the boot sequence) and have code to copy it into a tmpfs filesystem (which can be paged out).
> Maybe put /proc/config.gz there too.
Building header files into the kernel
Building header files into the kernel
Building header files into the kernel
Building header files into the kernel
Building header files into the kernel
Building header files into the kernel
Building header files into the kernel
I don't think there is an existing way to mount this into a mount namespace, but I suspect you could add a mount option to tmpfs to say "use pre-exist superblock named foo".
Building header files into the kernel
static struct path *mem_path;
static int __init ikheaders_init(void)
{
struct proc_dir_entry *entry;
size_t kernel_headers_data_size = &kernel_headers_data_end - &kernel_headers_data;
int err;
ssize_t written;
loff_t pos = 0;
/* copy data to a new tmpfs file, so it can be swapped out */
mem_file = shmem_kernel_file_setup("", kernel_headers_data_size, 0);
if (IS_ERR(mem_file)) {
err = PTR_ERR(mem_file);
goto err_no_fput;
}
written = kernel_write(mem_file, kernel_headers_data, kernel_headers_data_size, &pos);
if (written != kernel_headers_data_size) {
err = written;
if (err >= 0)
err = -ENOMEM;
goto error;
}
/* create the current headers file */
entry = proc_create("kheaders.tar.xz", S_IRUGO, NULL,
&ikheaders_file_ops);
if (!entry) {
err = -ENOMEM;
goto error;
}
proc_set_size(entry,
kernel_headers_data_size);
return 0;
error:
fput(mem_file);
error_no_fput:
return ret;
}
static void __exit ikheaders_cleanup(void)
{
remove_proc_entry("kheaders.tar.xz", NULL);
path_put(mem_path);
}
asm (
" .pushsection .init.data, \"a\" \n"
/*...*/
);
static int ikheaders_open_current(struct inode *inode, struct file *file)
{
struct file *mem_file
file = dentry_open(path, O_RDONLY, current_cred());
if (IS_ERR(file))
return PTR_ERR(file);
file->private_data = mem_file;
return 0;
}
static ssize_t
ikheaders_read_current(struct file *file, char __user *buf,
size_t len, loff_t *offset)
{
struct file *mem_file = file->private_data;
return vfs_read(mem_file, buf, len, offset);
}
static int ikheaders_release_current(struct inode *inode, struct file *file)
{
struct file *mem_file = file->private_data;
fput(mem_file);
return 0;
}
static const struct file_operations ikheaders_file_ops = {
.open = ikheaders_open_current,
.release = ikheaders_release_current,
.read = ikheaders_read_current,
.llseek = default_llseek,
};
Building header files into the kernel
Building header files into the kernel
If the data is used rarely or incompressible, it will be written back to the storage from the memory of zram.
