User: Password:
|
|
Subscribe / Log in / New account

[RFC 2/2] x86_64: expand kernel stack to 16K

From:  Minchan Kim <minchan-AT-kernel.org>
To:  linux-kernel-AT-vger.kernel.org, Andrew Morton <akpm-AT-linux-foundation.org>
Subject:  [RFC 2/2] x86_64: expand kernel stack to 16K
Date:  Wed, 28 May 2014 15:53:59 +0900
Message-ID:  <1401260039-18189-2-git-send-email-minchan@kernel.org>
Cc:  linux-mm-AT-kvack.org, "H. Peter Anvin" <hpa-AT-zytor.com>, Ingo Molnar <mingo-AT-kernel.org>, Peter Zijlstra <a.p.zijlstra-AT-chello.nl>, Mel Gorman <mgorman-AT-suse.de>, Rik van Riel <riel-AT-redhat.com>, Johannes Weiner <hannes-AT-cmpxchg.org>, Hugh Dickins <hughd-AT-google.com>, rusty-AT-rustcorp.com.au, mst-AT-redhat.com, Dave Hansen <dave.hansen-AT-intel.com>, Steven Rostedt <rostedt-AT-goodmis.org>, Minchan Kim <minchan-AT-kernel.org>
Archive-link:  Article

While I play inhouse patches with much memory pressure on qemu-kvm,
3.14 kernel was randomly crashed. The reason was kernel stack overflow.

When I investigated the problem, the callstack was a little bit deeper
by involve with reclaim functions but not direct reclaim path.

I tried to diet stack size of some functions related with alloc/reclaim
so did a hundred of byte but overflow was't disappeard so that I encounter
overflow by another deeper callstack on reclaim/allocator path.

Of course, we might sweep every sites we have found for reducing
stack usage but I'm not sure how long it saves the world(surely,
lots of developer start to add nice features which will use stack
agains) and if we consider another more complex feature in I/O layer
and/or reclaim path, it might be better to increase stack size(
meanwhile, stack usage on 64bit machine was doubled compared to 32bit
while it have sticked to 8K. Hmm, it's not a fair to me and arm64
already expaned to 16K. )

So, my stupid idea is just let's expand stack size and keep an eye
toward stack consumption on each kernel functions via stacktrace of ftrace.
For example, we can have a bar like that each funcion shouldn't exceed 200K
and emit the warning when some function consumes more in runtime.
Of course, it could make false positive but at least, it could make a
chance to think over it.

I guess this topic was discussed several time so there might be
strong reason not to increase kernel stack size on x86_64, for me not
knowing so Ccing x86_64 maintainers, other MM guys and virtio
maintainers.

[ 1065.604404] kworker/-5766    0d..2 1071625990us : stack_trace_call:         Depth    Size
Location    (51 entries)
[ 1065.604404]         -----    ----   --------
[ 1065.604404] kworker/-5766    0d..2 1071625991us : stack_trace_call:   0)     7696      16
lookup_address+0x28/0x30
[ 1065.604404] kworker/-5766    0d..2 1071625991us : stack_trace_call:   1)     7680      16
_lookup_address_cpa.isra.3+0x3b/0x40
[ 1065.604404] kworker/-5766    0d..2 1071625991us : stack_trace_call:   2)     7664      24
__change_page_attr_set_clr+0xe0/0xb50
[ 1065.604404] kworker/-5766    0d..2 1071625991us : stack_trace_call:   3)     7640     392
kernel_map_pages+0x6c/0x120
[ 1065.604404] kworker/-5766    0d..2 1071625992us : stack_trace_call:   4)     7248     256
get_page_from_freelist+0x489/0x920
[ 1065.604404] kworker/-5766    0d..2 1071625992us : stack_trace_call:   5)     6992     352
__alloc_pages_nodemask+0x5e1/0xb20
[ 1065.604404] kworker/-5766    0d..2 1071625992us : stack_trace_call:   6)     6640       8
alloc_pages_current+0x10f/0x1f0
[ 1065.604404] kworker/-5766    0d..2 1071625992us : stack_trace_call:   7)     6632     168
new_slab+0x2c5/0x370
[ 1065.604404] kworker/-5766    0d..2 1071625992us : stack_trace_call:   8)     6464       8
__slab_alloc+0x3a9/0x501
[ 1065.604404] kworker/-5766    0d..2 1071625993us : stack_trace_call:   9)     6456      80
__kmalloc+0x1cb/0x200
[ 1065.604404] kworker/-5766    0d..2 1071625993us : stack_trace_call:  10)     6376     376
vring_add_indirect+0x36/0x200
[ 1065.604404] kworker/-5766    0d..2 1071625993us : stack_trace_call:  11)     6000     144
virtqueue_add_sgs+0x2e2/0x320
[ 1065.604404] kworker/-5766    0d..2 1071625993us : stack_trace_call:  12)     5856     288
__virtblk_add_req+0xda/0x1b0
[ 1065.604404] kworker/-5766    0d..2 1071625993us : stack_trace_call:  13)     5568      96
virtio_queue_rq+0xd3/0x1d0
[ 1065.604404] kworker/-5766    0d..2 1071625994us : stack_trace_call:  14)     5472     128
__blk_mq_run_hw_queue+0x1ef/0x440
[ 1065.604404] kworker/-5766    0d..2 1071625994us : stack_trace_call:  15)     5344      16
blk_mq_run_hw_queue+0x35/0x40
[ 1065.604404] kworker/-5766    0d..2 1071625994us : stack_trace_call:  16)     5328      96
blk_mq_insert_requests+0xdb/0x160
[ 1065.604404] kworker/-5766    0d..2 1071625994us : stack_trace_call:  17)     5232     112
blk_mq_flush_plug_list+0x12b/0x140
[ 1065.604404] kworker/-5766    0d..2 1071625994us : stack_trace_call:  18)     5120     112
blk_flush_plug_list+0xc7/0x220
[ 1065.604404] kworker/-5766    0d..2 1071625995us : stack_trace_call:  19)     5008      64
io_schedule_timeout+0x88/0x100
[ 1065.604404] kworker/-5766    0d..2 1071625995us : stack_trace_call:  20)     4944     128
mempool_alloc+0x145/0x170
[ 1065.604404] kworker/-5766    0d..2 1071625995us : stack_trace_call:  21)     4816      96
bio_alloc_bioset+0x10b/0x1d0
[ 1065.604404] kworker/-5766    0d..2 1071625995us : stack_trace_call:  22)     4720      48
get_swap_bio+0x30/0x90
[ 1065.604404] kworker/-5766    0d..2 1071625995us : stack_trace_call:  23)     4672     160
__swap_writepage+0x150/0x230
[ 1065.604404] kworker/-5766    0d..2 1071625996us : stack_trace_call:  24)     4512      32
swap_writepage+0x42/0x90
[ 1065.604404] kworker/-5766    0d..2 1071625996us : stack_trace_call:  25)     4480     320
shrink_page_list+0x676/0xa80
[ 1065.604404] kworker/-5766    0d..2 1071625996us : stack_trace_call:  26)     4160     208
shrink_inactive_list+0x262/0x4e0
[ 1065.604404] kworker/-5766    0d..2 1071625996us : stack_trace_call:  27)     3952     304
shrink_lruvec+0x3e1/0x6a0
[ 1065.604404] kworker/-5766    0d..2 1071625996us : stack_trace_call:  28)     3648      80
shrink_zone+0x3f/0x110
[ 1065.604404] kworker/-5766    0d..2 1071625997us : stack_trace_call:  29)     3568     128
do_try_to_free_pages+0x156/0x4c0
[ 1065.604404] kworker/-5766    0d..2 1071625997us : stack_trace_call:  30)     3440     208
try_to_free_pages+0xf7/0x1e0
[ 1065.604404] kworker/-5766    0d..2 1071625997us : stack_trace_call:  31)     3232     352
__alloc_pages_nodemask+0x783/0xb20
[ 1065.604404] kworker/-5766    0d..2 1071625997us : stack_trace_call:  32)     2880       8
alloc_pages_current+0x10f/0x1f0
[ 1065.604404] kworker/-5766    0d..2 1071625997us : stack_trace_call:  33)     2872     200
__page_cache_alloc+0x13f/0x160
[ 1065.604404] kworker/-5766    0d..2 1071625998us : stack_trace_call:  34)     2672      80
find_or_create_page+0x4c/0xb0
[ 1065.604404] kworker/-5766    0d..2 1071625998us : stack_trace_call:  35)     2592      80
ext4_mb_load_buddy+0x1e9/0x370
[ 1065.604404] kworker/-5766    0d..2 1071625998us : stack_trace_call:  36)     2512     176
ext4_mb_regular_allocator+0x1b7/0x460
[ 1065.604404] kworker/-5766    0d..2 1071625998us : stack_trace_call:  37)     2336     128
ext4_mb_new_blocks+0x458/0x5f0
[ 1065.604404] kworker/-5766    0d..2 1071625998us : stack_trace_call:  38)     2208     256
ext4_ext_map_blocks+0x70b/0x1010
[ 1065.604404] kworker/-5766    0d..2 1071625999us : stack_trace_call:  39)     1952     160
ext4_map_blocks+0x325/0x530
[ 1065.604404] kworker/-5766    0d..2 1071625999us : stack_trace_call:  40)     1792     384
ext4_writepages+0x6d1/0xce0
[ 1065.604404] kworker/-5766    0d..2 1071625999us : stack_trace_call:  41)     1408      16
do_writepages+0x23/0x40
[ 1065.604404] kworker/-5766    0d..2 1071625999us : stack_trace_call:  42)     1392      96
__writeback_single_inode+0x45/0x2e0
[ 1065.604404] kworker/-5766    0d..2 1071625999us : stack_trace_call:  43)     1296     176
writeback_sb_inodes+0x2ad/0x500
[ 1065.604404] kworker/-5766    0d..2 1071626000us : stack_trace_call:  44)     1120      80
__writeback_inodes_wb+0x9e/0xd0
[ 1065.604404] kworker/-5766    0d..2 1071626000us : stack_trace_call:  45)     1040     160
wb_writeback+0x29b/0x350
[ 1065.604404] kworker/-5766    0d..2 1071626000us : stack_trace_call:  46)      880     208
bdi_writeback_workfn+0x11c/0x480
[ 1065.604404] kworker/-5766    0d..2 1071626000us : stack_trace_call:  47)      672     144
process_one_work+0x1d2/0x570
[ 1065.604404] kworker/-5766    0d..2 1071626000us : stack_trace_call:  48)      528     112
worker_thread+0x116/0x370
[ 1065.604404] kworker/-5766    0d..2 1071626001us : stack_trace_call:  49)      416     240
kthread+0xf3/0x110
[ 1065.604404] kworker/-5766    0d..2 1071626001us : stack_trace_call:  50)      176     176
ret_from_fork+0x7c/0xb0

Signed-off-by: Minchan Kim <minchan@kernel.org>
---
 arch/x86/include/asm/page_64_types.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h
index 8de6d9cf3b95..678205195ae1 100644
--- a/arch/x86/include/asm/page_64_types.h
+++ b/arch/x86/include/asm/page_64_types.h
@@ -1,7 +1,7 @@
 #ifndef _ASM_X86_PAGE_64_DEFS_H
 #define _ASM_X86_PAGE_64_DEFS_H
 
-#define THREAD_SIZE_ORDER	1
+#define THREAD_SIZE_ORDER	2
 #define THREAD_SIZE  (PAGE_SIZE << THREAD_SIZE_ORDER)
 #define CURRENT_MASK (~(THREAD_SIZE - 1))
 
-- 
1.9.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>



(Log in to post comments)


Copyright © 2014, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds