systemtap release 0.7 announcement

Hi -


The systemtap team is proud to announce release 0.7.  The previous
release was version 0.6.2 from March.  There are several performance
and capability improvements summarized below.  Users are encouraged to
upgrade.


 = Where to get it

   http://sourceware.org/systemtap/ - our project page   

   http://sourceware.org/systemtap/ftp/releases/systemtap-0....
   git commit f1743f00e6058d2dee44510f3ef165c549a6291e


 = How to build it

   See the README file: http://tinyurl.com/6xjecv
   Further information at http://sourceware.org/systemtap/wiki/


 = Systemtap frontend (stap) changes

 - A probe listing mode is available.
   % stap -l vm.*
   vm.brk
   vm.mmap
   vm.munmap
   vm.oom_kill
   vm.pagefault
   vm.write_shared

 - Support for limited kernel probing without debuginfo via symbol
   tables -- on i386, x86_64, and powerpc so far.  (See --kelf, --kmap,
   --ignore-vmlinux, and --ignore-dwarf documentation)
   http://sourceware.org/systemtap/wiki/MakeDoWithoutDebugInfo

 - The environment variable SYSTEMTAP_DEBUGINFO_PATH now refers to a
   base directory where kernel and module debuginfo when not installed
   in the default location.

 For a full overview see the stap(1) manual page.


 = Systemtap script language changes

 - Globals now accept ; terminators

       global odds, evens;
       global little[10], big[5];

 - The vector of script command line arguments is available in a
   tapset-provided global array argv[].  It is indexed 1 ... argc,
   another global.  This can substitute for of preprocessor
   directives @NNN that fail at parse time if there are not
   enough arguments.
       printf("argv: %s %s %s", argv[1], argv[2], argv[3])

 For a full overview see the SystemTap Language Reference manual.


 = Systemtap probe points changes

 - More user-space probe types are added:
   probe process(PID).begin { }
   probe process("PATH").begin { }
   probe process(PID).thread.begin { }
   probe process("PATH").thread.begin { }
   probe process(PID).end { }
   probe process("PATH").end { }
   probe process(PID).thread.end { }
   probe process("PATH").thread.end { }
   probe process(PID).syscall { }
   probe process("PATH").syscall { }
   probe process(PID).syscall.return { }
   probe process("PATH").syscall.return { }

 - .statement("func@file+line") probes are now supported to allow
   probing by line-number offset relative to the "{" that begins the
   function.  This allows using the same systemtap script without
   change if source changes are confined to other areas of the .c file.

 For a full overview see the stapprobes(5) manual page.


 = Standard tapsets functions changes

 - Added functions to access different user space data types:
   user_string_n, user_string_n2, user_string_n_warn, user_string_n_quoted,
   user_short, user_short_warn, user_int, user_int_warn, user_long,
   user_long_warn, user_char, user_char_warn.

 - Added "cpu register" and "numbered functions argument" support
   functions which help writing "dwarfless" probes.
   register:long (name:string), u_register:long (name:string),
   int_arg:long (n:long), uint_arg:long (n:long), long_arg:long
   (n:long), ulong_arg:long (n:long), longlong_arg:long (n:long),
   ulonglong_arg:long (n:long), pointer_arg:long (n:long), s32_arg:long
   (n:long), u32_arg:long (n:long), s64_arg:long (n:long), u64_arg:long
   (n:long), asmlinkage:unknown (), fastcall:unknown () and
   regparm:unknown (n:long).

 - New function returnval:long () for usage with syscall tapset probes
   as counterpart to the returnstr:string (returnp:long) function.

 - New function user_mode:long () that returns 1 if the probe point
   occurred in user-mode.

 For a full overview see the stapfuncs(5) manual page.


 = Standard tapset global variable changes

 - NULL is now a new global variable (defined as global NULL = 0).

 For a full overview see the stapvars(5) manual page.


 = Systemtap examples changes

 - examples are now installed by default by make install under
   <prefix>/doc/systemtap*/examples
   The examples are now explicitly run during make check and
   make installcheck to make sure they are always up to date
   (they can be found under testsuite/systemtap.examples).

 - New examples:

  * io_submit.stp - When a reschedule occurs during an AIO io_submit
    call, accumulate the traceback in a histogram. When the script
    exits prints out a sorted list from most common to least common
    backtrace.

  * traceio.stp - Every second print out the top ten executables
    sorted in descending order based on cumulative I/O traffic
    observed.

  * iotop.stp - Every five seconds print out the top ten executables
    generating I/O traffic during that interval sorted in descending
    order.

  * disktop.stp - Get the status of reading/writing disk every 5
    seconds, output top ten entries during that period.

  * sigkill.stp - The script traces any SIGKILL signals. When that
    SIGKILL signal is sent to a process, the script prints out the
    signal name, the desination executable and process ID, the
    executable name user ID that sent the signal.

  * thread-times.stp - Sets up time-based sampling. Every five seconds
    it prints out a sorted list with the top twenty processes with
    samples broken down into percentage total time spent in user-space
    and kernel-space.

  * functioncallcount.stp - Takes one argument, a list of functions to
    probe. The script will run and count the number of times that each
    of the functions on the list is called. On exit the script will
    print a sorted list from most frequently to least frequently called
    function.

  * para-callgraph.stp - Takes two arguments: the first argument is the
    function to starts/stops the per thread call graph traces and the
    second argument is the list of functions to generate trace
    information on. The script prints out a timestap for the thread,
    the function name and pid, followed by entry or exit symboly and
    function name.

  * traceio2.stp - Print out the executable name and process number as
    reads and writes to the specified device occur.

  * sleepingBeauties.stp - Monitors time threads spend waiting for IO
    operations (in "D" state) in the wait_for_completion function. If a
    thread spends over 10ms wall-clock time waiting, information is
    printed out describing the thread number and executable name. When
    slow the wait_for_completion function complete, backtraces for the
    long duration calls are printed out.

  * graphs.stp - Tracks the disk and CPU utilization. The resulting
    output of the script can be piped into gnuplot to generate a graph
    of disk and CPU USE.

 For more information about examples see also the stapex(5) manual page.


 = Miscellaneous changes

 - Much faster batch unregistering of (thousands of) kprobes implemented
   (needs linux 2.6.25).

 - Scripts relying on probe point wildcards like "syscall.*" that expand
   to distinct kprobes are processed significantly faster than before.

 - Shiny new Vim highlighting (see vim/syntax/stap.vim).

 - stap now warns about variables read but not set ("read-only
   variables"), which are usually due to script typos.

 - Configure with --enable-dejazilla for optional automatic test result
   reporting.


 = The following people helped with code contributions for this release:

   Ananth N Mavinakayanahalli, Dave Brolley, David Smith, Eugene Teo,
   Frank Ch. Eigler, James Bottomley, Jim Keniston, Josh Stone,
   Mark Wielaard, Martin Hunt, Masami Hiramatsu, Shaohua Li, Srikar
   Dronamraju, Srinivasa DS, Stan Cox, Theodore Ts'o, Tim Moore,
   Wenji Huang, William Cohen and Zhaolei

   Special thanks to Mark Wielaard for drafting these release notes.


 = Known issues with this release:

   Generally, see our bug tracker at http://sourceware.org/bugzilla/

   - http://sourceware.org/bugzilla/show_bug.cgi?id=6707
     oops crashes with 2.6.25 - onoffprobe
     Feedback appreciated since this is a very hard problem to reproduce.

   - http://sourceware.org/bugzilla/show_bug.cgi?id=2725
     overbroad probing wildcards can sometimes crash the machine


 = Bugs closed for this release:

 #1165 implement .relative(nnn)
 #1194 check operation of "-o" (output direction) flag
 #2071 Probes on ISR with probes on task thread's prehandler crash the
       system
 #2600 should optimize away assignments in other contexts
 #2949 need "probe listing" command line option
 #3051 print_backtrace() prints most of stack on a single line on ppc64
 #3542 request new batch registration/unregistration API
 #4014 improve command line option processing
 #4904 hard to use statement probes for line-by-line tracing
 #5001 ctime() vs _stp_ctime() duplication
 #5101 stap abort in loc2c.c
 #5102 improve code generation of error-free constructs
 #5106 Remove old map and histogram formatting code
 #5189 Support a '*' dynamic field width specifier in printf
 #5231 Setting correct argument for bio_endio in ioblock.stp
 #5528 more user_* data conversions needed
 #5636 $builddir/SNAPSHOT should get recomputed at make check time
 #5643 profile syscall tests to improve translator speed
 #5645 Too many sub-buffers generated with -s option
 #5648 Unaligned access to memory in _stp_vsnprintf() on ia64
 #5716 staprun/stapio setuid/capability simplification
 #5759 probes in sched.c on preempt kernel cause reboot
 #5770 Seen lot of "eof" errors on systemtap-20080216 snapshot
 #5891 Insertion of kprobe module crashes the system on 2.6.25-rc3-mm1,
       if CONFIG_DEBUG_RODATA is set on x86 system.
 #5897 derived_probe derivation chain tweak
 #5928 CFLAGS vs. EXTRA_CFLAGS in generated kbuild makefile
 #5955 declaration of global variables with semicolon gives semantic error
 #5956 Usage of "NULL" in systemtap script throws non-apparent error
       messages
 #5963 testsuite/systemtap.maps/pmap_agg_overflow.stp crashes on
       2.6.25-0.121.rc5.git4.fc9
 #5975 Systemtap build error with -DNDEBUG
 #5980 set dejagnu snapshot from stap -V as fallback
 #6008 support larger on-memory buffer
 #6393 tag build with more precise git commit code
 #6400 New runtime unwinder code does not make use of locally built elfutils
 #6410 New unwind code causes errors on ia64
 #6416 SystemTap should recognize and disble probes on __devinit/__devexit
       functions
 #6417 Wildcard probe hangs system
 #6429 unwinding/symbol stuff broken on ppc*
 #6432 use 2.6.26 probe_kernel_{read,write}
 #6451 staprun shutdown deadlock
 #6454 stap -l cut off the format of marker-based probe point
 #6455 marker probe: Format mismatch error for markings defined with
       MARK_NOARGS
 #6466 more elision needed to support iffy tapsets
 #6469 duplicate warning elimination
 #6470 export command line arguments or run-time script parsing
 #6474 build setuid staprun with -fpie
 #6481 timers: stp_time_timer_callback calls mod_timer with interrupts
       disabled
 #6492 stap -l outputs so many WARNINGS.
 #6499 BUG with systemtap's utrace support
 #6500 Systemtap's utrace support needs to follow new threads
 #6509 syscall.get*id probes broken
 #6520 Systemtap 20080503 and 20080510 snapshot fails on s390
 #6524 ctime() on bad values hangs system
 #6529 error() from nested function can be lost
 #6534 utsname() becoming unusable
 #6538 warn about read-only variables
 #6550 add --enable-dejazilla
 #6563 sys.stp fails on latest systemtap-20080524 snapshot
 #6588 syscall.exit[_group].return should define name, retstr
 #6601 systemtap with --kelf option looks for data symbol to probe a function
       on powerpc systemtap
 #6644 Impure arguments prevent eliding pure function calls
 #6703 tapsets.cxx does not build on gcc 4.10
 #6732 task_struct parent field removed from RHEL-5 and fedora 9 kernels
 #6736 switch to compiled-in symbol and unwind data