[ Top ]

gpss

Table of Contents

  1. gpss : Interactive and batch gnuploting of various data sources, including but not limited to snort, various linux /proc entries, vmstat, nagios, ping, traceroute and snmp.


Contents

  1. gpss
  2. Interactive and batch gnuploting of various data sources, including but not limited to snort, various linux /proc entries, vmstat, nagios, ping, traceroute and snmp.

    The Source:

    gpss/dist/latest

    Description:

    gpss is a command line glue layer between various data sources and gnuplot.

    It allows simple and easy interactive and batch construction of 2d and 3d graphs.

    Some Examples

    Or a screen shot

    Note: The test data set was gathered with a broken pcap, so the libpcap based pkt values are all broked up. The data set was taken during test runs of artificially created traffic, and are probably not what would be seen in normal operations.

    Usage

    For a list of arguments use the '--help' flag.
    './gpss --tutorial | less'
    './gpss --faq | less'

    Will get you to either the tutorial or the faq.

    A short version however is this, and it works for me.

    	./gpss tlgraph 
    
    Assuming your on a machine with a /var/snort/snort.stats, you'll get a nice 3d graph to play with on your X display. It'll look something like the following, but you can rotate it around. By default it'll check back with the file every 5 seconds and replot if there is new data.

    Using the standard gnuplot keys and your mouse you can rotate around the data set. Press 'h' for the help screen.

    
    	./gpss abppsyn --tail 200 
    
    
    Will result in a 2d graph, not unlike the following. The '--tail' flag does just about what you'd think it would do, it reads the last 200 lines of the snort.stats file. This can allow you to setup your own little noc on your desktop if you wished. Realtime and live analysis of your snort instance. Just turn up the frequency of the perfmonitor output and away you go.

     
    
    	./gpss --list 
    
    

    Gives a list of all the available data columns, the precanned graphs, and personas. It will change based on what persona is use, and what data is is being read at the time. For instance, depending on which snmp variables are polled, different variables will be avalible.

    
    	Showing list of things
    Number   Name                  Type
    ----------------------------------------
    1        time_t                time_t
    2        drops                 pkts%
    3        megabits              bits/sec
    4        alerts                cnt
    5        kpackets              kilocnt
    6        avg_bytes             bytes/pkt
    7        patmperc              perc
    8        syn                   pkt
    9        syn_ack               pkt
    10       new_sessions          session
    11       del_sessions          session
    12       open_sessions         session
    13       max_sessions          session
    14       stream_flush          cnt
    15       stream_faults         cnt
    16       stream_timeouts       cnt
    17       frag_create           cnt
    18       frag_complete         cnt
    19       frag_inserts          cnt
    20       frag_del              cnt
    21       frag_autofree         cnt
    22       frag_flushes          cnt
    23       frag_cur              cnt
    24       frag_max              cnt
    25       frag_timeouts         cnt
    26       frag_faults           cnt
    27       ncpus                 cnt
    28       cpu_0_usr             cpu%
    29       cpu_0_sys             cpu%
    30       cpu_0_idle            cpu%
    31       mbits_wire            bits/sec
    32       mbits_ipfrag          bits/sec
    33       mbits_ipreass         bits/sec
    34       mbits_rebuilt         bits/sec
    35       mbits                 bits/sec
    36       abpp_wire             bytes/pkt
    37       abpp_ipfrag           bytes/pkt
    38       abpp_ipreass          bytes/pkt
    39       abpp_rebuilt          bytes/pkt
    40       abpp                  bytes/pkt
    41       kpkts_wire            kilopkts
    42       kpkts_ipfrag          kilopkts
    43       kpkts_ipreass         kilopkts
    44       kpkts_rebuilt         kilopkts
    45       kpkts                 kilopkts
    46       pkts_rcev             pkts
    47       pkts_drop             pkts
    
    
    
    Graphs : 
    --------------------------------------------------------------------------------
                    1 D
    abppg            :  abpp,abpp_wire,abpp_ipfrag,abpp_ipreass,abpp_rebuilt
    cpu              :  cpu_0_usr,cpu_0_sys,cpu_0_idle
    frags            :  frag_create,frag_complete,frag_inserts,frag_del,frag_autofree,frag_flushes,frag_cur,frag_max,frag_timeouts,frag_faults
    kpktsg           :  kpkts,kpkts_wire,kpkts_ipfrag,kpkts_ipreass,kpkts_rebuilt
    mbitsg           :  mbits,mbits_wire,mbits_ipfrag,mbits_ipreass,mbits_rebuilt
    pktsg            :  pkts_rcev,pkts_drop
    sessions         :  new_sessions,del_sessions,open_sessions
    sessionsm        :  new_sessions,del_sessions,open_sessions,max_sessions
    stream           :  stream_flush,stream_faults,stream_timeouts
                    2 D
    abppsyn          :  abpp_wire    syn,syn_ack
    kbfoot           :  kpackets    megabits
    lgraph           :  abpp    kpkts
    lgraph_rebuilt   :  abpp_rebuilt    kpkts_rebuilt
    lgraph_wire      :  abpp_wire    kpkts_wire
                    3 D
    chongo           :  time_t    abpp_wire    syn,syn_ack
    monkey           :  time_t    abpp_wire    cpu_0_usr
    noc1             :  time_t    drops    alerts
    noc2             :  pkts_rcev,pkts_drop    mbits_wire    alerts
    noc3             :  time_t    abpp_wire,syn,syn_ack    alerts
    slgraph          :  syn,syn_ack    abpp    kpkts
    tlgraph          :  time_t    abpp    kpkts
    
    
    
    
    
    
    
    
    Personalities : 
    
    selected         personality           description
    ------------------------------------------------------------------------------------------------------------------------
                     /proc/diskstats       a linux 2.6  /proc/diskstats reader  aproximatly equivilant to iostat
                     /proc/meminfo         a linux 2.4 / 2.6  /proc/meminfo reader
                     /proc/stat            a linux 2.4/2.6  /proc/stat reader
                     /proc/vmstat          linux  2.x /proc/vmstat reader
                     icmp_ping             icmp ping a host : --poll_cmd_args " -n -i 2 -c 5 www.grotto-group.com " 
                     keyvalue              accept a file filled with key = value pairs 
                     none                  no persona, used mostly for manualy post processing gpss data files.  Requires a --poll_src gpss://
                     ping                  icmp ping a host : --poll_cmd_args " -n -i 2 -c 5 www.grotto-group.com " 
                     snmpget               netsnmp : snmpget shell : --poll_cmd_args -Of -c public -v 1 192.168.1.51  ifInOctets.1 ifOutOctets.1  
      current        snort_stats           The original, a personality for graphing the snort.stats file that comes from the snort IDS
                     traceroute            traceroute a host : --poll_cmd_args "-q 1 www.google.com" 
    
    
    

    If you got an error message and a usage page, you probibly don't ahve snort installed so the default snort persona could not find the data where it was expected.

    To get an idea of how to use different personas and how to operate one of htem try. ./gpss --persona ping --poll_cmd_args " -n -i 2 -c 5 www.grotto-group.com " --list Note, several rows of data need to be gathered before the process will return so it may take a few seconds. I get.

    
    
    Showing list of things
    Number   Name                  Type
    ----------------------------------------
    1        time_t                time_t
    2        stddiv_rtt            ms
    3        min_rtt               ms
    4        packets_received      cnt
    5        max_rtt               ms
    6        avg_rtt               ms
    7        loss_perc             percent
    8        median_rtt            ms
    9        packets_sent          cnt
    
    
    
    Graphs : 
    --------------------------------------------------------------------------------
    io               :  pktcounts sent and received
    rtt              :  pingtime rtt, max/min/avg/median 
    rtt_drop         :  rtt vs loss perc
    rtt_dropt        :  time vs loss perc vs rtt
    
    
    
    
    
    
    
    
    Personalities : 
    
    selected         personality           description
    ------------------------------------------------------------------------------------------------------------------------
                     /proc/PID/stat        a linux /proc/PID/stat reader
                     /proc/PID/statm       a linux /proc/PID/statm reader
                     /proc/diskstats       a linux 2.6  /proc/diskstats reader  aproximatly equivilant to iostat
                     /proc/interrupts      a linux /proc/interrupts reader
                     /proc/loadavg         a linux /proc/laodavg reader
                     /proc/meminfo         a linux 2.4 / 2.6  /proc/meminfo reader
                     /proc/net/dev         a linux /proc/net/dev reader
                     /proc/slabinfo        a linux /proc/slabinfo reader : versions(2.0)
                     /proc/stat            a linux 2.4/2.6  /proc/stat reader
                     /proc/vmstat          linux  2.x /proc/vmstat reader
                     fbsd_vmstat           a freebsd vmstat reader (flags : -i -f -m -s -z ) 
                     icmp_ping             icmp ping a host : --poll_cmd_args " -n -i 2 -c 5 www.grotto-group.com " 
                     keyvalue              accept a file filled with key = value pairs 
                     nagios_perf           a generic ish nagios_perf stats reader
                     none                  no persona, used mostly for manualy post processing gpss data files.  Requires a --poll_src gpss://
      current        ping                  icmp ping a host : --poll_cmd_args " -n -i 2 -c 5 www.grotto-group.com " 
                     snmpget               netsnmp : snmpget shell : --poll_cmd_args -Of -c public -v 1 192.168.1.51  ifInOctets.1 ifOutOctets.1  
                     snort_stats           The original, a personality for graphing the snort.stats file that comes from the snort IDS
                     traceroute            traceroute a host : --poll_cmd_args "-q 1 www.google.com" 
                     vmstat                a linux vmstat reader
    
    
    
    
    

    You can see the graphs, and the avalible data colums. To get a live graph of the data, just replace --list with the graph name to the previous command line. ./gpss --persona ping --poll_cmd_args " -n -i 2 -c 5 www.grotto-group.com " rtt

    If you would like to see one of the data times, list that instead, or a list of them ./gpss --persona ping --poll_cmd_args " -n -i 2 -c 5 www.grotto-group.com " min_rtt,max_rtt,avg_rtt Just a little note about ping, to get stddiv you'll need to increase the number of packets to greater than 5. Even at 5 stdiv is barely meaningfull.

    Lets get back to the snort persona, it's much simpler and cleaner to watch and understand.

    The rest is rather strait forward. Just list things on the command line you want graphed.

    Graphing pkts/sec is just

     
    	 ./gpss pkts
    
    

    Graphing pkts vs mbits is just

     
    
    	./gpss pkts mbits 
    
    

    Graphing abpp, abpp_wire and abpp_rebuilt is just as easy.

     
    
    	./pgss abpp,abpp_wire,abpp_rebuilt
    
    

    The same graph but vs mbits /sec is

    
    	./pgss abpp,abpp_wire,abpp_rebuilt  mbits 
    
    

    To add a third dimention to the graph, just add another arugment. For example.

     
    	./gpss abpp mbits pkts 
    

    ./gpss --help for usage information

    Installing:

    1. The script should be everything you need, plus some gnuplot and misc. unix utilities.
    2. Use your version of perl not mine, though /usr/bin/perl is rather standard.
    3. send me gulfie@grotto-group.com bug reports if anything goes wrong, or something could go better.

    Requisites

    Note: The test data set was gathered with a broken pcap, so the libpcap based pkt values are all broked up. The data set was taken during test runs of artificially created traffic, and are probably not what would be seen in normal operations.

[ Top ]