How to Set Columns for NetProfiler and NetShark Reports

SteelScript provides powerful features to quickly and easily retrieve data using simple Python scripts. Such a script basically involves following steps:

     1. Establishing a connection to the appliance;

     2. Setting data source/range;

     3. Selecting the traffic columns;

     4. Getting data from the appliance.

 

This article will discuss how to get the set of available columns and choose the desired ones. We will use both NetProfiler and NetShark as examples.

 

1. General methodology of setting columns

 

When you set out to write such a script, you should have answers to the following questions: How the result data should be grouped? by time or by host? What are the desired metrics? average bytes per second? number of connections? or something else?

 

Once you have a rough idea about the data, it is suggested you follow 4 steps as below:

  • Obtain all the available columns.
  • Identify the name of key columns and value columns.
  • Create a list of python objects by utilizing the column names.
  • Sort the data based on some columns.

 

2. How to set columns in NetProfiler scripts

 

Many previous posts have talked about writing up NetProfiler scripts for different use cases. In this post, the columns are set as below:

columns = [p.columns.key.host_ip,  
           p.columns.value.avg_bytes,  
           p.columns.value.total_conns_active,  
           p.columns.value.network_rtt]

 

We can tell the script tries to get traffic data in the metrics of average bytes, active connections and network round trip time, which are grouped by host ip addresses. This document talks about how to find all available columns for a specific type of report. Note that the eventual chosen columns need have at least one key column, otherwise the resulting data would not be very useful.

 

In order to choose the sorting column, just set the sort_column keyword argument when calling the report.run method as below:

 

sort_column = p.columns.value.avg_bytes 
report.run('hos', columns, sort_col=sort_column)






Note that resulting data records will be sorted by the descending order.

 

3. How to set columns in NetShark scripts


There are some introduction about NetShark columns here (Note that "Extractor fields" are similar in concept to NetProfiler columns).


This post talks about how to do packet analysis with NetShark by collecting byte counts grouped by server port. In this post, the columns used are:

columns = [  
    Key(netshark.columns.tcp.server_port),  
    Value(netshark.columns.generic.bytes)]


 

To create the list of NetShark columns, first import the Key and Value classes:

from steelscript.netshark.core.types import Value, Key  

 

To show all available columns for this netshark appliance, simply run the following command:

steel netshark fields <netshark_hostname> -u <username> -p <password>

 

Then all the columns will be presented as below:

ID                                                   Description                                         Type

------------------------------------------------------------------------------------------------------------------------

arp.bits                                             Bit count of ARP packets                            UINT64

...                                                  ...                                                  ...

 

Then identify the columns in the output and use IDs to create NetShark Key/Value Column objects as below:

 

Key(netshark.columns.<ID>)
Value(netshark.columns.<ID>)





 

Note the ID will include a '.' in the middle, such as 'tcp.server_port'.

 

To sort the results, use the arguments 'sortby' and 'sorttype' as below:

data = view.get_data(aggregated=True, sortby=1, sorttype='descending')  

 

Note that '1' shows that the sort column is the first one of all the value columns. 'descending' sort type ensures the results are sorted from big to small.

 

4. Conclusion

 

Hopefully this article has cleared the myth about setting columns for NetProfiler and Netshark reports.  As always, happy scripting!