Category: solaris

IPERF CSV Data Summary Script

Recently I was required to do a network performance test between a Head Office and a WAN site. I knocked up this quick python script to parse the data collect to see the results. Thought it could be handy for others so here it is to download / share.

Download Script

Sample Output:

[plain]
cooper@dfbit:~/scripts/iperf-parse$ ./iperf-summary.py
————————————
— IPERF CSV Summariser —
— Cooper Lees <me@cooperlees.com —
————————————
— SUMMARY —
– 20111212103043 to 20120103090052
– 1004 runs of IPERF
– Averages:
– Average Sent = 2.64M
– Average Received = 2.28M
– Average Send Bandwidth = 985.75K
– Average Receive Bandwidth = 805.12K
– Max Send Bandwidth = 1.08M (at 20111230183021)
– Max Receive Bandwidth = 837.16K (at 20120102113052)
————————————
[/plain]

Code:

[python]
#!/usr/bin/python

# date,sender-ip,sender-port,receiver-ip,receiver-port,id,interval,transfer,bandwidth
# 20111212103043,10.120.15.8,45020,10.120.13.120,5001,5,0.0-21.4,2490368,931080
# 20111212103109,10.120.15.8,5001,10.120.13.120,57022,4,0.0-24.2,2228224,736145

FILENAME = ‘client-iperf.log’

RUNS = 1

MAX_BANDWIDTH_SENT = 0
MAX_BANDWIDTH_SENT_DATE = 0
MAX_BANDWIDTH_RECEIVED = 0
MAX_BANDWIDTH_RECEIVED_DATE = 0

TOTAL_BANDWIDTH_SENT = 0
TOTAL_BANDWIDTH_RECEIVED = 0

TOTAL_SENT = 0
TOTAL_RECEIVED = 0

def convert_bytes(bytes):
bytes = float(bytes)
if bytes >= 1099511627776:
terabytes = bytes / 1099511627776
size = ‘%.2fT’ % terabytes
elif bytes >= 1073741824:
gigabytes = bytes / 1073741824
size = ‘%.2fG’ % gigabytes
elif bytes >= 1048576:
megabytes = bytes / 1048576
size = ‘%.2fM’ % megabytes
elif bytes >= 1024:
kilobytes = bytes / 1024
size = ‘%.2fK’ % kilobytes
else:
size = ‘%.2fb’ % bytes
return size

f = open(FILENAME)
l1 = f.readline().strip().split(‘,’)
l2 = f.readline().strip().split(‘,’)
while l2 and l2[0] != ”:
if RUNS == 1:
START = l1[0]

BW_SENT = int(l1[8])
BW_RECEIVED = int(l2[8])

TOTAL_SENT = TOTAL_SENT + int(l1[7])
TOTAL_RECEIVED = TOTAL_RECEIVED + int(l2[7])

TOTAL_BANDWIDTH_SENT = TOTAL_BANDWIDTH_SENT + int(l1[8])
TOTAL_BANDWIDTH_RECEIVED = TOTAL_BANDWIDTH_RECEIVED + int(l2[8])

if BW_SENT > MAX_BANDWIDTH_SENT:
MAX_BANDWIDTH_SENT = BW_SENT
MAX_BANDWIDTH_SENT_DATE = l1[0]

if BW_RECEIVED > MAX_BANDWIDTH_RECEIVED:
MAX_BANDWIDTH_RECEIVED = BW_RECEIVED
MAX_BANDWIDTH_RECEIVED_DATE = l2[0]

END = l2[0]
RUNS = RUNS + 1
l1 = f.readline().strip().split(‘,’)
l2 = f.readline().strip().split(‘,’)

f.close()

print "————————————"
print " — IPERF CSV Summariser — "
print "– Cooper Lees <me@cooperlees.com –"
print "————————————"
print "– SUMMARY –"
print "- %s to %s" % ( START, END )
print "- %d runs of IPERF" % RUNS
print "- Averages:"
print "-tAverage Sentttt= %s" % convert_bytes((TOTAL_SENT / RUNS))
print "-tAverage Receivedtt= %s" % convert_bytes((TOTAL_RECEIVED / RUNS))
print "-tAverage Send Bandwidthtt= %s" % convert_bytes((TOTAL_BANDWIDTH_SENT / RUNS))
print "-tAverage Receive Bandwidtht= %s" % convert_bytes((TOTAL_BANDWIDTH_RECEIVED / RUNS))
print "-tMax Send Bandwidthtt= %s (at %s)" % (convert_bytes((MAX_BANDWIDTH_SENT)), MAX_BANDWIDTH_SENT_DATE)
print "-tMax Receive Bandwidthtt= %s (at %s)" % (convert_bytes((MAX_BANDWIDTH_RECEIVED)), MAX_BANDWIDTH_RECEIVED_DATE)
print "————————————"
[/python]

Cron Job Script to Collect Data:

[bash]
#!/bin/bash

SERVER="x.x.x.x"
LOG="client-iperf.log"
TIME="5"

echo "–> Starting iperf client @ $(date) …" | tee -a $LOG

if [ "$1" == "-v" ]; then
iperf -t $TIME -c $SERVER -r -y C | tee -a $LOG
else
iperf -t $TIME -c $SERVER -r -y C >> $LOG
fi

echo "–> Finished iperf client @ $(date)" | tee -a $LOG
[/bash]

mplayer2 (and other goodies) on OpenIndiana (possibly Solaris 11)

Want mplayer2 (the newer fork of mplayer) that uses ffmpeg goodness to play all the latest avi’s with funky codecs and mp4 hidef goodness. Some Openindiana contributors are building a lot of friendly packages in the oi-sfe repo.

[plain]
sudo pkg set-authority -O http://staticdev.uk.openindiana.org:10002 oi-sfe
sudo pkg refresh –full
sudo pkg install SFEmplayer2 SFEgccruntime
[/plain]
P.S. Need to manually install SFEgccruntime as it was a forgotten dependency – This will be fixed I have been informed.
P.P.S – ALSO I have only tested and is only expect to work on OI_148

ipadm Script for Openindiana / Solaris 11 Ex

I love ipadm. It rocks. Much needed for Solaris and derivatives. I seem to keep forgetting the dam new ipadm commands tho. So I wrote this script to help me set up new Solaris machines and thought I would share.

[bash]
#!/usr/bin/bash

INTERFACE=$1
ADDRESS=$2
GATEWAY=$3

V6AUTO=1

errorCheck()
{
if [ $? -ne 0 ]; then
echo "ERROR: $@"
exit 69
fi
}

# Handle Args
if [ $# -ne 3 ]; then
echo "ERROR: Invalid arguments"
echo "Usage:nt$0: INTERFACE ADDRESS/CIDR-MASK GATEWAY/DEFAULTROUTER"
exit 1
fi

ipadm create-if $INTERFACE
errorCheck "Unable to create-if $INTERFACE"

ipadm create-addr -T static -a local=${ADDRESS} ${INTERFACE}/v4static
errorCheck "Unable to set static v4 on $INTERFACE"

if [ $V6AUTO -ne 0 ]; then
ipadm create-addr -T addrconf ${INTERFACE}/v6addr
errorCheck "Unable to set v6 autoconf on $INTERFACE"
fi

if [ $GATEWAY != "" ]; then
route add default $GATEWAY
errorCheck "Unable to set default router to $GATEWAY"
else
echo "!–> Not setting gateway as none was set …"
fi

echo "–> Finished setting $ADDRESS on $INTERFACE with $GATEWAY default route …"
[/bash]

OpenSolaris / OpenIndiana IPF

After I spent hours thinking I have lost the plot, I finally read about how IPF is configured by default now. I am not sure what build this was changed, but now, by default IPF on OpenIndiana does not look @ /etc/ipf/ipf.conf for default IPv4 IPF Rules to load @ start. To make it look @ this file apply the following.

[bash]
#!/bin/bash

IPFW_CFG="/etc/ipf/ipf.conf"
PRIV_EXEC="pfexec"

# Turn IPF back to legacy text file usage

$PRV_EXEC svccfg -s ipfilter:default setprop firewall_config_default/policy = astring: "custom"

$PRV_EXEC svccfg -s ipfilter:default setprop firewall_config_default/custom_policy_file = astring: "$IPFW_CFG"

$PRV_EXEC svcadm refresh ipfilter:default

echo "Done – Edit $IPFW_CFG and enable IPF now …"
[/bash]

Sample Conf:

[bash]
# Default policies
pass out all keep state
block in all
block return-rst in log first proto tcp all
block return-icmp(host-unr) in log proto udp all

# Allow Loopback
pass in quick on lo0 all
pass out quick on lo0 all

# Allow ICMP
pass out quick proto icmp all keep state
pass in quick proto icmp all keep state

# Allow SSH
pass in quick proto tcp from any to any port = 22 flags S/FSRPAU keep state keep frags

# Allow SSH
pass in quick proto tcp from any to any port = 80 keep state
[/bash]

Now just enable the service
pfexec svcadm enable svc:/network/ipfilter:default

Handy IPF Commands

ipf -E                          : Enable ipfilter when running
                                : for the first time.
				: (Needed for ipf on Tru64)

ipf -f /etc/ipf/ipf.conf        : Load rules in /etc/ipf/ipf.conf file
                                : into the active firewall.

ipf -Fa -f /etc/ipf/ipf.conf    : Flush all rules, then load rules in
                                : /etc/ipf/ipf.conf into active firwall.

ipf -Fi                         : Flush all input rules.

ipf -I -f /etc/ipf/ipf.conf     : Load rules in /etc/ipf/ipf.conf file
                                : into inactive firewall.

ipf -V                          : Show version info and active list.

ipf -s                          : Swap active and inactive firewalls.

ipfstat                         : Show summary

ipfstat -i                      : Show input list

ipfstat -o                      : Show output list

ipfstat -hio                    : Show hits against all rules

ipfstat -t -T 5			: Monitor the state table and refresh every  
				: 5 seconds. Output is similiar to	
				: 'top' monitoring the process table.

ipmon -s S                      : Watch state table.

ipmon -sn                       : Write logged entries to syslog, and
                                : convert back to hostnames and servicenames.

ipmon -s [file]                 : Write logged entries to some file.

ipmon -Ds			: Run ipmon as a daemon, and log to
				: default location. 
				: (/var/adm/messages for Solaris)

IPMP + Zones CFG in OpenIndiana / OpenSolaris

IPMP in Solaris allows you to have redundancy with your network on mission critical servers. It is really excellent as it allows you to mix speed of NICs (e.g. a Ten Gigabit Ethernet with Gigabit Ethernet). Below will demonstrate the ‘passive’ IPMP configuration. Active mode allows you to check the ability to ping a defined host as well as the same checks of passive, but does require more extensive configuration.
NIC Setup
Stop the scary nwam service
  • svcadm disable svc:/network/physical:nwam
Plumb each nic
  • ifconfig NIC0 plumb
  • ifconfig NIC1 plumb
Add to group
  • ifconfig NIC0 group GROUPNAME
  • ifconfig NIC1 group GROUPNAME
Edit /etc/hostname.interface
  • Primary NIC: IP/NETMASK group GROUPNAME up
  • Secondary NIC: group GROUPNAME standby
Enable traditional Solaris Networking
  • svcadm enable svc:/network/physical:default

You should now be done. Have fun yanking network cables and seeing your server stay online.

Zone Setup

Once your networking is set up just set the zone up as you normally would, but use the ipmpX as your physical zone NIC in zonecfg.

  • add net
  • set physical=ipmpX
  • end
  • verify
  • commit

For more zone configuration command information visit GenUnix Wiki.

Monitoring Opensolaris from iPhone / iPad using iStat

How to get istatd running on Opensolaris …

Well after I tried myself months ago to get istatd to compile on my Opensolaris box (token) someone smarter and who is not as lazy has worked it all out and got it to compile. One thing he did not do tho was write an init script so that it would start @ boot time. Here you can find it.

Please follow the install instructions on the following blog – I have written scripts, included a sample configuration and written a init script.

Karim Berrah’s Weblog

Compile Script (Place in parent dir to source code):

[bash]
#!/bin/bash

DIR=istatd-
VER=0.5.7

ACLOCAL110=aclocal-1.10
export ACLOCAL110
AUTOMAKE110=automake-1.10
export AUTOMAKE110
AUTOCONF26=autoconf
export AUTOCONF26
AUTOHEADER26=autoheader
export AUTOHEADER26
LIBTOOLIZE15=libtoolize
export LIBTOOLIZE15

cd ${DIR}${VER}
pwd
./configure
if [ $? -ne 0 ]; then
echo "ERROR with configure"
fi

make

echo "–> Finished compiling …"

pfexec useradd istat
pfexec groupadd istat

if [ ! -d /var/cache/istat ]; then
mkdir /var/cache/istat
fi
pfexec chown istat:istat /var/cache/istat

echo "–> Finished …"
[/bash]

Sample Config:

[plain]
#
# /etc/istat.conf: Configuration for iStat server
#

# network_addr 127.0.0.1
network_port 5901
server_code 55551
server_user istat
server_group istat
# server_socket /tmp/istatd.sock
server_pid /var/run/istat/istatd.pid
cache_dir /var/cache/istat

# Note: Only support for one network interface, limited by client.
monitor_net ( bge0 )

# Array of disks to monitor. Specify mount path or device name.
monitor_disk ( / /home )

# Set to 1 if you want to use mount path as label instead of the device name.
disk_mount_path_label 0

# Try to probe the filesystem for disk label, will override the mount path label.
disk_filesystem_label 1

# Set custom disk label. Will override all other labels.
# disk_rename_label /dev/sda1 "root"
# disk_rename_label /home "home"

# End of file
[/plain]

Init Script:
[bash]
#!/bin/bash

# Basic support for chkconfig
###
# chkconfig: 35 99 55
# description: start and stop istatd – iphone monitoring tool daemon
###

DIR=/usr/local/bin/
BINARY=istatd
OPTS="-d"

PIDFILE=/var/run/istat/istatd.pid

case "$1" in
start)
echo -n "Starting $BINARY … "
$DIR/$BINARY $OPTS
if [ $? -ne 0 ]; then
echo "! Failed !"
else
echo "done"
fi
;;
stop)
echo -n "Stopping $BINARY … "
kill $(cat $PIDFILE)
echo "done"
;;
status)
ps -ef | grep "$BINARY $OPTS"
;;
*)
echo "Usage: $0 {start|status|stop}"
exit 1
esac

exit 0
[/bash]

ln -s /etc/init.d/istatd /etc/rc3.d/S99istatd
ln -s /etc/init.d/istatd /etc/rc3.d/K10istatd

If someone has written a SMF service I would love to take it from you 🙂

If you have any questions – feel free to ask …