[Flow-tools] cpu utilisation on flowscan processes

Thu Feb 21 00:05:53 EST 2008

Hi Dave,

Thanks again for responding.

I had a look at the rc scripts, I did notice flow-capture had 2 startup
scripts, which was wierd.

I removed that.

But not flowscan, it's only being started once.

Here's a top with the required info.

top - 16:04:06 up 1 day,  5:34,  1 user,  load average: 9.20, 8.68, 8.46

Tasks:  83 total,   7 running,  75 sleeping,   0 stopped,   1 zombie

Cpu0  : 96.0%us,  4.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,
0.0%st

Cpu1  : 98.0%us,  2.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,
0.0%st

Mem:   1036428k total,  1022864k used,    13564k free,    17768k buffers

Swap:  5301368k total,       52k used,  5301316k free,   758620k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   PPID P
SWAP   TIME COMMAND

 9013 root      18   0 30280  26m 1732 R   44  2.7   2:38.67     1 0
2776   2:38 flowscan

16158 root      18   0 30284  26m 1732 R   44  2.7   0:51.91     1 1
2780   0:51 flowscan

24721 root      18   0 30116  26m 1728 R   40  2.6   6:12.83     1 0
2656   6:12 flowscan

 3869 root      18   0 30120  26m 1728 R   39  2.6   3:51.55     1 0
2656   3:51 flowscan

 2384 mysql     15   0  128m  36m 5220 S   13  3.6 252:44.53  2347 0
91m 252:44 mysqld

29815 root      23   0 30284  26m 1732 R    9  2.7   5:44.42     1 1
2780   5:44 flowscan

Regards,

Ross 

From: Dave Plonka [mailto:plonka at doit.wisc.edu] 
Sent: Thursday, 21 February 2008 3:38 PM
To: flow-tools at list.splintered.net
Cc: flowscan at lists.wiscnet.net; Ross Tsolakidis;
flow-tools at googlegroups.com
Subject: Re: [Flow-tools] cpu utilisation on flowscan processes

Hi Ross,

I'm Cc'ing this to flowscan at lists.wiscnet.net since that is the
flowscan-specific mailing list.

I've replied in context below:

On Thu, Feb 21, 2008 at 02:13:15PM +1100, Ross Tsolakidis wrote:
> Hi all,
> 
> I just needed some clarification on whether this is normal.
> We run flowscan with flow-tools then extract the data from rrd into a
mysql DB for usage figures.
> 
> One thing I'm noticing though is it's very CPU intensive.
<snip>
> Flowscan runs every 5 minutes, using the CUFlow class.
> 
> 2008/02/21 07:20:07 working on file
/var/netflow/ft-v05.2008-02-21.071500+1100...
> 2008/02/21 07:20:10 flowscan-1.020 CUFlow: Cflow::find took 3
wallclock secs ( 3.36 usr + 0.00 sys = 3.36 CPU) for 585896 flow file
bytes, flow hit ratio: 32073/32970
> 2008/02/21 07:20:11 flowscan-1.020 CUFlow: report took 1 wallclock
secs ( 0.00 usr 0.00 sys + 0.11 cusr 0.04 csys = 0.15 CPU)

OK, looks good (more details below).

> The CUFlow.cf has approx 31 Class Cs in it.
> I am analysing every IP, eg, every IP in those Class Cs has it's own
RRD.
> Not sure if this is too much for it to do.

Not at all.

> 2336 ? Ss 0:37 /usr/bin/flow-capture -w /var/netflow/ft 0/0/2055 -S5
-V5 -E1G -n 287 -N 0 -R /usr/local/netflow/bin/linkme
> 3099 ? S 4:20 /usr/bin/perl /usr/bin/flowscan
> 4941 ? R 1:31 /usr/bin/perl /usr/bin/flowscan
> 
> As you can see it???s running 2 sometimes 3 processes of flowscan.
> Is this normal ?

No, it's not normal to have two flowscan processes... probably a
mistake with your rc scripts, i.e. started it twice.
I'd kill off the older one. (Of course look at the PPID first, to
verify they are unrelated.)

> Am I doing this right ? ???
> 
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 4941 root 25 0 21708 18m 1700 R 100 0.9 3:19.58 flowscan
> 2336 root 15 0 2896 1244 504 S 1 0.1 0:37.41 flow-capture
> 
> I missed the 2nd flowscan in top while writing this email,
> but it basically flatlines the cpu 24/7 doing this.

Doing what? Your flowscan log shows that it only worked for 4 seconds
to process 5 minutes of flows.

If you haven't cleared it up after this email, I suggest telling
us the load average on the machine. If there's only <5 seconds of
CPU-intensive flowscan work every five minutes, it should be golden.
If the load average is high, then perhaps there's something unrelated
wrong with your machine, in which case sar will get you far...
(Look at running sadc to collect performance information.)

> Can someone point out anything that I am doing wrong ?

>From what you've shown in the log and ps output, it looks good to me
('cept
the two flowscan processes, but that should at most incorrectly double
the load).

To reiterate, it's only taking a few seconds for the find and
report phases (shown as 3 and 1 wallclock seconds, respecively, in
the flowscan logs above) for you to process 5 minutes of flow data.
So, as soon as everything is caught up (so that it's processing in
real time, flowscan will only be hitting the CPU for about 5 seconds
every five minutes...

As for what is using the CPU inside flowscan, it is the Cflow perl
module (that translates the flow files for your perl script) and the
report code (in your case CUFlow) that uses the CPU - its because it's
perl code... lots of management of data structures and also a lot of
converting raw flow records (from the flow files) into perl variables.

Dave

-- 
plonka at doit.wisc.edu http://net.doit.wisc.edu/~plonka/ Madison, WI

________________________________

Total Control Panel

Login <https://asp-6.reflexion.net/login?domain=day3.com.au> 

To: ross.tsolakidis at day3.com.au
<https://asp-6.reflexion.net/address-properties?aID=6477361> 

From: plonka at doit.wisc.edu
<https://asp-6.reflexion.net/address-properties?aID=31239355> 

Message Score: 50

High (60): Pass

My Spam Blocking Level: Medium

Medium (75): Pass

	Low (90): Pass

Block
<https://asp-6.reflexion.net/FooterAction?ver=1&bl-sender-address=1&rID=
6477361&aID=31239355>  this sender / Block
<https://asp-6.reflexion.net/FooterAction?ver=1&ent=1&bl-sender-address=
1&rID=6477361&aID=31239355>  this sender enterprise-wide

Block
<https://asp-6.reflexion.net/FooterAction?ver=1&bl-sender-domain=1&rID=6
477361&aID=31239355>  doit.wisc.edu / Block
<https://asp-6.reflexion.net/FooterAction?ver=1&ent=1&bl-sender-domain=1
&rID=6477361&aID=31239355>  doit.wisc.edu enterprise-wide

This message was delivered because the content filter score did not
exceed your filter level.

DISCLAIMER: 
This e-mail and any files transmitted with it may be privileged and confidential, and are intended only for the use of the intended recipient. If you are not the intended recipient or responsible for delivering this e-mail to the intended recipient, any use, dissemination, forwarding, printing or copying of this e-mail and any attachments is strictly prohibited. 
If you have received this e-mail in error, please REPLY TO the SENDER to advise the error AND then DELETE the e-mail from your system.
Any views expressed in this e-mail and any files transmitted with it are those of the individual sender, except where the sender specifically states them to be the views of our organisation.
Our organisation does not represent or warrant that the attached files are free from computer viruses or other defects. 
21/2/2008
The user assumes all responsibility for any loss or damage resulting directly or indirectly from the use of the attached files. In any event, the liability to our organisation is limited to either the resupply of the attached files or the cost of having the attached files resupplied.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.splintered.net/pipermail/flow-tools/attachments/20080221/d1b9a33c/attachment-0001.htm