<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.5pt;
font-family:Consolas;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:Consolas;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-GB link=blue vlink=purple>
<div class=Section1>
<p class=MsoPlainText>Hello<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>I’m running softflowd + flow-capture on a SuSE
linux machine (Intel Celeron 3.2GHz CPU, 1GB RAM), with softflowd exporting to
flow-capture locally via the loopback interface. Based on the /var/log/messages
(see below), it seems that flow-capture drops (?) quite a few of the flows. The
CPU and the memory are not maxed out and the machine is not running anything
else. <o:p></o:p></p>
<p class=MsoPlainText>I've tried various fixes, including renice both
flow-capture and softflowd. I also noticed that, for some reason, softflowd
seems to be exporting data (most of the times) at one second past the minute -
I'm not sure whether this is normal (or whether it can be changed), but it is
likely to be putting some additional burden on the export process, as the
activity is very bursty.<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>Trying to capture the loopback traffic with tcpdump is
even more worrying: at each one second past the minute tcpdump writes 32
packets than does nothing until the next minute, but it reports in the end
thousands of lost packets (I've noticed that the interface type is reported
EN10MB, but I'm not sure whether that is relevant or where I could change it)<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>---<o:p></o:p></p>
<p class=MsoPlainText>suse:/ # tcpdump -i lo -n -c 100 -w tmp.dump<o:p></o:p></p>
<p class=MsoPlainText>tcpdump: listening on lo, link-type EN10MB (Ethernet),
capture size 96 bytes 100 packets captured<o:p></o:p></p>
<p class=MsoPlainText>10759 packets received by filter<o:p></o:p></p>
<p class=MsoPlainText>10396 packets dropped by kernel<o:p></o:p></p>
<p class=MsoPlainText>---<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>So, any ideas why flow-capture seems to be dropping
some/quite a bit of the data?<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>Best regards<o:p></o:p></p>
<p class=MsoPlainText>Bogdan<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText>[…]<o:p></o:p></p>
<p class=MsoPlainText>14:26:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406638931
received=1406644520 lost=5589<o:p></o:p></p>
<p class=MsoPlainText>14:27:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406649084 received=1406663463
lost=14379<o:p></o:p></p>
<p class=MsoPlainText>14:27:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406667986
received=1406676860 lost=8874<o:p></o:p></p>
<p class=MsoPlainText>14:27:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406681381
received=1406690419 lost=9038<o:p></o:p></p>
<p class=MsoPlainText>14:28:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406694939
received=1406697126 lost=2187<o:p></o:p></p>
<p class=MsoPlainText>14:28:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406701639
received=1406704768 lost=3129<o:p></o:p></p>
<p class=MsoPlainText>14:28:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406705742
received=1406732343 lost=26601<o:p></o:p></p>
<p class=MsoPlainText>14:29:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406733293
received=1406745465 lost=12172<o:p></o:p></p>
<p class=MsoPlainText>14:29:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406754215
received=1406776762 lost=22547<o:p></o:p></p>
<p class=MsoPlainText>14:29:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406781291
received=1406792840 lost=11549 14:30:00 netflow flow-capture[27294]: STAT:
now=1223299800 startup=1202406747 src_ip=127.0.0.1 dst_ip=127.0.0.1 d_ver=5
pkts=63516629 flows=1796718087 lost=4330166 reset=335275 filter_drops=0<o:p></o:p></p>
<p class=MsoPlainText>14:30:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406802122
received=1406820941 lost=18819<o:p></o:p></p>
<p class=MsoPlainText>14:31:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406823965
received=1406847591 lost=23626<o:p></o:p></p>
<p class=MsoPlainText>14:31:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406852155
received=1406870846 lost=18691<o:p></o:p></p>
<p class=MsoPlainText>14:31:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406875371
received=1406886191 lost=10820<o:p></o:p></p>
<p class=MsoPlainText>14:31:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406890711
received=1406892804 lost=2093<o:p></o:p></p>
<p class=MsoPlainText>14:32:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406897356
received=1406900343 lost=2987<o:p></o:p></p>
<p class=MsoPlainText>14:32:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406904873
received=1406920897 lost=16024<o:p></o:p></p>
<p class=MsoPlainText>14:32:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406925416
received=1406938234 lost=12818<o:p></o:p></p>
<p class=MsoPlainText>14:33:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406944516
received=1406953467 lost=8951<o:p></o:p></p>
<p class=MsoPlainText>14:33:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406957995
received=1406971996 lost=14001<o:p></o:p></p>
<p class=MsoPlainText>14:34:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406976524
received=1406988011 lost=11487<o:p></o:p></p>
<p class=MsoPlainText>14:34:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1406993540
received=1407018568 lost=25028<o:p></o:p></p>
<p class=MsoPlainText>14:34:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1407019663
received=1407036331 lost=16668 14:35:00 netflow flow-capture[27294]: STAT:
now=1223300100 startup=1202406747 src_ip=127.0.0.1 dst_ip=127.0.0.1 d_ver=5
pkts=63518578 flows=1796775650 lost=4330166 reset=335288 filter_drops=0<o:p></o:p></p>
<p class=MsoPlainText>14:35:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1407040855
received=1407051221 lost=10366<o:p></o:p></p>
<p class=MsoPlainText>14:35:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1407051929
received=1407078617 lost=26688<o:p></o:p></p>
<p class=MsoPlainText>14:36:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1407079557
received=1407085158 lost=5601<o:p></o:p></p>
<p class=MsoPlainText>14:36:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1407089689
received=1407109716 lost=20027<o:p></o:p></p>
<p class=MsoPlainText>14:36:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1407110132
received=1407134873 lost=24741<o:p></o:p></p>
<p class=MsoPlainText>14:37:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1407139405
received=1407145876 lost=6471<o:p></o:p></p>
<p class=MsoPlainText>14:37:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1407146290
received=1407173718 lost=27428<o:p></o:p></p>
<p class=MsoPlainText>14:38:01 netflow flow-capture[27294]: ftpdu_seq_check():
src_ip=127.0.0.1 dst_ip=127.0.0.1 d_version=5 expecting=1407174842
received=1407178461 lost=3619 […]<o:p></o:p></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>