From splintered-flow-tools-owner@splintered.net Mon Dec 01 18:01:53 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 76853 invoked by uid 4001); 1 Dec 2003 18:01:53 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 76851 invoked by alias); 1 Dec 2003 18:01:53 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 1 Dec 2003 18:01:53 -0000 Mime-Version: 1.0 (Apple Message framework v606) Content-Transfer-Encoding: 7bit Message-Id: <7174894C-2428-11D8-BC39-000A95DA1C38@eng.oar.net> Content-Type: text/plain; charset=US-ASCII; format=flowed To: flow-tools@splintered.net From: Mark Fullmer Date: Mon, 1 Dec 2003 13:01:52 -0500 X-Mailer: Apple Mail (2.606) Cc: Subject: [Flow-tools] New list host X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Dec 2003 18:01:53 -0000 I've moved the flow-tools mailing list to a new host, hopefully this will fix the dropped mail problems I've had with pair.com's mailing list service. The new list home is http://mailman.splintered.net/mailman/listinfo/flow-tools In the process any accounts which were disabled due to excessive bounces have been re-enabled. You may need to set your digest flag again... mark From splintered-flow-tools-owner@splintered.net Tue Dec 02 00:35:43 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 79445 invoked by uid 4001); 2 Dec 2003 00:35:43 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 79442 invoked by alias); 2 Dec 2003 00:35:43 -0000 Received: from dhcp9578217.columbus.rr.com (HELO ?10.0.0.25?) (24.95.78.217) by 66.250.216.131 with SMTP; 2 Dec 2003 00:35:43 -0000 In-Reply-To: <00e001c3b817$23798170$61af0a0a@Accenture.com> References: <00e701c3b4f4$319787a0$61af0a0a@Accenture.com> <20031127202819.GE91301@elvis.mu.org> <002001c3b5b2$2e7f80e0$61af0a0a@Accenture.com> <43B31182-220A-11D8-9AA7-000A95DA1C38@splintered.net> <002801c3b7f3$7d645850$61af0a0a@Accenture.com> <1D50FC29-2400-11D8-BC39-000A95DA1C38@splintered.net> <00e001c3b817$23798170$61af0a0a@Accenture.com> Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <753B3D6F-245F-11D8-BC39-000A95DA1C38@splintered.net> Content-Transfer-Encoding: 7bit From: Mark Fullmer Subject: Re: [flow-tools] format flow description Date: Mon, 1 Dec 2003 19:35:41 -0500 To: "Orlando Onorato" X-Mailer: Apple Mail (2.606) Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 00:35:43 -0000 Flow-tools does not store these fields, they're only necessary to get the data from the exporter to the collector. The 'pad' field is there to provide alignment. You could look in lib/ftdecode.c or src/fdg.c for some examples on how to find these fields mark On Dec 1, 2003, at 9:17 AM, Orlando Onorato wrote: > In particular I need of these header fields: > ushort version, ushort count, ulong flow_sequence. > > Why the field "uchar pad" isn't there in the flow record? > > Thanks for your patience! > > bye > > > ----- Original Message ----- > From: "Mark Fullmer" > To: "Orlando Onorato" > Cc: > Sent: Monday, December 01, 2003 2:13 PM > Subject: Re: [flow-tools] format flow description > > >> Which header are you trying to look at? >> >> mark >> >> On Dec 1, 2003, at 5:09 AM, Orlando Onorato wrote: >> >>> Thank you very much. I've solved my 2nd problem. >>> >>> By mean the flow-export I'm not able to view the header yet, >>> and I don't want to use the tcpdump. >>> >>> Can you help me? >>> >>> >>> ----- Original Message ----- >>> From: "Mark Fullmer" >>> To: "Orlando Onorato" >>> Cc: >>> Sent: Saturday, November 29, 2003 2:20 AM >>> Subject: Re: [flow-tools] format flow description >>> >>> >>>> It's there. Use flow-export -f2 (ASCII). >>>> >>>> #: >>>> unix_secs,unix_nsecs,sysuptime,exaddr,dpkts,doctets,first,last,engin >>>> e_ >>>> ty >>>> pe,eng >>>> ine_id,srcaddr,dstaddr,nexthop,input,output,srcport,dstport,prot,tos >>>> ,t >>>> cp >>>> _flags,s >>>> rc_mask,dst_mask,src_as,dst_as >>>> 1018404899,502659552,3685185404,192.88.192.245,1,40,3685169372,36851 >>>> 69 >>>> 37 >>>> 2,4,0,12 >>>> 9.137.254.5,129.21.7.49,192.88.192.33,40,25,56279,33508,17,0,16,16,1 >>>> 6, >>>> 20 >>>> 126,4385 >>>> ^^^^^^^^^^^^^ >>>> >>>> mark >>>> >>>> >>>> On Nov 28, 2003, at 8:18 AM, Orlando Onorato wrote: >>>> >>>>> By means of flow-export I'm not able to view the Nexhop field, >>>>> although this filed there is in Netflow ver.5. >>>>> >>>>> >>>>> ----- Original Message ----- >>>>> From: "Bill Fumerola" >>>>> To: "Orlando Onorato" >>>>> Cc: >>>>> Sent: Thursday, November 27, 2003 9:28 PM >>>>> Subject: Re: [flow-tools] format flow description >>>>> >>>>> >>>>>> On Thu, Nov 27, 2003 at 03:38:57PM +0100, Orlando Onorato wrote: >>>>>> >>>>>>> 1) View header description of datagramm generated. >>>>>> >>>>>> dunno what you mean here, but increasing the debug level will show >>>>>> the >>>>>> header of the flow file. if you want to see ip or udp header >>>>>> information >>>>>> from the actual netflow packet, i'd suggest tcpdump. >>>>>> >>>>>>> 2) View all fields of flow description (e.g. nexthop field. >>>>>> >>>>>> flow-export >>>>>> >>>>>> -- >>>>>> - bill fumerola / fumerola@yahoo-inc.com / billf@FreeBSD.org >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> flow-tools@splintered.net >>>>> http://www.splintered.net/sw/flow-tools >>>>> >>>> >>>> >>>> _______________________________________________ >>>> flow-tools@splintered.net >>>> http://www.splintered.net/sw/flow-tools >> >> >> _______________________________________________ >> flow-tools@splintered.net >> http://www.splintered.net/sw/flow-tools > From splintered-flow-tools-owner@splintered.net Tue Dec 02 02:43:40 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 80607 invoked by uid 4001); 2 Dec 2003 02:43:40 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 80603 invoked by alias); 2 Dec 2003 02:43:39 -0000 Received: from mail.sunet.com.au (HELO jupiter.sunet.com.au) (203.166.102.39) by 66.250.216.131 with SMTP; 2 Dec 2003 02:43:39 -0000 Received: from ganymede.internal.sunet.com.au (canopus.sunet.com.au [::ffff:203.166.102.49]) by jupiter.sunet.com.au with esmtp; Tue, 02 Dec 2003 13:38:15 +1100 Date: Tue, 2 Dec 2003 13:43:59 +1100 (EST) From: Systems Administrator X-X-Sender: sysadmin@ganymede.bcc.local To: Mark Fullmer Subject: Re: [flow-tools] flow-nfilter end-time comparisons In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 02:43:40 -0000 On Mon, 1 Dec 2003, Mark Fullmer wrote: > Yep. It's a one line fix. Great! I'd been wanting that for a year or so :). But if I needed it *that* badly, I would've done it myself :). I presume this is making it into the next version of flow-tools? (is it 0.67 next?) Thanks, -- Tim Nelson Systems Administrator Sunet Internet Tel: +61 3 5241 1155 Fax: +61 3 5241 6187 Web: http://www.sunet.com.au/ Email: sysadmin@sunet.com.au From foulonneau@net-outremer.nc Tue Dec 02 06:35:23 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 82308 invoked by alias); 2 Dec 2003 06:35:22 -0000 Received: from gw-smtp.net-outremer.nc (HELO gw-smtp.canl.nc) (202.171.64.2) by 66.250.216.131 with SMTP; 2 Dec 2003 06:35:22 -0000 Received: from mail1.canl.nc ([202.87.159.22]) by gw-smtp.canl.nc with esmtp (Exim 4.14) id 1AR480-0003XJ-Fp for flow-tools@list.splintered.net; Tue, 02 Dec 2003 17:35:20 +1100 Received: from [192.168.0.66] (helo=callispe.net-outremer.nc) by mail1.canl.nc with asmtp (Exim 4.24) id 1AR47z-0002DG-Ul for flow-tools@list.splintered.net; Tue, 02 Dec 2003 17:35:19 +1100 Message-Id: <5.1.0.14.2.20031202173526.02b82ac0@mail.canl.nc> X-Sender: Foulonneau@net-outremer.nc@mail.canl.nc X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Tue, 02 Dec 2003 17:35:33 +1100 To: flow-tools@list.splintered.net From: Laurent Foulonneau Subject: [flow-tools] some php scripts Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 06:35:23 -0000 Great ! it works fine for me... Thank you From splintered-flow-tools-owner@splintered.net Tue Dec 02 08:29:23 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 83586 invoked by uid 4001); 2 Dec 2003 08:29:23 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 83583 invoked by alias); 2 Dec 2003 08:29:23 -0000 Received: from hal-5.inet.it (213.92.5.24) by 66.250.216.131 with SMTP; 2 Dec 2003 08:29:23 -0000 Received: from 194-177-126-137.f4.ngi.it [::ffff:194.177.126.137] by hal-5.inet.it via I-SMTP-4.7.0-470 id ::ffff:194.177.126.137+wrYt1WvYG; Tue, 02 Dec 2003 09:29:21 +0100 Message-ID: <3FCC4AAD.7070700@tekNico.net> Date: Tue, 02 Dec 2003 09:17:49 +0100 From: Nicola Larosa User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5) Gecko/20031105 Thunderbird/0.3 X-Accept-Language: en-us, en MIME-Version: 1.0 To: flow-tools@splintered.net Subject: Re: [flow-tools] flow-nfilter end-time comparisons References: In-Reply-To: X-Enigmail-Version: 0.81.6.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 08:29:23 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > I presume this is making it into the next version of flow-tools? (is it > 0.67 next?) Speaking of which, what's the timeline for the release? I have to find the time to report patches and suggestions for three items related to flow-capture. They're admittedly minor, but it would be nice if they had a chance to be at least considered for the next release, since they seem to appear not that often. :^) - -- "Workaholics and others who can't tear themselves away from the mouse and keyboard need to keep their legs active and get away from the computer for some exercise every now and then. So, please, get up and stretch every so often. You certainly don't want to wind up eDead from eThrombosis." -- Jenny Thompson, HSI Nicola Larosa - nico@tekNico.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQE/zEqsXv0hgDImBm4RAgW8AJ4+8KXQJjObE2kTZfxHpXeli/5EUwCguVss 9oYZR80KQRQkwFAcPXYqiNA= =7MqC -----END PGP SIGNATURE----- From splintered-flow-tools-owner@splintered.net Tue Dec 02 14:38:57 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 85355 invoked by uid 4001); 2 Dec 2003 14:38:57 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 85351 invoked by alias); 2 Dec 2003 14:38:57 -0000 Received: from smtp0.libero.it (193.70.192.33) by 66.250.216.131 with SMTP; 2 Dec 2003 14:38:57 -0000 Received: from H7J31KT8Z604D (151.24.171.246) by smtp0.libero.it (7.0.020-DD01) id 3F6F1CE701678CC9; Tue, 2 Dec 2003 15:38:54 +0100 Message-ID: <00eb01c3b8e1$a503d4f0$61af0a0a@Accenture.com> From: "Orlando Onorato" To: "Mark Fullmer" , Subject: Re: [flow-tools] format flow description Date: Tue, 2 Dec 2003 15:36:10 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Cc: X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 14:38:57 -0000 Ok! I want understand if flow_gen produce this fields (ushort version, ushort count, ulong flow_sequence). I believe yes! Then, for obtain this fields in ASCII CSV format I must: 1) Add this fields in struct fts3rec_v5_gen or struct fts3rec_v5 ? 2) Modify the function ftio_read(). Is it ok? If you can to advice some specific code solution, I will be very happy! Thanks a lot. ----- Original Message ----- From: "Orlando Onorato" Sent: Tuesday, December 02, 2003 2:14 PM Subject: Fw: [flow-tools] format flow description > > ----- Original Message ----- > From: "Mark Fullmer" > To: "Orlando Onorato" > Cc: > Sent: Tuesday, December 02, 2003 1:35 AM > Subject: Re: [flow-tools] format flow description > > > > > > Flow-tools does not store these fields, they're only necessary > > to get the data from the exporter to the collector. > > > > The 'pad' field is there to provide alignment. > > > > You could look in lib/ftdecode.c or src/fdg.c for some examples on > > how to find these fields > > > > mark > > > > On Dec 1, 2003, at 9:17 AM, Orlando Onorato wrote: > > > > > In particular I need of these header fields: > > > ushort version, ushort count, ulong flow_sequence. > > > > > > Why the field "uchar pad" isn't there in the flow record? > > > > > > Thanks for your patience! > > > > > > bye > > > > > > > > > ----- Original Message ----- > > > From: "Mark Fullmer" > > > To: "Orlando Onorato" > > > Cc: > > > Sent: Monday, December 01, 2003 2:13 PM > > > Subject: Re: [flow-tools] format flow description > > > > > > > > >> Which header are you trying to look at? > > >> > > >> mark > > >> > > >> On Dec 1, 2003, at 5:09 AM, Orlando Onorato wrote: > > >> > > >>> Thank you very much. I've solved my 2nd problem. > > >>> > > >>> By mean the flow-export I'm not able to view the header yet, > > >>> and I don't want to use the tcpdump. > > >>> > > >>> Can you help me? > > >>> > > >>> > > >>> ----- Original Message ----- > > >>> From: "Mark Fullmer" > > >>> To: "Orlando Onorato" > > >>> Cc: > > >>> Sent: Saturday, November 29, 2003 2:20 AM > > >>> Subject: Re: [flow-tools] format flow description > > >>> > > >>> > > >>>> It's there. Use flow-export -f2 (ASCII). > > >>>> > > >>>> #: > > >>>> unix_secs,unix_nsecs,sysuptime,exaddr,dpkts,doctets,first,last,engin > > >>>> e_ > > >>>> ty > > >>>> pe,eng > > >>>> ine_id,srcaddr,dstaddr,nexthop,input,output,srcport,dstport,prot,tos > > >>>> ,t > > >>>> cp > > >>>> _flags,s > > >>>> rc_mask,dst_mask,src_as,dst_as > > >>>> 1018404899,502659552,3685185404,192.88.192.245,1,40,3685169372,36851 > > >>>> 69 > > >>>> 37 > > >>>> 2,4,0,12 > > >>>> 9.137.254.5,129.21.7.49,192.88.192.33,40,25,56279,33508,17,0,16,16,1 > > >>>> 6, > > >>>> 20 > > >>>> 126,4385 > > >>>> ^^^^^^^^^^^^^ > > >>>> > > >>>> mark > > >>>> > > >>>> > > >>>> On Nov 28, 2003, at 8:18 AM, Orlando Onorato wrote: > > >>>> > > >>>>> By means of flow-export I'm not able to view the Nexhop field, > > >>>>> although this filed there is in Netflow ver.5. > > >>>>> > > >>>>> > > >>>>> ----- Original Message ----- > > >>>>> From: "Bill Fumerola" > > >>>>> To: "Orlando Onorato" > > >>>>> Cc: > > >>>>> Sent: Thursday, November 27, 2003 9:28 PM > > >>>>> Subject: Re: [flow-tools] format flow description > > >>>>> > > >>>>> > > >>>>>> On Thu, Nov 27, 2003 at 03:38:57PM +0100, Orlando Onorato wrote: > > >>>>>> > > >>>>>>> 1) View header description of datagramm generated. > > >>>>>> > > >>>>>> dunno what you mean here, but increasing the debug level will show > > >>>>>> the > > >>>>>> header of the flow file. if you want to see ip or udp header > > >>>>>> information > > >>>>>> from the actual netflow packet, i'd suggest tcpdump. > > >>>>>> > > >>>>>>> 2) View all fields of flow description (e.g. nexthop field. > > >>>>>> > > >>>>>> flow-export > > >>>>>> > > >>>>>> -- > > >>>>>> - bill fumerola / fumerola@yahoo-inc.com / billf@FreeBSD.org > > >>>>>> > > >>>>>> > > >>>>> > > >>>>> _______________________________________________ > > >>>>> flow-tools@splintered.net > > >>>>> http://www.splintered.net/sw/flow-tools > > >>>>> > > >>>> > > >>>> > > >>>> _______________________________________________ > > >>>> flow-tools@splintered.net > > >>>> http://www.splintered.net/sw/flow-tools > > >> > > >> > > >> _______________________________________________ > > >> flow-tools@splintered.net > > >> http://www.splintered.net/sw/flow-tools > > > > > > From splintered-flow-tools-owner@splintered.net Tue Dec 02 15:33:33 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 86317 invoked by uid 4001); 2 Dec 2003 15:33:33 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 86314 invoked by alias); 2 Dec 2003 15:33:32 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 2 Dec 2003 15:33:32 -0000 In-Reply-To: <3FCC4AAD.7070700@tekNico.net> References: <3FCC4AAD.7070700@tekNico.net> Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Mark Fullmer Subject: Re: [flow-tools] flow-nfilter end-time comparisons Date: Tue, 2 Dec 2003 10:33:32 -0500 To: Nicola Larosa X-Mailer: Apple Mail (2.606) Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 15:33:33 -0000 0.67 will be released in the next few days. Yes, the last snapshot was over 6 months ago. I had to take a break from ft development to work on other projects. mark On Dec 2, 2003, at 3:17 AM, Nicola Larosa wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > >> I presume this is making it into the next version of flow-tools? (is >> it >> 0.67 next?) > > Speaking of which, what's the timeline for the release? I have to find > the > time to report patches and suggestions for three items related to > flow-capture. > > They're admittedly minor, but it would be nice if they had a chance to > be at > least considered for the next release, since they seem to appear not > that > often. :^) > > > - -- > "Workaholics and others who can't tear themselves away from the mouse > and > keyboard need to keep their legs active and get away from the computer > for > some exercise every now and then. So, please, get up and stretch every > so > often. You certainly don't want to wind up eDead from eThrombosis." > -- Jenny Thompson, HSI > > Nicola Larosa - nico@tekNico.net > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.2.3 (GNU/Linux) > > iD8DBQE/zEqsXv0hgDImBm4RAgW8AJ4+8KXQJjObE2kTZfxHpXeli/5EUwCguVss > 9oYZR80KQRQkwFAcPXYqiNA= > =7MqC > -----END PGP SIGNATURE----- > > > _______________________________________________ > Flow-tools mailing list > flow-tools@splintered.net > http://mailman.splintered.net/mailman/listinfo/flow-tools From splintered-flow-tools-owner@splintered.net Tue Dec 02 15:37:50 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 87056 invoked by uid 4001); 2 Dec 2003 15:37:50 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 87053 invoked by alias); 2 Dec 2003 15:37:50 -0000 Received: from dhcp9578217.columbus.rr.com (HELO ?10.0.0.25?) (24.95.78.217) by 66.250.216.131 with SMTP; 2 Dec 2003 15:37:50 -0000 In-Reply-To: <00eb01c3b8e1$a503d4f0$61af0a0a@Accenture.com> References: <00eb01c3b8e1$a503d4f0$61af0a0a@Accenture.com> Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <7C0EBFEA-24DD-11D8-87CF-000A95DA1C38@splintered.net> Content-Transfer-Encoding: 7bit From: Mark Fullmer Subject: Re: [flow-tools] format flow description Date: Tue, 2 Dec 2003 10:37:49 -0500 To: "Orlando Onorato" X-Mailer: Apple Mail (2.606) Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 15:37:50 -0000 These fields are only necessary when sending data in Cisco NetFlow format over a network. Modifying flow-tools to store them is probably not the correct approach. What exactly are you trying to do? The "wire" option to flow-export will produce output that looks like what would be sent over the wire. This can then be stored to a file. flow-send will transmit flow-tools data in Cisco NetFlow format over a network. Both the above two options will create the fields you're looking for. mark On Dec 2, 2003, at 9:36 AM, Orlando Onorato wrote: > Ok! > I want understand if flow_gen produce this fields (ushort version, > ushort > count, ulong flow_sequence). I believe yes! > > Then, for obtain this fields in ASCII CSV format I must: > 1) Add this fields in struct fts3rec_v5_gen or struct fts3rec_v5 ? > 2) Modify the function ftio_read(). > > Is it ok? If you can to advice some specific code solution, I will be > very > happy! > > Thanks a lot. > > ----- Original Message ----- > From: "Orlando Onorato" > Sent: Tuesday, December 02, 2003 2:14 PM > Subject: Fw: [flow-tools] format flow description > > >> >> ----- Original Message ----- >> From: "Mark Fullmer" >> To: "Orlando Onorato" >> Cc: >> Sent: Tuesday, December 02, 2003 1:35 AM >> Subject: Re: [flow-tools] format flow description >> >> >>> >>> Flow-tools does not store these fields, they're only necessary >>> to get the data from the exporter to the collector. >>> >>> The 'pad' field is there to provide alignment. >>> >>> You could look in lib/ftdecode.c or src/fdg.c for some examples on >>> how to find these fields >>> >>> mark >>> >>> On Dec 1, 2003, at 9:17 AM, Orlando Onorato wrote: >>> >>>> In particular I need of these header fields: >>>> ushort version, ushort count, ulong flow_sequence. >>>> >>>> Why the field "uchar pad" isn't there in the flow record? >>>> >>>> Thanks for your patience! >>>> >>>> bye >>>> >>>> >>>> ----- Original Message ----- >>>> From: "Mark Fullmer" >>>> To: "Orlando Onorato" >>>> Cc: >>>> Sent: Monday, December 01, 2003 2:13 PM >>>> Subject: Re: [flow-tools] format flow description >>>> >>>> >>>>> Which header are you trying to look at? >>>>> >>>>> mark >>>>> >>>>> On Dec 1, 2003, at 5:09 AM, Orlando Onorato wrote: >>>>> >>>>>> Thank you very much. I've solved my 2nd problem. >>>>>> >>>>>> By mean the flow-export I'm not able to view the header yet, >>>>>> and I don't want to use the tcpdump. >>>>>> >>>>>> Can you help me? >>>>>> >>>>>> >>>>>> ----- Original Message ----- >>>>>> From: "Mark Fullmer" >>>>>> To: "Orlando Onorato" >>>>>> Cc: >>>>>> Sent: Saturday, November 29, 2003 2:20 AM >>>>>> Subject: Re: [flow-tools] format flow description >>>>>> >>>>>> >>>>>>> It's there. Use flow-export -f2 (ASCII). >>>>>>> >>>>>>> #: >>>>>>> > unix_secs,unix_nsecs,sysuptime,exaddr,dpkts,doctets,first,last,engin >>>>>>> e_ >>>>>>> ty >>>>>>> pe,eng >>>>>>> > ine_id,srcaddr,dstaddr,nexthop,input,output,srcport,dstport,prot,tos >>>>>>> ,t >>>>>>> cp >>>>>>> _flags,s >>>>>>> rc_mask,dst_mask,src_as,dst_as >>>>>>> > 1018404899,502659552,3685185404,192.88.192.245,1,40,3685169372,36851 >>>>>>> 69 >>>>>>> 37 >>>>>>> 2,4,0,12 >>>>>>> > 9.137.254.5,129.21.7.49,192.88.192.33,40,25,56279,33508,17,0,16,16,1 >>>>>>> 6, >>>>>>> 20 >>>>>>> 126,4385 >>>>>>> ^^^^^^^^^^^^^ >>>>>>> >>>>>>> mark >>>>>>> >>>>>>> >>>>>>> On Nov 28, 2003, at 8:18 AM, Orlando Onorato wrote: >>>>>>> >>>>>>>> By means of flow-export I'm not able to view the Nexhop field, >>>>>>>> although this filed there is in Netflow ver.5. >>>>>>>> >>>>>>>> >>>>>>>> ----- Original Message ----- >>>>>>>> From: "Bill Fumerola" >>>>>>>> To: "Orlando Onorato" >>>>>>>> Cc: >>>>>>>> Sent: Thursday, November 27, 2003 9:28 PM >>>>>>>> Subject: Re: [flow-tools] format flow description >>>>>>>> >>>>>>>> >>>>>>>>> On Thu, Nov 27, 2003 at 03:38:57PM +0100, Orlando Onorato >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> 1) View header description of datagramm generated. >>>>>>>>> >>>>>>>>> dunno what you mean here, but increasing the debug level will > show >>>>>>>>> the >>>>>>>>> header of the flow file. if you want to see ip or udp header >>>>>>>>> information >>>>>>>>> from the actual netflow packet, i'd suggest tcpdump. >>>>>>>>> >>>>>>>>>> 2) View all fields of flow description (e.g. nexthop field. >>>>>>>>> >>>>>>>>> flow-export >>>>>>>>> >>>>>>>>> -- >>>>>>>>> - bill fumerola / fumerola@yahoo-inc.com / billf@FreeBSD.org >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> flow-tools@splintered.net >>>>>>>> http://www.splintered.net/sw/flow-tools >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> flow-tools@splintered.net >>>>>>> http://www.splintered.net/sw/flow-tools >>>>> >>>>> >>>>> _______________________________________________ >>>>> flow-tools@splintered.net >>>>> http://www.splintered.net/sw/flow-tools >>>> >>> >> > From splintered-flow-tools-owner@splintered.net Tue Dec 02 15:58:24 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 87913 invoked by uid 4001); 2 Dec 2003 15:58:24 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 87909 invoked by alias); 2 Dec 2003 15:58:24 -0000 Received: from smtp1.libero.it (193.70.192.51) by 66.250.216.131 with SMTP; 2 Dec 2003 15:58:24 -0000 Received: from H7J31KT8Z604D (151.24.221.152) by smtp1.libero.it (7.0.020-DD01) id 3F6F0E4801683E3F; Tue, 2 Dec 2003 16:58:56 +0100 Message-ID: <00f801c3b8ec$bf0a1ed0$61af0a0a@Accenture.com> From: "Orlando Onorato" To: "Mark Fullmer" , Subject: Fw: [flow-tools] format flow description Date: Tue, 2 Dec 2003 16:55:43 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Cc: X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 15:58:24 -0000 Ok! Perhaps is arrive to speak one's mind! #;-) This is my goal: I receive real-time traffic from Router CiscoNetFlow then I store it into files for future, not real-time, sent to CiscoNetFlow Reader, over network. Since CiscoNetFlow Reader aims some size-format control I must be able to verified the correctness of results. An example CiscoNetFlow Reader aims the control of flow_sequence for reject flow not corret, then I must view this field. If you need about other details, please write me again. You are very kind #;-) ----- Original Message ----- From: "Mark Fullmer" To: "Orlando Onorato" Cc: Sent: Tuesday, December 02, 2003 4:37 PM Subject: Re: [flow-tools] format flow description > > These fields are only necessary when sending data in Cisco NetFlow > format > over a network. Modifying flow-tools to store them is probably not the > correct approach. > > What exactly are you trying to do? > > The "wire" option to flow-export will produce output that looks like > what > would be sent over the wire. This can then be stored to a file. > > flow-send will transmit flow-tools data in Cisco NetFlow format over a > network. > > Both the above two options will create the fields you're looking for. > > mark > > > On Dec 2, 2003, at 9:36 AM, Orlando Onorato wrote: > > > Ok! > > I want understand if flow_gen produce this fields (ushort version, > > ushort > > count, ulong flow_sequence). I believe yes! > > > > Then, for obtain this fields in ASCII CSV format I must: > > 1) Add this fields in struct fts3rec_v5_gen or struct fts3rec_v5 ? > > 2) Modify the function ftio_read(). > > > > Is it ok? If you can to advice some specific code solution, I will be > > very > > happy! > > > > Thanks a lot. > > > > ----- Original Message ----- > > From: "Orlando Onorato" > > Sent: Tuesday, December 02, 2003 2:14 PM > > Subject: Fw: [flow-tools] format flow description > > > > > >> > >> ----- Original Message ----- > >> From: "Mark Fullmer" > >> To: "Orlando Onorato" > >> Cc: > >> Sent: Tuesday, December 02, 2003 1:35 AM > >> Subject: Re: [flow-tools] format flow description > >> > >> > >>> > >>> Flow-tools does not store these fields, they're only necessary > >>> to get the data from the exporter to the collector. > >>> > >>> The 'pad' field is there to provide alignment. > >>> > >>> You could look in lib/ftdecode.c or src/fdg.c for some examples on > >>> how to find these fields > >>> > >>> mark > >>> > >>> On Dec 1, 2003, at 9:17 AM, Orlando Onorato wrote: > >>> > >>>> In particular I need of these header fields: > >>>> ushort version, ushort count, ulong flow_sequence. > >>>> > >>>> Why the field "uchar pad" isn't there in the flow record? > >>>> > >>>> Thanks for your patience! > >>>> > >>>> bye > >>>> > >>>> > >>>> ----- Original Message ----- > >>>> From: "Mark Fullmer" > >>>> To: "Orlando Onorato" > >>>> Cc: > >>>> Sent: Monday, December 01, 2003 2:13 PM > >>>> Subject: Re: [flow-tools] format flow description > >>>> > >>>> > >>>>> Which header are you trying to look at? > >>>>> > >>>>> mark > >>>>> > >>>>> On Dec 1, 2003, at 5:09 AM, Orlando Onorato wrote: > >>>>> > >>>>>> Thank you very much. I've solved my 2nd problem. > >>>>>> > >>>>>> By mean the flow-export I'm not able to view the header yet, > >>>>>> and I don't want to use the tcpdump. > >>>>>> > >>>>>> Can you help me? > >>>>>> > >>>>>> > >>>>>> ----- Original Message ----- > >>>>>> From: "Mark Fullmer" > >>>>>> To: "Orlando Onorato" > >>>>>> Cc: > >>>>>> Sent: Saturday, November 29, 2003 2:20 AM > >>>>>> Subject: Re: [flow-tools] format flow description > >>>>>> > >>>>>> > >>>>>>> It's there. Use flow-export -f2 (ASCII). > >>>>>>> > >>>>>>> #: > >>>>>>> > > unix_secs,unix_nsecs,sysuptime,exaddr,dpkts,doctets,first,last,engin > >>>>>>> e_ > >>>>>>> ty > >>>>>>> pe,eng > >>>>>>> > > ine_id,srcaddr,dstaddr,nexthop,input,output,srcport,dstport,prot,tos > >>>>>>> ,t > >>>>>>> cp > >>>>>>> _flags,s > >>>>>>> rc_mask,dst_mask,src_as,dst_as > >>>>>>> > > 1018404899,502659552,3685185404,192.88.192.245,1,40,3685169372,36851 > >>>>>>> 69 > >>>>>>> 37 > >>>>>>> 2,4,0,12 > >>>>>>> > > 9.137.254.5,129.21.7.49,192.88.192.33,40,25,56279,33508,17,0,16,16,1 > >>>>>>> 6, > >>>>>>> 20 > >>>>>>> 126,4385 > >>>>>>> ^^^^^^^^^^^^^ > >>>>>>> > >>>>>>> mark > >>>>>>> > >>>>>>> > >>>>>>> On Nov 28, 2003, at 8:18 AM, Orlando Onorato wrote: > >>>>>>> > >>>>>>>> By means of flow-export I'm not able to view the Nexhop field, > >>>>>>>> although this filed there is in Netflow ver.5. > >>>>>>>> > >>>>>>>> > >>>>>>>> ----- Original Message ----- > >>>>>>>> From: "Bill Fumerola" > >>>>>>>> To: "Orlando Onorato" > >>>>>>>> Cc: > >>>>>>>> Sent: Thursday, November 27, 2003 9:28 PM > >>>>>>>> Subject: Re: [flow-tools] format flow description > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Thu, Nov 27, 2003 at 03:38:57PM +0100, Orlando Onorato > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> 1) View header description of datagramm generated. > >>>>>>>>> > >>>>>>>>> dunno what you mean here, but increasing the debug level will > > show > >>>>>>>>> the > >>>>>>>>> header of the flow file. if you want to see ip or udp header > >>>>>>>>> information > >>>>>>>>> from the actual netflow packet, i'd suggest tcpdump. > >>>>>>>>> > >>>>>>>>>> 2) View all fields of flow description (e.g. nexthop field. > >>>>>>>>> > >>>>>>>>> flow-export > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> - bill fumerola / fumerola@yahoo-inc.com / billf@FreeBSD.org > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> _______________________________________________ > >>>>>>>> flow-tools@splintered.net > >>>>>>>> http://www.splintered.net/sw/flow-tools > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> flow-tools@splintered.net > >>>>>>> http://www.splintered.net/sw/flow-tools > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> flow-tools@splintered.net > >>>>> http://www.splintered.net/sw/flow-tools > >>>> > >>> > >> > > > From splintered-Flow-tools-owner@splintered.net Tue Dec 02 19:42:53 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 90080 invoked by uid 4001); 2 Dec 2003 19:42:53 -0000 Delivered-To: splintered-Flow-tools@splintered.net Received: (qmail 90077 invoked by alias); 2 Dec 2003 19:42:53 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 2 Dec 2003 19:42:53 -0000 In-Reply-To: <71E9FA687B04C94EBD1CCEE7DCEDA8EE5EA74B@rodan.motive.com> References: <71E9FA687B04C94EBD1CCEE7DCEDA8EE5EA74B@rodan.motive.com> Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Mark Fullmer Subject: Re: [flow-tools] Networks Date: Tue, 2 Dec 2003 14:42:51 -0500 To: "Samson Martinez" X-Mailer: Apple Mail (2.606) Cc: flowscan@net.doit.wisc.edu, Flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2003 19:42:53 -0000 Redirected to the FlowScan mailing list.... mark On Dec 1, 2003, at 9:16 AM, Samson Martinez wrote: > Hello, > > > I've configured flow-tools with Flowscan & CUFlow and, so far, > everything appears to be working well. However, I'm trying to identify > traffic that is showing up as a substantial part of the reported flows. > Our network, in a nutshell, is as follows: > > 2 Cisco 7204VXRs that sit on our network boundary, both exporting > version 5 flows to a Sun Solaris server. > > I have configured 7 subnets as local as these are networks that sit > behind the 7204s. I then added these same subnets in their subnetted > form to the "Networks Interested In" portion of the configuration. > > All the traffic appears to be properly accounted for but when I show > the > graphs I see 20% In & 41% Out traffic identified as "Other networks". > > What is the best way to isolate and identify those "Other Networks"? > > Many thanks for all your assistance. > > By the way, the same is occurring with the "Services" and "Protocols". > > Regards, > > -Samson Martinez > > > _______________________________________________ > flow-tools@splintered.net > http://www.splintered.net/sw/flow-tools > From splintered-flow-tools-owner@splintered.net Wed Dec 03 16:56:36 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 95043 invoked by uid 4001); 3 Dec 2003 16:56:36 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 95039 invoked by alias); 3 Dec 2003 16:56:35 -0000 Received: from smtp2.libero.it (193.70.192.52) by 66.250.216.131 with SMTP; 3 Dec 2003 16:56:35 -0000 Received: from H7J31KT8Z604D (151.24.216.135) by smtp2.libero.it (7.0.020-DD01) id 3F6F0DA9016D6964; Wed, 3 Dec 2003 17:57:19 +0100 Message-ID: <002401c3b9be$09cc88f0$61af0a0a@Accenture.com> From: "Orlando Onorato" To: "Mark Fullmer" References: <00eb01c3b8e1$a503d4f0$61af0a0a@Accenture.com> <7C0EBFEA-24DD-11D8-87CF-000A95DA1C38@splintered.net> Subject: Re: [flow-tools] format flow description Date: Wed, 3 Dec 2003 17:53:54 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2003 16:56:36 -0000 Hi Mark, probably I've success about my goal - thanks to you #;-) Now I've this problem: whit flow-gen the flow_sequence is ever equals to 0. Since for now I haven't got a real netflow traffic I wold be test my changes... #;-( have you got some real netflow traffic stored into files? Thanks a lot. Orlando ----- Original Message ----- From: "Mark Fullmer" To: "Orlando Onorato" Cc: Sent: Tuesday, December 02, 2003 4:37 PM Subject: Re: [flow-tools] format flow description > > These fields are only necessary when sending data in Cisco NetFlow > format > over a network. Modifying flow-tools to store them is probably not the > correct approach. > > What exactly are you trying to do? > > The "wire" option to flow-export will produce output that looks like > what > would be sent over the wire. This can then be stored to a file. > > flow-send will transmit flow-tools data in Cisco NetFlow format over a > network. > > Both the above two options will create the fields you're looking for. > > mark > > > On Dec 2, 2003, at 9:36 AM, Orlando Onorato wrote: > > > Ok! > > I want understand if flow_gen produce this fields (ushort version, > > ushort > > count, ulong flow_sequence). I believe yes! > > > > Then, for obtain this fields in ASCII CSV format I must: > > 1) Add this fields in struct fts3rec_v5_gen or struct fts3rec_v5 ? > > 2) Modify the function ftio_read(). > > > > Is it ok? If you can to advice some specific code solution, I will be > > very > > happy! > > > > Thanks a lot. > > > > ----- Original Message ----- > > From: "Orlando Onorato" > > Sent: Tuesday, December 02, 2003 2:14 PM > > Subject: Fw: [flow-tools] format flow description > > > > > >> > >> ----- Original Message ----- > >> From: "Mark Fullmer" > >> To: "Orlando Onorato" > >> Cc: > >> Sent: Tuesday, December 02, 2003 1:35 AM > >> Subject: Re: [flow-tools] format flow description > >> > >> > >>> > >>> Flow-tools does not store these fields, they're only necessary > >>> to get the data from the exporter to the collector. > >>> > >>> The 'pad' field is there to provide alignment. > >>> > >>> You could look in lib/ftdecode.c or src/fdg.c for some examples on > >>> how to find these fields > >>> > >>> mark > >>> > >>> On Dec 1, 2003, at 9:17 AM, Orlando Onorato wrote: > >>> > >>>> In particular I need of these header fields: > >>>> ushort version, ushort count, ulong flow_sequence. > >>>> > >>>> Why the field "uchar pad" isn't there in the flow record? > >>>> > >>>> Thanks for your patience! > >>>> > >>>> bye > >>>> > >>>> > >>>> ----- Original Message ----- > >>>> From: "Mark Fullmer" > >>>> To: "Orlando Onorato" > >>>> Cc: > >>>> Sent: Monday, December 01, 2003 2:13 PM > >>>> Subject: Re: [flow-tools] format flow description > >>>> > >>>> > >>>>> Which header are you trying to look at? > >>>>> > >>>>> mark > >>>>> > >>>>> On Dec 1, 2003, at 5:09 AM, Orlando Onorato wrote: > >>>>> > >>>>>> Thank you very much. I've solved my 2nd problem. > >>>>>> > >>>>>> By mean the flow-export I'm not able to view the header yet, > >>>>>> and I don't want to use the tcpdump. > >>>>>> > >>>>>> Can you help me? > >>>>>> > >>>>>> > >>>>>> ----- Original Message ----- > >>>>>> From: "Mark Fullmer" > >>>>>> To: "Orlando Onorato" > >>>>>> Cc: > >>>>>> Sent: Saturday, November 29, 2003 2:20 AM > >>>>>> Subject: Re: [flow-tools] format flow description > >>>>>> > >>>>>> > >>>>>>> It's there. Use flow-export -f2 (ASCII). > >>>>>>> > >>>>>>> #: > >>>>>>> > > unix_secs,unix_nsecs,sysuptime,exaddr,dpkts,doctets,first,last,engin > >>>>>>> e_ > >>>>>>> ty > >>>>>>> pe,eng > >>>>>>> > > ine_id,srcaddr,dstaddr,nexthop,input,output,srcport,dstport,prot,tos > >>>>>>> ,t > >>>>>>> cp > >>>>>>> _flags,s > >>>>>>> rc_mask,dst_mask,src_as,dst_as > >>>>>>> > > 1018404899,502659552,3685185404,192.88.192.245,1,40,3685169372,36851 > >>>>>>> 69 > >>>>>>> 37 > >>>>>>> 2,4,0,12 > >>>>>>> > > 9.137.254.5,129.21.7.49,192.88.192.33,40,25,56279,33508,17,0,16,16,1 > >>>>>>> 6, > >>>>>>> 20 > >>>>>>> 126,4385 > >>>>>>> ^^^^^^^^^^^^^ > >>>>>>> > >>>>>>> mark > >>>>>>> > >>>>>>> > >>>>>>> On Nov 28, 2003, at 8:18 AM, Orlando Onorato wrote: > >>>>>>> > >>>>>>>> By means of flow-export I'm not able to view the Nexhop field, > >>>>>>>> although this filed there is in Netflow ver.5. > >>>>>>>> > >>>>>>>> > >>>>>>>> ----- Original Message ----- > >>>>>>>> From: "Bill Fumerola" > >>>>>>>> To: "Orlando Onorato" > >>>>>>>> Cc: > >>>>>>>> Sent: Thursday, November 27, 2003 9:28 PM > >>>>>>>> Subject: Re: [flow-tools] format flow description > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Thu, Nov 27, 2003 at 03:38:57PM +0100, Orlando Onorato > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> 1) View header description of datagramm generated. > >>>>>>>>> > >>>>>>>>> dunno what you mean here, but increasing the debug level will > > show > >>>>>>>>> the > >>>>>>>>> header of the flow file. if you want to see ip or udp header > >>>>>>>>> information > >>>>>>>>> from the actual netflow packet, i'd suggest tcpdump. > >>>>>>>>> > >>>>>>>>>> 2) View all fields of flow description (e.g. nexthop field. > >>>>>>>>> > >>>>>>>>> flow-export > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> - bill fumerola / fumerola@yahoo-inc.com / billf@FreeBSD.org > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> _______________________________________________ > >>>>>>>> flow-tools@splintered.net > >>>>>>>> http://www.splintered.net/sw/flow-tools > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> flow-tools@splintered.net > >>>>>>> http://www.splintered.net/sw/flow-tools > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> flow-tools@splintered.net > >>>>> http://www.splintered.net/sw/flow-tools > >>>> > >>> > >> > > > From splintered-flow-tools-owner@splintered.net Wed Dec 03 21:55:09 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 96790 invoked by uid 4001); 3 Dec 2003 21:55:09 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 96788 invoked by alias); 3 Dec 2003 21:55:08 -0000 Received: from web21501.mail.yahoo.com (66.163.169.12) by 66.250.216.131 with SMTP; 3 Dec 2003 21:55:08 -0000 Message-ID: <20031203215507.51408.qmail@web21501.mail.yahoo.com> Received: from [138.222.250.65] by web21501.mail.yahoo.com via HTTP; Wed, 03 Dec 2003 13:55:07 PST Date: Wed, 3 Dec 2003 13:55:07 -0800 (PST) From: johann lafer To: flow-tools@splintered.net MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-985996298-1070488507=:50736" Cc: Subject: [Flow-tools] Performance / compile question X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2003 21:55:09 -0000 --0-985996298-1070488507=:50736 Content-Type: text/plain; charset=us-ascii Hello Mark, in the mailing list i found http://www.pairlist.net/pipermail/flow-tools/2003-January/001007.html where you wrote, that compiling flow-tools with -O or -O2 increases the performance. I tried to use the cflags, but running "make" still shows -g -Wall. Which modifications do i have to do where? Have you ever tried to compile flow-tools with the processor option ?(i know this is more a linux question). Thanks Janno --------------------------------- Do you Yahoo!? Free Pop-Up Blocker - Get it now --0-985996298-1070488507=:50736 Content-Type: text/html; charset=us-ascii
Hello Mark,
 
in the mailing list i found
 
 
where you wrote, that compiling flow-tools with -O or -O2 increases the performance.
 
I tried to use the cflags, but running "make" still shows -g -Wall. Which modifications do i have to do where? Have you ever tried to compile flow-tools with the processor option ?(i know this is more a linux question). 
 
Thanks
 
Janno

Do you Yahoo!?
Free Pop-Up Blocker - Get it now --0-985996298-1070488507=:50736-- From splintered-flow-tools-owner@splintered.net Wed Dec 03 23:42:03 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 97945 invoked by uid 4001); 3 Dec 2003 23:42:03 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 97942 invoked by alias); 3 Dec 2003 23:42:03 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 3 Dec 2003 23:42:03 -0000 In-Reply-To: <20031203215507.51408.qmail@web21501.mail.yahoo.com> References: <20031203215507.51408.qmail@web21501.mail.yahoo.com> Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=ISO-8859-1; format=flowed Message-Id: <4B20BDF4-25EA-11D8-A8C6-000A95DA1C38@splintered.net> Content-Transfer-Encoding: quoted-printable From: Mark Fullmer Subject: Re: [Flow-tools] Performance / compile question Date: Wed, 3 Dec 2003 18:42:02 -0500 To: johann lafer X-Mailer: Apple Mail (2.606) Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2003 23:42:03 -0000 src/Makefile.am and lib/Makefile.am. You'll need to have automake installed. Haven't tried any of the processor options to gcc. Are you having performance problems? mark On Dec 3, 2003, at 4:55 PM, johann lafer wrote: > Hello Mark, > =A0 > in the mailing list i found > =A0 > http://www.pairlist.net/pipermail/flow-tools/2003-January/001007.html > =A0 > where you wrote, that compiling flow-tools with -O or -O2 increases=20 > the performance. > =A0 > I tried to use the cflags, but running "make" still shows -g -Wall.=20 > Which modifications do i have to do where? Have you ever tried to=20 > compile flow-tools with the processor option ?(i know this is more a=20= > linux question).=A0 > =A0 > Thanks > =A0 > Janno > > Do you Yahoo!? > Free Pop-Up Blocker - Get it=20 > now_______________________________________________ > Flow-tools mailing list > flow-tools@splintered.net > http://mailman.splintered.net/mailman/listinfo/flow-tools= From splintered-flow-tools-owner@splintered.net Thu Dec 04 05:08:23 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 99956 invoked by uid 4001); 4 Dec 2003 05:08:23 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 99953 invoked by alias); 4 Dec 2003 05:08:23 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 4 Dec 2003 05:08:23 -0000 In-Reply-To: <002401c3b9be$09cc88f0$61af0a0a@Accenture.com> References: <00eb01c3b8e1$a503d4f0$61af0a0a@Accenture.com> <7C0EBFEA-24DD-11D8-87CF-000A95DA1C38@splintered.net> <002401c3b9be$09cc88f0$61af0a0a@Accenture.com> Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Mark Fullmer Subject: Re: [flow-tools] format flow description Date: Thu, 4 Dec 2003 00:08:22 -0500 To: "Orlando Onorato" X-Mailer: Apple Mail (2.606) Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 05:08:23 -0000 It sounds like you want to be using flow-fanout to replicate the data from a router to multiple destinations... The data does not have sequence numbers in it. It's possible flow-tools will create the first flow with a sequence number with the number of flows encoded, this is just how its implemented. There's nothing special about when the sequence numbers start. mark On Dec 3, 2003, at 11:53 AM, Orlando Onorato wrote: > Hi Mark, > probably I've success about my goal - thanks to you #;-) > > Now I've this problem: whit flow-gen the flow_sequence is ever equals > to 0. > Since for now I haven't got a real netflow traffic I wold be test my > changes... #;-( > have you got some real netflow traffic stored into files? > > Thanks a lot. > > Orlando > > > ----- Original Message ----- > From: "Mark Fullmer" > To: "Orlando Onorato" > Cc: > Sent: Tuesday, December 02, 2003 4:37 PM > Subject: Re: [flow-tools] format flow description > > >> >> These fields are only necessary when sending data in Cisco NetFlow >> format >> over a network. Modifying flow-tools to store them is probably not >> the >> correct approach. >> >> What exactly are you trying to do? >> >> The "wire" option to flow-export will produce output that looks like >> what >> would be sent over the wire. This can then be stored to a file. >> >> flow-send will transmit flow-tools data in Cisco NetFlow format over a >> network. >> >> Both the above two options will create the fields you're looking for. >> >> mark >> >> >> On Dec 2, 2003, at 9:36 AM, Orlando Onorato wrote: >> >>> Ok! >>> I want understand if flow_gen produce this fields (ushort version, >>> ushort >>> count, ulong flow_sequence). I believe yes! >>> >>> Then, for obtain this fields in ASCII CSV format I must: >>> 1) Add this fields in struct fts3rec_v5_gen or struct fts3rec_v5 ? >>> 2) Modify the function ftio_read(). >>> >>> Is it ok? If you can to advice some specific code solution, I will be >>> very >>> happy! >>> >>> Thanks a lot. >>> >>> ----- Original Message ----- >>> From: "Orlando Onorato" >>> Sent: Tuesday, December 02, 2003 2:14 PM >>> Subject: Fw: [flow-tools] format flow description >>> >>> >>>> >>>> ----- Original Message ----- >>>> From: "Mark Fullmer" >>>> To: "Orlando Onorato" >>>> Cc: >>>> Sent: Tuesday, December 02, 2003 1:35 AM >>>> Subject: Re: [flow-tools] format flow description >>>> >>>> >>>>> >>>>> Flow-tools does not store these fields, they're only necessary >>>>> to get the data from the exporter to the collector. >>>>> >>>>> The 'pad' field is there to provide alignment. >>>>> >>>>> You could look in lib/ftdecode.c or src/fdg.c for some examples on >>>>> how to find these fields >>>>> >>>>> mark >>>>> >>>>> On Dec 1, 2003, at 9:17 AM, Orlando Onorato wrote: >>>>> >>>>>> In particular I need of these header fields: >>>>>> ushort version, ushort count, ulong flow_sequence. >>>>>> >>>>>> Why the field "uchar pad" isn't there in the flow record? >>>>>> >>>>>> Thanks for your patience! >>>>>> >>>>>> bye >>>>>> >>>>>> >>>>>> ----- Original Message ----- >>>>>> From: "Mark Fullmer" >>>>>> To: "Orlando Onorato" >>>>>> Cc: >>>>>> Sent: Monday, December 01, 2003 2:13 PM >>>>>> Subject: Re: [flow-tools] format flow description >>>>>> >>>>>> >>>>>>> Which header are you trying to look at? >>>>>>> >>>>>>> mark >>>>>>> >>>>>>> On Dec 1, 2003, at 5:09 AM, Orlando Onorato wrote: >>>>>>> >>>>>>>> Thank you very much. I've solved my 2nd problem. >>>>>>>> >>>>>>>> By mean the flow-export I'm not able to view the header yet, >>>>>>>> and I don't want to use the tcpdump. >>>>>>>> >>>>>>>> Can you help me? >>>>>>>> >>>>>>>> >>>>>>>> ----- Original Message ----- >>>>>>>> From: "Mark Fullmer" >>>>>>>> To: "Orlando Onorato" >>>>>>>> Cc: >>>>>>>> Sent: Saturday, November 29, 2003 2:20 AM >>>>>>>> Subject: Re: [flow-tools] format flow description >>>>>>>> >>>>>>>> >>>>>>>>> It's there. Use flow-export -f2 (ASCII). >>>>>>>>> >>>>>>>>> #: >>>>>>>>> >>> unix_secs,unix_nsecs,sysuptime,exaddr,dpkts,doctets,first,last,engin >>>>>>>>> e_ >>>>>>>>> ty >>>>>>>>> pe,eng >>>>>>>>> >>> ine_id,srcaddr,dstaddr,nexthop,input,output,srcport,dstport,prot,tos >>>>>>>>> ,t >>>>>>>>> cp >>>>>>>>> _flags,s >>>>>>>>> rc_mask,dst_mask,src_as,dst_as >>>>>>>>> >>> 1018404899,502659552,3685185404,192.88.192.245,1,40,3685169372,36851 >>>>>>>>> 69 >>>>>>>>> 37 >>>>>>>>> 2,4,0,12 >>>>>>>>> >>> 9.137.254.5,129.21.7.49,192.88.192.33,40,25,56279,33508,17,0,16,16,1 >>>>>>>>> 6, >>>>>>>>> 20 >>>>>>>>> 126,4385 >>>>>>>>> ^^^^^^^^^^^^^ >>>>>>>>> >>>>>>>>> mark >>>>>>>>> >>>>>>>>> >>>>>>>>> On Nov 28, 2003, at 8:18 AM, Orlando Onorato wrote: >>>>>>>>> >>>>>>>>>> By means of flow-export I'm not able to view the Nexhop field, >>>>>>>>>> although this filed there is in Netflow ver.5. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ----- Original Message ----- >>>>>>>>>> From: "Bill Fumerola" >>>>>>>>>> To: "Orlando Onorato" >>>>>>>>>> Cc: >>>>>>>>>> Sent: Thursday, November 27, 2003 9:28 PM >>>>>>>>>> Subject: Re: [flow-tools] format flow description >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Thu, Nov 27, 2003 at 03:38:57PM +0100, Orlando Onorato >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> 1) View header description of datagramm generated. >>>>>>>>>>> >>>>>>>>>>> dunno what you mean here, but increasing the debug level will >>> show >>>>>>>>>>> the >>>>>>>>>>> header of the flow file. if you want to see ip or udp header >>>>>>>>>>> information >>>>>>>>>>> from the actual netflow packet, i'd suggest tcpdump. >>>>>>>>>>> >>>>>>>>>>>> 2) View all fields of flow description (e.g. nexthop field. >>>>>>>>>>> >>>>>>>>>>> flow-export >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> - bill fumerola / fumerola@yahoo-inc.com / billf@FreeBSD.org >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> flow-tools@splintered.net >>>>>>>>>> http://www.splintered.net/sw/flow-tools >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> flow-tools@splintered.net >>>>>>>>> http://www.splintered.net/sw/flow-tools >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> flow-tools@splintered.net >>>>>>> http://www.splintered.net/sw/flow-tools >>>>>> >>>>> >>>> >>> >> > > _______________________________________________ > Flow-tools mailing list > flow-tools@splintered.net > http://mailman.splintered.net/mailman/listinfo/flow-tools From splintered-flow-tools-owner@splintered.net Thu Dec 04 06:08:28 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 1176 invoked by uid 4001); 4 Dec 2003 06:08:28 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 1172 invoked by alias); 4 Dec 2003 06:08:28 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 4 Dec 2003 06:08:28 -0000 In-Reply-To: <3EE9EC1A0F3FA640B5F99FF3B7C915FD02EE61A5@snkxs001.scrippsnetworks.com> References: <3EE9EC1A0F3FA640B5F99FF3B7C915FD02EE61A5@snkxs001.scrippsnetworks.com> Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <46977FA2-2620-11D8-A8C6-000A95DA1C38@splintered.net> Content-Transfer-Encoding: 7bit From: Mark Fullmer Subject: Re: [flow-tools] Tools to watch for viruses or worms? Date: Thu, 4 Dec 2003 01:08:27 -0500 To: "Cowell, Andrew" X-Mailer: Apple Mail (2.606) Cc: "'flow-tools@splintered.net'" X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 06:08:28 -0000 You can IP catch scanners with the ip-source-address-destination-count and/or ip-destination-address-source-count. The ip-destination-address-source count was broken in 0.66. It's fixed in 0.67. mark On Oct 22, 2003, at 1:00 PM, Cowell, Andrew wrote: > Hey, I'm trying to use flow exports to watch for viruses and worms on > our > network. I haven't found any tools to do so yet. Anybody know of one? > Does anybody already have flow-tools filters for various worm > signatures? > The main suspicious activity I've been watching for has been sequential > network mapping, but I don't see how to catch that with flow-tools. > Any > ideas? > > -- > Andy Cowell > acowell@scrippsops.com > Senior Network Administrator > E.W. Scripps Corp. IT Operations and Engineering > ph: (865) 560-4652 > > > _______________________________________________ > flow-tools@splintered.net > http://www.splintered.net/sw/flow-tools > > From splintered-flow-tools-owner@splintered.net Thu Dec 04 06:41:55 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 2157 invoked by uid 4001); 4 Dec 2003 06:41:55 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 2155 invoked by alias); 4 Dec 2003 06:41:55 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 4 Dec 2003 06:41:55 -0000 Mime-Version: 1.0 (Apple Message framework v606) Content-Transfer-Encoding: 7bit Message-Id: Content-Type: text/plain; charset=US-ASCII; format=flowed To: 'flow-tools@splintered.net' From: Mark Fullmer Date: Thu, 4 Dec 2003 01:41:54 -0500 X-Mailer: Apple Mail (2.606) Cc: Subject: [Flow-tools] flow-tools 0.67 X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 06:41:55 -0000 A new snapshot of flow-tools is available at http://www.splintered.net/sw/flow-tools. I've been away from ft development for about 6 months, so there are a lot of accumulated contributions, fixes for reported bugs, and some new code. I'm expecting to have another snapshot out around the first of the year which will include initial support for NetFlow v9. Full support for v9 requires a file format change which probably won't be done until a little later. If you've sent a bug fix, request, or just a bug report that didn't make it in to 0.67 please send it on again, it probably got lost in my junk mailbox. mark * 12-4-2003 flow-tools 0.67 released. * flow-export: pgsql support from wyu@ateneo.edu * docs: flow-report: Added description of reports. * ftlib: ftfil.c - match_end_time() broken - noted by "Joe Loiacono" * ftlib: fttag.c - better syntax checking for or-src/dst and set-src/dst * ftlib: ftlib.h FT_TAG_TYPE_MATCH_NEXTHOP duplicated - flow-tag crash with next-hop type noted by Maxim Grigoriev * ftlib: ftstat.c - broken ip-destination-address-source-count. patch from "Shigeki Taniguchi" * flow-fanout: filters not loaded - noted by RAR@syssrc.com * ftlib: missing function prototypes for ftstat_*, rename bind to binding to prevent shadowing bind(). patch from Bill Fumerola * flow-fanout, flow-capture. Process SIGTERM like SIGQUIT so flow-tools will work better under daemontools - req by Bernhard Weisshuhn * docs: flow-nfilter and flow-cat TIME/DATE parsing section. * flow-dscan: drp->flags not updated when loading saved state - patch from Jon Snyder * flow-dscan: allow concurrent -w and -W, patch from Dan Thorson * docs: flow-print -f24 - noted by noted by Christian Bauer * dist: tag.sym and tag.cfg example files reversed - noted by * ftlib: ftlib.h - FT_TAG_SET and FT_TAG_OR are broken - patch from Valtteri Vuorikoski * ftlib: ftrec.c - add 1005to5 translation - patch from Valtteri Vuorikoski * flow-stat -f0 will try to divide by 0 with an empty flow file - noted by Mike Hunter * flow-capture: -u preserve unherited umask - patch from Everton da Silva Marques * flow-receive: remove -m and -A. * flow-capture: remove -m and -A, functionality is now in xlate -x -X. * flow-xlate: - config file based now. * docs: flow-report: note which fields are sortable and what the key field is. * flow-capture: accept()'s 3rd arg should be casted to socklen_t*, noted by Alistair.McGlinchy@marks-and-spencer.com * docs: flow-nfilter, port is 0..65535 not 0..255 - noted by Mike Hunter * ftlib: ftlib.h - set-{dst,src} and or-{dst,src} constants not correct - patch from Valtteri Vuorikoski * ftlib: ftchash.c - ftchash_sort() should not try to sort 0 entry table - noted by "Shane D." * flow-import: missing !HAVE_STRSEP compatability - patch from Alistair.McGlinchy@marks-and-spencer.com * ftlib: ftstat.c - output path not parsed correctly with leading whitespace -- noted by Maxim Grigoriev * ftlib: fttag.c - src->source dst->destination * ftlib: fttag.c - ip-address, exporter, interface tag actions, requested by Tim Irwin * ftlib: ftsym.c - ftsym_new() should handle null filename - noted by Celso Alves Vieira * flow-dscan: buf len 64, not 54 - Anil Madhavapeddy From splintered-flow-tools-owner@splintered.net Thu Dec 04 10:30:16 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 4198 invoked by uid 4001); 4 Dec 2003 10:30:16 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 4195 invoked by alias); 4 Dec 2003 10:30:15 -0000 Received: from webmail7.rediffmail.com (HELO rediffmail.com) (202.54.124.152) by 66.250.216.131 with SMTP; 4 Dec 2003 10:30:15 -0000 Received: (qmail 16476 invoked by uid 510); 4 Dec 2003 10:29:50 -0000 Date: 4 Dec 2003 10:29:50 -0000 Message-ID: <20031204102950.16474.qmail@webmail7.rediffmail.com> Received: from unknown (203.200.25.5) by rediffmail.com via HTTP; 04 dec 2003 10:29:50 -0000 MIME-Version: 1.0 From: "Chitman Kaur " To: flow-tools@splintered.net Content-type: multipart/alternative; boundary="Next_1070533790---0-202.54.124.152-16465" Cc: Subject: [Flow-tools] (no subject) X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list Reply-To: Chitman Kaur List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 10:30:16 -0000 This is a multipart mime message --Next_1070533790---0-202.54.124.152-16465 Content-type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline

=0AHi All....
=0AEven after putting the option -z0 I am getting lost = flows.....
=0AAny reason why....
=0AI am sure that there is no conges= tion between my router and collector.....
=0AAm I giving the option corr= ectly....
=0A-----------------------------------------------------------= ------------
=0A/usr/local/netflow/bin/flow-capture -w /var/netflow/ft 0= /0/2055 -z0 -S5 -V5 -E1G -n 287 -N 0 -R /usr/local/netflow/bin/linkme
= =0Atouch /var/lock/subsys/startflows
=0A--------------------------------= ---------------------------------------
=0ARegards
=0AChitman=0A

= =0A=0A=0A

=0A=0A --Next_1070533790---0-202.54.124.152-16465 Content-type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hi All....=0AEven after putting the option -z0 I am getting lost flows.....= =0AAny reason why....=0AI am sure that there is no congestion between my ro= uter and collector.....=0AAm I giving the option correctly....=0A----------= -------------------------------------------------------------=0A/usr/local/= netflow/bin/flow-capture -w /var/netflow/ft 0/0/2055 -z0 -S5 -V5 -E1G -n 28= 7 -N 0 -R /usr/local/netflow/bin/linkme=0Atouch /var/lock/subsys/startflows= =0A-----------------------------------------------------------------------= =0ARegards=0AChitman=0A=0A --Next_1070533790---0-202.54.124.152-16465-- From splintered-flow-tools-owner@splintered.net Thu Dec 04 15:55:03 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 5981 invoked by uid 4001); 4 Dec 2003 15:55:03 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 5979 invoked by alias); 4 Dec 2003 15:55:02 -0000 Received: from web21504.mail.yahoo.com (66.163.169.15) by 66.250.216.131 with SMTP; 4 Dec 2003 15:55:02 -0000 Message-ID: <20031204155458.86623.qmail@web21504.mail.yahoo.com> Received: from [138.222.250.65] by web21504.mail.yahoo.com via HTTP; Thu, 04 Dec 2003 07:54:58 PST Date: Thu, 4 Dec 2003 07:54:58 -0800 (PST) From: johann lafer Subject: Re: [Flow-tools] Performance / compile question To: flow-tools@splintered.net In-Reply-To: <4B20BDF4-25EA-11D8-A8C6-000A95DA1C38@splintered.net> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-875184259-1070553298=:86102" Cc: X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 15:55:03 -0000 --0-875184259-1070553298=:86102 Content-Type: text/plain; charset=us-ascii Hm, that wasn't so easy. First I had to install autoconf and automake. Then I changed Makefile.am in both dirs. Running make, there is still a "-g -Wall" visible. So I am not really shure, if the optimization happend. I also appended -mpentium. But i do not notice any performance optimization. I don't know if it is really a performance problem, but the whole story is, that i have written a php frontend with flow-tools (flow-cat,flow-nfilter,flow-stat) in the background. If a web-user wants to generate a 1,3,12 or 24 hour report, it takes up to 5 minutes. But creating a "week-report" or "month-report" takes more than 25 minutes, depending on the generated data. The size of 5 minute flow-file differs between 1 and 4 Mbytes. Maybe i am impatient?! It looks like a hanging application if you have to wait to long. Yesterday i tried flow-export -z to compress all files generated the day before to single compressed file. The volume seems to be compress 10% and a report was generated 30% faster, but i do not know if this is an accident. Another idea is to run more than 1 process for a report (up to 31 process for a month report, bottleneck competing CPU and/or HD access?!) Is there a possibility to make the data volume smaller without losing important information? Thanks Janno Mark Fullmer wrote: src/Makefile.am and lib/Makefile.am. You'll need to have automake installed. Haven't tried any of the processor options to gcc. Are you having performance problems? mark On Dec 3, 2003, at 4:55 PM, johann lafer wrote: > Hello Mark, > > in the mailing list i found > > http://www.pairlist.net/pipermail/flow-tools/2003-January/001007.html > > where you wrote, that compiling flow-tools with -O or -O2 increases > the performance. > > I tried to use the cflags, but running "make" still shows -g -Wall. > Which modifications do i have to do where? Have you ever tried to > compile flow-tools with the processor option ?(i know this is more a > linux question). > > Thanks > > Janno > > Do you Yahoo!? > Free Pop-Up Blocker - Get it > now_______________________________________________ > Flow-tools mailing list > flow-tools@splintered.net > http://mailman.splintered.net/mailman/listinfo/flow-tools --------------------------------- Do you Yahoo!? Free Pop-Up Blocker - Get it now --0-875184259-1070553298=:86102 Content-Type: text/html; charset=us-ascii
Hm,
 
that wasn't so easy. First I had to install autoconf and automake. Then I changed Makefile.am in both dirs. Running make, there is still a "-g -Wall" visible. So I am not really shure, if the optimization happend. I also appended -mpentium. But i do not notice any performance optimization.
 
I don't know if it is really a performance problem, but the whole story is, that i have written a php frontend with flow-tools (flow-cat,flow-nfilter,flow-stat) in the background. If a web-user wants to generate a 1,3,12 or 24 hour report, it takes up to 5 minutes. But creating a "week-report" or "month-report"  takes more than 25 minutes, depending on the generated data. The size of 5 minute flow-file differs between 1 and 4 Mbytes. Maybe i am impatient?! It looks like a hanging application if you have to wait to long.
 
Yesterday i tried flow-export -z to compress all files generated the day before to single compressed file. The volume seems to be compress 10% and a report was generated 30% faster, but i do not know if this is an accident.
 
Another idea is to run more than 1 process for a report (up to 31 process for a month report, bottleneck competing CPU and/or HD access?!)
 
Is there a possibility to make the data volume smaller without losing important information?
 
Thanks
Janno

Mark Fullmer <maf@splintered.net> wrote:
src/Makefile.am and lib/Makefile.am. You'll need to have automake
installed. Haven't tried any of the processor options to gcc.

Are you having performance problems?

mark

On Dec 3, 2003, at 4:55 PM, johann lafer wrote:

> Hello Mark,
>  
> in the mailing list i found
>  
> http://www.pairlist.net/pipermail/flow-tools/2003-January/001007.html
>  
> where you wrote, that compiling flow-tools with -O or -O2 increases
> the performance.
>  
> I tried to use the cflags, but running "make" still shows -g -Wall.
> Which modifications do i have to do where? Have you ever tried to
> compile flow-tools with the processor option ?(i know this is more a
> linux question). 
>  
> Thanks
>  
> Janno
>
> Do you Yahoo!?
> Free Pop-Up Blocker - Get it
> now_______________________________________________
> Flow-tools mailing list
> flow-tools@splintered.net
> http://mailman.splintered.net/mailman/listinfo/flow-tools

Do you Yahoo!?
Free Pop-Up Blocker - Get it now --0-875184259-1070553298=:86102-- From splintered-flow-tools-owner@splintered.net Thu Dec 04 18:53:13 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 7254 invoked by uid 4001); 4 Dec 2003 18:53:13 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 7252 invoked by alias); 4 Dec 2003 18:53:12 -0000 Received: from ip166.usw253.dsl-acs2.sea.iinet.com (HELO ran.psg.com) (209.20.253.166) by 66.250.216.131 with SMTP; 4 Dec 2003 18:53:12 -0000 Received: from localhost ([127.0.0.1] helo=ran.psg.com) by ran.psg.com with esmtp (Exim 4.24; FreeBSD) id 1ARyb9-000FBz-LZ for flow-tools@splintered.net; Thu, 04 Dec 2003 10:53:11 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Thu, 4 Dec 2003 10:53:11 -0800 To: flow list Message-Id: Cc: Subject: [Flow-tools] diagnosing source of ping flood X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 18:53:13 -0000 so, i am experiencing a mild ping attack, only a few hundred k. but i would like to use the experience to sort out how to diagnose this. it seems to be targeting a single ip address inside the lan. how do i ask flow-tools to tell me the source ip addresses of all echo requests toward a specific ip address? also, how do i know if manufacture c's box is giving me the snmp and flow stats before or after rate limiting? thanks. randy From splintered-flow-tools-owner@splintered.net Thu Dec 04 19:37:24 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 8188 invoked by uid 4001); 4 Dec 2003 19:37:24 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 8186 invoked by alias); 4 Dec 2003 19:37:24 -0000 Received: from seabridg01.gettyimages.com (HELO seabridg01.seattle.gettyimages.com) (207.1.176.170) by 66.250.216.131 with SMTP; 4 Dec 2003 19:37:24 -0000 Received: by seabridg01.seattle.gettyimages.com with Internet Mail Service (5.5.2657.72) id ; Thu, 4 Dec 2003 11:37:23 -0800 Message-ID: From: Alex Shepard To: flow list Subject: RE: [Flow-tools] diagnosing source of ping flood Date: Thu, 4 Dec 2003 11:37:19 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain Cc: X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 19:37:24 -0000 Randy, One way to do this is to build yourself a filter and run your flow files through it. [alexs@mybox alexs]$ cat filter.txt filter-primitive icmp type ip-protocol permit 1 filter-primitive host_being_attacked type ip-address permit 10.10.10.10 filter-definition ping_attack match ip-destination-address host_being_attacked match ip-protocol icmp [alexs@mybox alexs]$ flow-nfilter lets you build primitives out of protocols, TCP flags, ip addresses, ports, AS, ifindex, mask, and a bunch of other stuff. then you can use build filters out of those primitives, which contain once you've got your filter built, run it with: flow-cat | flow-nfilter -f -F | flow-print you should see something like: [alexs@mybox alexs]$ flow-cat /var/local/flows/processed/ft-v05.2003-12-04.115231-0800 | flow-nfilter -f filter.txt -F ping_attack | flow-print srcIP dstIP prot srcPort dstPort octets packets 10.8.8.10 10.10.10.10 1 0 0 92 1 192.168.13.1 10.10.10.10 1 0 0 92 1 192.168.14.1 10.10.10.10 1 0 0 92 1 10.9.9.10 10.10.10.10 1 0 0 92 1 192.168.15.1 10.10.10.10 1 0 0 92 1 192.168.14.1 10.10.10.10 1 0 0 78 1 [... etc ...] [alexs@mybox alexs] I'm not sure if flow-tools looks deep enough into icmp packets to differentiate between echo request and other icmp packet types. HTH, alex -----Original Message----- From: flow-tools-bounces@list.splintered.net [mailto:flow-tools-bounces@list.splintered.net] On Behalf Of Randy Bush Sent: Thursday, December 04, 2003 10:53 AM To: flow list Subject: [Flow-tools] diagnosing source of ping flood so, i am experiencing a mild ping attack, only a few hundred k. but i would like to use the experience to sort out how to diagnose this. it seems to be targeting a single ip address inside the lan. how do i ask flow-tools to tell me the source ip addresses of all echo requests toward a specific ip address? also, how do i know if manufacture c's box is giving me the snmp and flow stats before or after rate limiting? thanks. randy _______________________________________________ Flow-tools mailing list flow-tools@splintered.net http://mailman.splintered.net/mailman/listinfo/flow-tools ======================================================= This email and its contents are confidential. If you are not the intended recipient, please do not disclose or use the information within this email or its attachments. If you have received this email in error, please delete it immediately. Thank you. ======================================================= From splintered-flow-tools-owner@splintered.net Thu Dec 04 19:37:32 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 8269 invoked by uid 4001); 4 Dec 2003 19:37:32 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 8266 invoked by alias); 4 Dec 2003 19:37:32 -0000 Received: from mx3.versus.com (HELO mx1.versus.com) (205.205.118.15) by 66.250.216.131 with SMTP; 4 Dec 2003 19:37:32 -0000 Received: from versus.com (boyle-new.versus.com [206.83.39.36]) by mx1.versus.com (Postfix) with ESMTP id B6F687379 for ; Thu, 4 Dec 2003 14:37:31 -0500 (EST) X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Thu, 4 Dec 2003 14:37:31 -0500 Message-ID: <5B38BEB87ED6C449BD0CC4770F69857F01657EE9@boyle.versus.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Working the right way... ;) Thread-Index: AcO6ng5f/pJNIj90TN+BUMup6mTDIw== From: "Eric Rousse" To: Cc: Subject: [Flow-tools] Working the right way... ;) X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 19:37:33 -0000 Hi, I got 2 questions. First, each day I run a command on the flows of the previous day.=20 /usr/local/netflow/bin/flow-merge * | = /usr/local/netflow/bin/flow-nfilter -f /usr/local/scripts/nfilter.cfg -F = backup-match | /usr/local/netflow/bin/flow-stat -f11 > = /export2/netflows/flow-stat/20031204.log Is that the best way of doing my merge and nfilter and flowstat to a = file ? First, I can't do a flow-merge to a file, the file gets too big (more = than 2 gigs, and I get an error about the filesize most probably because of my glib version). Also doing all that at the same time use a lot of memory, I have 512MB = installed on that machine. And sometimes, the hole process gets stuck and it stays there for a = while... So is there a way to improve this ? Also here's the content of my file nfilter.cfg and my second question: filter-primitive backup-hosts type ip-address deny x.x.x.x deny x.x.x.x deny x.x.x.x default permit filter-definition backup-match match ip-source-address backup-hosts or match ip-destination-address backup-hosts Is that the right way of doing this kind of things ? I wanna filter 3 IP (backup servers) out from my flow files. Incoming or outgoing traffic. Is that the right way of doing this, because it doesn't seems to filter all the time, some days I have nothing from these IP, other days I have a huge load of traffic. mmm after all I have another question, I posted that = (http://www.pairlist.net/pipermail/flow-tools/2003-October/001674.html) a few weeks earlier, I changed few things since then, but things are not fixed, I still see that error in a strace, is that normal ? Anyone seeing this also ? Thanks! _________________________________________________ Eric Rousse Versus=AE=20 2050, rue De Bleury, bureau 520 Montr=E9al (Qu=E9bec) H3A 2J5=20 Canada T=E9l.: 514.284.9001 ext. 221 Fax: 514.284.9002 From splintered-flow-tools-owner@splintered.net Thu Dec 04 21:40:10 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 10245 invoked by uid 4001); 4 Dec 2003 21:40:10 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 10243 invoked by alias); 4 Dec 2003 21:40:10 -0000 Received: from ip166.usw253.dsl-acs2.sea.iinet.com (HELO ran.psg.com) (209.20.253.166) by 66.250.216.131 with SMTP; 4 Dec 2003 21:40:10 -0000 Received: from localhost ([127.0.0.1] helo=ran.psg.com) by ran.psg.com with esmtp (Exim 4.24; FreeBSD) id 1AS1Ci-000JhX-Pg; Thu, 04 Dec 2003 13:40:08 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Thu, 4 Dec 2003 13:40:08 -0800 To: Alex Shepard Subject: RE: [Flow-tools] diagnosing source of ping flood References: Message-Id: Cc: flow list X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 21:40:10 -0000 hmmm. % cat flow.filter filter-primitive icmp type ip-protocol permit 1 filter-primitive host_being_attacked type ip-address permit 666.42.7.11 filter-definition ping_attack match ip-destination-address host_being_attacked match ip-protocol icmp % flow-cat $FS | flow-nfilter -f flow.filter -F ping_attack | flow-print srcIP dstIP prot srcPort dstPort octets packets 38.113.11.4 666.42.7.11 1 0 771 405 4 38.113.11.4 666.42.7.11 1 0 771 591 6 81.28.0.133 666.42.7.11 1 0 2816 56 1 140.251.0.25 666.42.7.11 1 0 2048 1500 1 38.113.11.4 666.42.7.11 1 0 771 279 3 but look at those first two lines. so i tried % flow-cat $FS | flow-nfilter -f flow.filter -F ping_attack \ | flow-print -f 12 flow-print: Flow record missing required field for format. clue missing From splintered-flow-tools-owner@splintered.net Thu Dec 04 22:12:17 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 11118 invoked by uid 4001); 4 Dec 2003 22:12:17 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 11116 invoked by alias); 4 Dec 2003 22:12:17 -0000 Received: from seabridg01.gettyimages.com (HELO seabridg01.seattle.gettyimages.com) (207.1.176.170) by 66.250.216.131 with SMTP; 4 Dec 2003 22:12:17 -0000 Received: by seabridg01.seattle.gettyimages.com with Internet Mail Service (5.5.2657.72) id ; Thu, 4 Dec 2003 14:12:16 -0800 Message-ID: From: Alex Shepard To: 'Randy Bush' Subject: RE: [Flow-tools] diagnosing source of ping flood Date: Thu, 4 Dec 2003 14:12:14 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain Cc: flow list X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2003 22:12:17 -0000 "flow-print -f12" tries to print flows reported in Netflow v8.3 format (source aggregation). flow-print options only really add or remove or change which fields are printed. if you want to aggregate data into reports, you'll want to use flow-stat or flow-report. try: % flow-cat $FS | flow-nfilter -f flow.filter -F ping_attack | flow-stat -f10 flow-stat has about 30 canned reports that are very fast. flow-report is much more flexible but is also more complicated. it works more like flow-nfilter. HTH, alex -----Original Message----- From: Randy Bush [mailto:randy@psg.com] Sent: Thursday, December 04, 2003 1:40 PM To: Alex Shepard Cc: flow list Subject: RE: [Flow-tools] diagnosing source of ping flood hmmm. % cat flow.filter filter-primitive icmp type ip-protocol permit 1 filter-primitive host_being_attacked type ip-address permit 666.42.7.11 filter-definition ping_attack match ip-destination-address host_being_attacked match ip-protocol icmp % flow-cat $FS | flow-nfilter -f flow.filter -F ping_attack | flow-print srcIP dstIP prot srcPort dstPort octets packets 38.113.11.4 666.42.7.11 1 0 771 405 4 38.113.11.4 666.42.7.11 1 0 771 591 6 81.28.0.133 666.42.7.11 1 0 2816 56 1 140.251.0.25 666.42.7.11 1 0 2048 1500 1 38.113.11.4 666.42.7.11 1 0 771 279 3 but look at those first two lines. so i tried % flow-cat $FS | flow-nfilter -f flow.filter -F ping_attack \ | flow-print -f 12 flow-print: Flow record missing required field for format. clue missing ======================================================= This email and its contents are confidential. If you are not the intended recipient, please do not disclose or use the information within this email or its attachments. If you have received this email in error, please delete it immediately. Thank you. ======================================================= From splintered-flow-tools-owner@splintered.net Fri Dec 05 01:21:15 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 12389 invoked by uid 4001); 5 Dec 2003 01:21:15 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 12387 invoked by alias); 5 Dec 2003 01:21:14 -0000 Received: from mail.sunet.com.au (HELO jupiter.sunet.com.au) (203.166.102.39) by 66.250.216.131 with SMTP; 5 Dec 2003 01:21:14 -0000 Received: from ganymede.internal.sunet.com.au (canopus.sunet.com.au [::ffff:203.166.102.49]) by jupiter.sunet.com.au with esmtp; Fri, 05 Dec 2003 12:15:45 +1100 Date: Fri, 5 Dec 2003 12:21:33 +1100 (EST) From: Systems Administrator X-X-Sender: sysadmin@ganymede.bcc.local To: johann lafer Subject: Re: [Flow-tools] Performance / compile question In-Reply-To: <20031204155458.86623.qmail@web21504.mail.yahoo.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Dec 2003 01:21:15 -0000 On Thu, 4 Dec 2003, johann lafer wrote: > I don't know if it is really a performance problem, but the whole story > is, that i have written a php frontend with flow-tools > (flow-cat,flow-nfilter,flow-stat) in the background. If a web-user wants > to generate a 1,3,12 or 24 hour report, it takes up to 5 minutes. But > creating a "week-report" or "month-report" takes more than 25 minutes, > depending on the generated data. The size of 5 minute flow-file differs > between 1 and 4 Mbytes. Maybe i am impatient?! It looks like a hanging > application if you have to wait to long. ... > Another idea is to run more than 1 process for a report (up to 31 > process for a month report, bottleneck competing CPU and/or HD access?!) Another option would be, if you've got a limited number of reports (ie. if people are going to be saying "data from my subnet", and not "port x on machine y"), you could generate summaries on a daily basis and log them to a database, and then pull them out when you need them. I'm going to be providing something to our customers where they can view their usage on a monthly basis, divided up by day, and so I can just log all the daily summaries to a database, and query that when I need the info. :) -- Tim Nelson Systems Administrator Sunet Internet Tel: +61 3 5241 1155 Fax: +61 3 5241 6187 Web: http://www.sunet.com.au/ Email: sysadmin@sunet.com.au From splintered-flow-tools-owner@splintered.net Fri Dec 05 02:52:04 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 13487 invoked by uid 4001); 5 Dec 2003 02:52:04 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 13485 invoked by alias); 5 Dec 2003 02:52:04 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 5 Dec 2003 02:52:04 -0000 Mime-Version: 1.0 (Apple Message framework v606) Content-Transfer-Encoding: 7bit Message-Id: <018B7A34-26CE-11D8-A8C6-000A95DA1C38@splintered.net> Content-Type: text/plain; charset=US-ASCII; format=flowed To: 'flow-tools@splintered.net' From: Mark Fullmer Date: Thu, 4 Dec 2003 21:52:03 -0500 X-Mailer: Apple Mail (2.606) Cc: Subject: [Flow-tools] rsync exploit X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Dec 2003 02:52:04 -0000 I reference rsync a lot for use in distributed flow processing, so this latest vulnerability may be of interest. Details at http://rsync.samba.org/ mark From splintered-flow-tools-owner@splintered.net Fri Dec 05 04:04:22 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 14464 invoked by uid 4001); 5 Dec 2003 04:04:22 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 14462 invoked by alias); 5 Dec 2003 04:04:22 -0000 Received: from ip166.usw253.dsl-acs2.sea.iinet.com (HELO ran.psg.com) (209.20.253.166) by 66.250.216.131 with SMTP; 5 Dec 2003 04:04:22 -0000 Received: from localhost ([127.0.0.1] helo=ran.psg.com) by ran.psg.com with esmtp (Exim 4.24; FreeBSD) id 1AS7CX-0003lh-BS; Thu, 04 Dec 2003 20:04:21 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Thu, 4 Dec 2003 20:04:20 -0800 To: Alex Shepard Subject: RE: [Flow-tools] diagnosing source of ping flood References: Message-Id: Cc: flow list X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Dec 2003 04:04:22 -0000 > % flow-cat $FS | flow-nfilter -f flow.filter -F ping_attack | flow-stat -f10 worked. any way i can tell which icmp type i am seeing? randy From splintered-flow-tools-owner@splintered.net Fri Dec 05 06:29:03 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 15707 invoked by uid 4001); 5 Dec 2003 06:29:03 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 15705 invoked by alias); 5 Dec 2003 06:29:03 -0000 Received: from mail3.panix.com (166.84.1.74) by 66.250.216.131 with SMTP; 5 Dec 2003 06:29:03 -0000 Received: from panix5.panix.com (panix5.panix.com [166.84.1.5]) by mail3.panix.com (Postfix) with ESMTP id 3444C98359; Fri, 5 Dec 2003 01:29:02 -0500 (EST) Received: (from eravin@localhost) by panix5.panix.com (8.11.6p2-a/8.8.8/PanixN1.1) id hB56T2N11597; Fri, 5 Dec 2003 01:29:02 -0500 (EST) Date: Fri, 5 Dec 2003 01:29:02 -0500 From: Ed Ravin To: Randy Bush Message-ID: <20031205062901.GA11475@panix.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-Y-Z: 1, 2, 3? Cc: flow list Subject: [Flow-tools] flow stats and rate limiting? X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Dec 2003 06:29:03 -0000 On Thu, Dec 04, 2003 at 10:53:11AM -0800, Randy Bush wrote: > also, how do i know if manufacture c's box is giving me the snmp > and flow stats before or after rate limiting? Empirical testing, methinks, might be the only way to be sure. I seem to recall that packets screened by ACLs are still reported in Netflow from my Cisco 7200 - I suspect it would do the same thing with rate-limited packets. From splintered-flow-tools-owner@splintered.net Fri Dec 05 11:41:14 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 17889 invoked by uid 4001); 5 Dec 2003 11:41:14 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 17887 invoked by alias); 5 Dec 2003 11:41:14 -0000 Received: from dymwsm08.mailwatch.com (204.253.83.44) by 66.250.216.131 with SMTP; 5 Dec 2003 11:41:14 -0000 Received: from MWSC0225.mw4.mailwatch.com (mwsc0225.mw4.mailwatch.com [204.253.83.172]) by dymwsm08.mailwatch.com (8.12.9/8.12.9) with ESMTP id hB5BfEtH016080 for ; Fri, 5 Dec 2003 06:41:14 -0500 Received: from mail pickup service by MWSC0225.mw4.mailwatch.com with Microsoft SMTPSVC; Fri, 5 Dec 2003 06:41:14 -0500 Received: from 204.253.83.77 ([204.253.83.77]) by MWSC0225 with SMTP id 000200197b2a9324-9343-472a-8ed0-b9203b4bea3d; Fri, 05 Dec 2003 06:41:13 -0500 Received: from fmpo1.azell.com (fmpo1.azell.com [136.1.7.9]) by dymwsm03.mailwatch.com (8.12.9/8.12.9) with ESMTP id hB5BecJ1028818 for ; Fri, 5 Dec 2003 06:40:38 -0500 Received: from na1ecs06.dearborn.ford.com ([19.5.116.123]) by fmpo1.azell.com (Mirapoint Messaging Server MOS 3.3.5-GR) with ESMTP id BIM11911; Fri, 5 Dec 2003 06:41:13 -0500 (EST) Received: by na1ecs06.dearborn.ford.com with Internet Mail Service (5.5.2657.72) id ; Fri, 5 Dec 2003 06:41:12 -0500 Message-ID: From: "Nunn, Mike (M.)" To: "'flow-tools@splintered.net'" Date: Fri, 5 Dec 2003 06:40:09 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain; charset="iso-8859-1" X-MW-BTID: 090625000020033394203800003 X-MW-CTIME: 1070624438 X-MW-SENDING-MTA: 136.1.7.9 HOP-COUNT: 1 X-MAILWATCH-INSTANCEID: 010200197b2a9324-9343-472a-8ed0-b9203b4bea3d X-OriginalArrivalTime: 05 Dec 2003 11:41:14.0061 (UTC) FILETIME=[AFE2F7D0:01C3BB24] Cc: Subject: [Flow-tools] flows dropped by acl X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Dec 2003 11:41:15 -0000 There seems to be conflicting evidence about Netflow reporting traffic dropped by an acl on the exporting router, is there a definitive answer ? Thanks, Mike From splintered-flow-tools-owner@splintered.net Sat Dec 06 01:41:00 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 22402 invoked by uid 4001); 6 Dec 2003 01:41:00 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 22400 invoked by alias); 6 Dec 2003 01:41:00 -0000 Received: from elvis.mu.org (192.203.228.196) by 66.250.216.131 with SMTP; 6 Dec 2003 01:41:00 -0000 Received: by elvis.mu.org (Postfix, from userid 1098) id 01DC72ED472; Fri, 5 Dec 2003 17:40:50 -0800 (PST) Date: Fri, 5 Dec 2003 17:40:49 -0800 From: Bill Fumerola To: Ed Ravin Subject: Re: [Flow-tools] flow stats and rate limiting? Message-ID: <20031206014049.GT91301@elvis.mu.org> References: <20031205062901.GA11475@panix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20031205062901.GA11475@panix.com> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.8-MUORG-20030806 i386 Cc: flow list X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Dec 2003 01:41:01 -0000 On Fri, Dec 05, 2003 at 01:29:02AM -0500, Ed Ravin wrote: > On Thu, Dec 04, 2003 at 10:53:11AM -0800, Randy Bush wrote: > > also, how do i know if manufacture c's box is giving me the snmp > > and flow stats before or after rate limiting? > > Empirical testing, methinks, might be the only way to be sure. I seem > to recall that packets screened by ACLs are still reported in Netflow > from my Cisco 7200 - I suspect it would do the same thing with rate-limited > packets. the destination interface may be zero for flows that summarize traffic that was dropped as part of policy. or it may not. i think it's architecture specific for vendor C. -- - bill fumerola / fumerola@yahoo-inc.com / billf@FreeBSD.org From splintered-flow-tools-owner@splintered.net Mon Dec 08 17:19:46 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 36427 invoked by uid 4001); 8 Dec 2003 17:19:46 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 36424 invoked by alias); 8 Dec 2003 17:19:46 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 8 Dec 2003 17:19:46 -0000 In-Reply-To: <20031204102950.16474.qmail@webmail7.rediffmail.com> References: <20031204102950.16474.qmail@webmail7.rediffmail.com> Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Mark Fullmer Subject: Re: [Flow-tools] (no subject) Date: Mon, 8 Dec 2003 12:19:46 -0500 To: Chitman Kaur X-Mailer: Apple Mail (2.606) Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2003 17:19:46 -0000 1) look for packet loss between the exporter and collector. 2) Make sure the collector isn't overloaded and dropping flows on input. on *BSD you can do this with netstat -s | grep buf 0 dropped due to full socket buffers 0 output packets dropped due to no bufs, etc. 0 output packets dropped due to no bufs, etc. Mbuf statistics: 519 one mbuf two or more mbuf: 0 one ext mbuf 0 two or more ext mbuf Note the "dropped due to full socket buffers" 3) If the collector is dropping flows raise the priority of flow-capture. With FreeBSD I use 'rtprio 5 flow-capture ...'. Rtprio makes a big difference if you're running other software on the same server. 4) If it's an IOS device look at the output of 'sh ip flow export' krc5>sh ip flow export Flow export v5 is enabled for main cache Exporting flows to X.X.X.X (7998) Exporting using source interface Loopback0 Version 5 flow records, origin-as 4044972686 flows exported in 134907285 udp datagrams 1827 flows failed due to lack of export packet 186 export packets were sent up to process level 0 export packets were dropped due to no fib 642587 export packets were dropped due to adjacency issues 0 export packets were dropped due to fragmentation failures 0 export packets were dropped due to encapsulation fixup failures 112151 export packets were dropped enqueuing for the RP 834609 export packets were dropped due to IPC rate limiting 0 export packets were dropped due to output drops Note lines like the "834609 export packets dropped to IPC rate limiting". In this case the router can't handle the flow export rate and is dropping the flows internally. mark On Dec 4, 2003, at 5:29 AM, Chitman Kaur wrote: > Hi All.... > Even after putting the option -z0 I am getting lost flows..... > Any reason why.... > I am sure that there is no congestion between my router and > collector..... > Am I giving the option correctly.... > ----------------------------------------------------------------------- > /usr/local/netflow/bin/flow-capture -w /var/netflow/ft 0/0/2055 -z0 > -S5 -V5 -E1G -n 287 -N 0 -R /usr/local/netflow/bin/linkme > touch /var/lock/subsys/startflows > ----------------------------------------------------------------------- > Regards > Chitman > > _______________________________________________ > Flow-tools mailing list > flow-tools@splintered.net > http://mailman.splintered.net/mailman/listinfo/flow-tools From splintered-flow-tools-owner@splintered.net Mon Dec 08 17:23:44 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 36999 invoked by uid 4001); 8 Dec 2003 17:23:44 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 36997 invoked by alias); 8 Dec 2003 17:23:44 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 8 Dec 2003 17:23:44 -0000 In-Reply-To: <5B38BEB87ED6C449BD0CC4770F69857F01657EE9@boyle.versus.com> References: <5B38BEB87ED6C449BD0CC4770F69857F01657EE9@boyle.versus.com> Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed Message-Id: <46966BF4-29A3-11D8-81D6-00039304FAA6@eng.oar.net> Content-Transfer-Encoding: quoted-printable From: Mark Fullmer Subject: Re: [Flow-tools] Working the right way... ;) Date: Mon, 8 Dec 2003 12:23:44 -0500 To: "Eric Rousse" X-Mailer: Apple Mail (2.606) Cc: flow-tools@splintered.net X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2003 17:23:44 -0000 flow-merge by its nature is going to be slow, it sounds like you want to be using flow-cat. For files larger than 2 gig you'll probably need to use the -m (disable =20= mmap) option to flow-cat. Yes flow-tools should do this automatically... Your filters look correct...Can you send me a more specific example when it's not filtering correctly? The strace you sent looks like you have tags configured, are using the =20= "OSU" tag name and it doesn't exist in your config fle. mark On Dec 4, 2003, at 2:37 PM, Eric Rousse wrote: > > Hi, > > I got 2 questions. > > First, each day I run a command on the flows of the previous > day. > > /usr/local/netflow/bin/flow-merge * | =20 > /usr/local/netflow/bin/flow-nfilter -f /usr/local/scripts/nfilter.cfg =20= > -F backup-match | /usr/local/netflow/bin/flow-stat -f11 > =20 > /export2/netflows/flow-stat/20031204.log > > Is that the best way of doing my merge and nfilter and flowstat to a =20= > file ? > First, I can't do a flow-merge to a file, the file gets too big (more =20= > than 2 gigs, and I get > an error about the filesize most probably because of my glib version). > Also doing all that at the same time use a lot of memory, I have 512MB = =20 > installed on that machine. > And sometimes, the hole process gets stuck and it stays there for a =20= > while... > So is there a way to improve this ? > > > Also here's the content of my file nfilter.cfg and my second question: > > filter-primitive backup-hosts > type ip-address > deny x.x.x.x > deny x.x.x.x > deny x.x.x.x > default permit > > filter-definition backup-match > match ip-source-address backup-hosts > or > match ip-destination-address backup-hosts > > > Is that the right way of doing this kind of things ? > I wanna filter 3 IP (backup servers) out from my flow files. > Incoming or outgoing traffic. > > Is that the right way of doing this, because it doesn't seems to = filter > all the time, some days I have nothing from these IP, other days > I have a huge load of traffic. > > > mmm after all I have another question, > I posted that =20 > (http://www.pairlist.net/pipermail/flow-tools/2003-October/=20 > 001674.html) > a few weeks earlier, I changed few things since then, > but things are not fixed, I still see that error in a > strace, is that normal ? Anyone seeing this also ? > > Thanks! > > _________________________________________________ > Eric Rousse > Versus=AE > 2050, rue De Bleury, bureau 520 > Montr=E9al (Qu=E9bec) H3A 2J5 > Canada > T=E9l.: 514.284.9001 ext. 221 Fax: 514.284.9002 > _______________________________________________ > Flow-tools mailing list > flow-tools@splintered.net > http://mailman.splintered.net/mailman/listinfo/flow-tools From splintered-flow-tools-owner@splintered.net Mon Dec 08 17:25:34 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 37197 invoked by uid 4001); 8 Dec 2003 17:25:34 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 37195 invoked by alias); 8 Dec 2003 17:25:34 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 8 Dec 2003 17:25:34 -0000 In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <88160A15-29A3-11D8-81D6-00039304FAA6@eng.oar.net> Content-Transfer-Encoding: 7bit From: Mark Fullmer Subject: Re: [Flow-tools] flows dropped by acl Date: Mon, 8 Dec 2003 12:25:34 -0500 To: "Nunn, Mike (M.)" X-Mailer: Apple Mail (2.606) Cc: "'flow-tools@splintered.net'" X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2003 17:25:34 -0000 On IOS based devices access lists and RPF checks will set the output interface to 0 of the packets in the flow are being dropped. Last time I asked CAR related drops are not reflected in the flow exports. In general if you want to look for dropped packets just filter on output interface 0. mark On Dec 5, 2003, at 6:40 AM, Nunn, Mike (M.) wrote: > There seems to be conflicting evidence about Netflow reporting traffic > dropped by an acl on the exporting router, is there a definitive > answer ? > > Thanks, Mike > > > _______________________________________________ > Flow-tools mailing list > flow-tools@splintered.net > http://mailman.splintered.net/mailman/listinfo/flow-tools From splintered-flow-tools-owner@splintered.net Mon Dec 08 17:35:23 2003 Return-Path: Delivered-To: splinteredlist-flow-tools@list.splintered.net Received: (qmail 38639 invoked by uid 4001); 8 Dec 2003 17:35:22 -0000 Delivered-To: splintered-flow-tools@splintered.net Received: (qmail 38637 invoked by alias); 8 Dec 2003 17:35:22 -0000 Received: from unknown (HELO ?IPv6:::1?) (66.250.216.130) by 66.250.216.130 with SMTP; 8 Dec 2003 17:35:22 -0000 In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v606) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Mark Fullmer Subject: Re: [Flow-tools] diagnosing source of ping flood Date: Mon, 8 Dec 2003 12:35:23 -0500 To: Randy Bush X-Mailer: Apple Mail (2.606) Cc: flow list X-BeenThere: flow-tools@list.splintered.net X-Mailman-Version: 2.1.3 Precedence: list List-Id: Discussion of flow-tools software List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2003 17:35:23 -0000 There's a separate CAR mib to count dropped traffic. The port fields are used with ICMP to indicate the type and code. For example 199.18.139.136% ping 192.148.251.71 SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts AT1/0.14 199.18.139.136 Fa0/0 192.148.251.71 01 0000 0800 6 Fa0/0 192.148.251.71 AT1/0.14 199.18.139.136 01 0000 0000 6 From RFC792: Summary of Message Types 0 Echo Reply 3 Destination Unreachable 4 Source Quench 5 Redirect 8 Echo 11 Time Exceeded 12 Parameter Problem 13 Timestamp 14 Timestamp Reply 15 Information Request 16 Information Reply mark On Dec 4, 2003, at 1:53 PM, Randy Bush wrote: > so, i am experiencing a mild ping attack, only a few hundred k. but > i would like to use the experience to sort out how to diagnose this. > > it seems to be targeting a single ip address inside the lan. how > do i ask flow-tools to tell me the source ip addresses of all echo > requests toward a specific ip address? > > also, how do i know if manufacture c's box i