[Flow-tools] Easy question, just need a hint..

[Benjamin Bach]

Dear all,

I frequently process flow-files to make copies that are easier to
digest for statistical purposes (e.g. I split source data per subnet).
But it doesn't seem to work. The first few days I have reasonable
results, but after a while, the files get corrupted (I think), because
flow-report says some terrible stuff about them (see attachment).

I have some thoughts that executing "the below command" in order to
add new flows to a flow-file is not the way, I just need confirmation
or the opposite.

# flow-cat -t last_run -T now /src/of/flow/data | flow-filter -k -o -f
myfilter -S subnetname -D subnetname >> subnet_file

What I do is that I use bash to concatenate my new flow-data to a
file. This file is producing sick results after a few days (I run the
above command every hour).

I haven't been able to reproduce the behavior, since it would take way
too much brute force, and I'm working on limited time. But I do know
that I can create a new data file for each subnet net all-at-once and
have a working file.. so it must be my concatenation of data that
fails.

/ Benjamin
-------------- next part --------------
shell:~# cat subnet-file | flow-report 
#  --- ---- ---- Report Information --- --- ---
# build-version:        flow-tools 0.68
# name:                 default
# type:                 summary-detail
# options:              +header,+xheader,+totals
# fields:               +other
# records:              0
# first-flow:           0 Thu Jan  1 01:00:00 1970
# last-flow:            4294928805 Wed Dec 31 14:18:29 1969
# now:                  1186400155 Mon Aug  6 13:35:55 2007
#
# mode:                 streaming
# capture start:        Wed Jul 11 00:00:01 2007
# capture end:          Fri Jul 13 12:30:00 2007
# capture period:       217799 seconds
# compress:             off
# byte order:           little
# stream version:       3
# export version:       5
#
#  ['/usr/bin/flow-rptfmt', '-f', 'ascii']
Ignores:                 457428
Total Flows:             1448045679
Total Octets:            732243564789594073
Total Packets:           1476931828582490236
Total Duration (ms):     2479037333939366180
Real Time:               4294928805
Average Flow Time:       1711988350.000000
Average Packets/Second:  0.000000
Average Flows/Second:    505677117.000000
Average Packets/Flow:    1019948369.000000
Flows/Second:            359.626035
Flows/Second (real):     0.337152