NetFlow for Catalyst 4500

ccie-rs July 20th, 2009

I had the pleasure of integrating brand new 4500s into production Friday night to alleviate performance bottlenecks on our 3845s.  For quite some time, we’ve been pushing the boundaries of UDP file transfers over our existing routers utilizing a product called Aspera.  While you may think that UDP is unreliable and connectionless, the folks at Asperasoft have devised a way to conduct transfers reliably.  There are a few kinks with the software, but the results are pretty impressive.  Just imagine transferring 1TB of data over a 1Gb connection from LA to NY in less than a day.  To increase the routing threshold, we had to migrate off the 3845s and introduce the 4506-E switches.  The transition went very smoothly and was completed within the maintenance window, which is a rare feat at times.

Once all the hardware was in place I cleaned up our monitoring devices and alerts with new interface information off the switches.  I quickly activated NetFlow and didn’t think anything of it at the time.  On Sunday, I had some time to VPN in and check on the performance of the new switches.  I found out that our ManageEngine software only detected 1 interface to monitor.  That wasn’t going to do me any good, so I decided to do some research.  I checked the config first to make sure I entered the commands correctly:

CRS01#sh run | i flow
ip flow-cache timeout active 1
ip flow-export source Vlan119
ip flow-export version 5
ip flow-export destination 172.16.119.194 9996
ip flow-top-talkers
top 25
sort-by bytes
ip route-cache flow

Looks correct so far.  According to Cisco’s Catalyst 4500 Series Switch Software Configuration Guide, 12.2(50)SG documentation, you should verify that the NetFlow Services daughter card (WS-F4531) is working and available:

CRS01#sh module all
Chassis Type : WS-C4506-E

Power consumed by backplane : 0 Watts

Mod Ports Card Type                              Model              Serial No.
—+—–+————————————–+——————+———–
1     6  Sup V-10GE 10GE (X2), 1000BaseX (SFP)  WS-X4516-10GE      XXXXXXXXXXX
2     6  SFP, 10/100/1000BaseT (RJ45)V, Cisco/I WS-X4506-GB-T      XXXXXXXXXXX
3    48  10/100/1000BaseT (RJ45)V, Cisco/IEEE   WS-X4548-GB-RJ45V  XXXXXXXXXXX

M MAC addresses                    Hw  Fw           Sw               Status
–+——————————–+—+————+—————-+———
1 0024.1423.78c0 to 0024.1423.78c5 5.1 12.2(31r)SGA 12.2(50)SG1      Ok
2 001b.5481.a2dc to 001b.5481.a2e1 1.4                               Ok
3 0021.a0de.dfc0 to 0021.a0de.dfef 4.1                               Ok

This next section I’m copying from the documentation was missing completely:

Mod  Submodule               Model             Serial No.   Hw   Status

—-+———————–+—————–+————+—-+———

1   Netflow Services Card   WS-F4531          JAB062209CG  0.2  Ok

2   Netflow Services Card   WS-F4531          JAB062209AG  0.2  Ok

After reading around in a few forums, I found that the NetFlow Services Card submodule is built into my Supervisor Engine V-10GE and may not appear when running the command.  I continued to read through the documentation and found that to activate NetFlow for the 4500 I should run the global command:

ip flow ingress

The moment you enter this command, additional command options start to appear in the IOS.

ip flow ingress infer-fields
ip flow ingress layer2-switched

Adding these additional commands still didn’t reveal all the interfaces I wanted to monitor within NetFlow Analyzer so I had to conduct more research.  I finally stumbled upon an additional command:

ip route-cache flow infer-fields

The infer-fields option doesn’t appear when you use the question mark, but the IOS accepts the command.  As soon as I hit enter, my analyzer paged refreshed and all the interfaces appeared.  I wanted to inquire more about the option and Googled ‘ip route-cache flow infer-fields’ and found ManageEngines supporting documentation for Configuring NDE.

If I just continued reading Cisco’s documentation, I would have found the command in the example section.

ASA 8.2 Released

ccie-sec May 1st, 2009

A few weeks ago, Cisco released version 8.2 for the ASA, which you can read more about the updated features here.  For me, one of the most appealing features that stands out the most is the NetFlow Secure Event Logging feature (NSEL).   Originally this feature was only made available on version 8.1 specifically for the 5580 models in January of this year.  Now it’s available from the 5505 models up to the 5580.

What’s the big deal?  Currently I’m dealing with a configuration where an Internet connection terminates directly on an ASA 5520.  Every month I have to deal with senior management inquiring about which department is consuming the most bandwidth so that finance can appropriately charge back the usage internally.  In order to appropriately collect this data, I had proposed to place a router in front of the ASA so that I could collect NetFlow statistics and report the information properly.  With NSEL on the ASA, I’d be able to collect the information directly without any additional equipment.

Hmm, if everything works as planned, I’ll have to figure out what to do with all these spare 3845 edge routers.  :)

Flexible NetFlow

ccie-sec February 26th, 2009

A while back I installed a router in our client’s facility and managed to establish a L2L firewall VPN connection to this remote device.  On top of the management traffic that is passed, we are securely transferring FTP content between our 2 locations.  The problem with this arrangement is that I’m unable to intrusively monitor the device for application flows.  I can see content exit my office, but can’t really verify whether the remote end is receiving the data or not.

Since the implementation, I’ve been toying around with router configurations on sending SNMP and NetFlow across the VPN so I can monitor the router from my NMS.  SNMP is working fine, but NetFlow doesn’t seem to pass the information across the VPN.  I found an old bug documented on Cisco’s website regarding my problem:

CSCef28662 - Self-generated Netflow packets not encrypted

Problem was reported last summer against 12.1E software, but 12.3 mainline also behaves in the same way. Your case will be the 6th attached to that problem. Currently no DE is assigned to resolve it yet….
I think the best you can do currently is to change the design accordingly.
If you feel that problem is very important for your business - then it makes sense to engage Cisco SE, so they can push DE to implement enhancements.

One workaround for me would be to migrate the termination off the firewall and onto another router with a VTI, but that really isn’t an option for me at this moment.  The other option at this point is Flexible NetFlow, which allows me the ability to export the NetFlow cache across the tunnel to my NMS.  If you want to learn more about the other features available, I recommend reading the white paper.  I’m just going to continue forward with the one feature I need right now.

Step 1: Configuring a Flow Exporter

Description: A flow exporter defines where the flow monitor cache will be delivered (i.e. NetFlow collector).

Configuration:

flow exporter FLEXFLOW-EXPORT
export-protocol netflow-v5 **This command appears in IOS 12.4(22)T.  Default is v9.**
source gi 0/1.25
destination 10.2.2.10
transport udp 2055 **This is the UDP port that my NMS is collecting flows on.**

Verification:

show flow exporter

Flow Exporter FLEXFLOW-EXPORT:
Description:              User defined
Export protocol:          NetFlow Version 5
Transport Configuration:
Destination IP address: 10.2.22.10
Source IP address:      172.16.2.13
Source Interface:       GigabitEthernet0/1.25
Transport Protocol:     UDP
Destination Port:       2055
Source Port:            57409
DSCP:                   0×0
TTL:                    255
Output Features:        Not Used

Step 2: Configuring a Flow Monitor

Description: A flow monitor requires a record to define the contents and layout of its cache entries.  For my example, I’ll be using the default record.

Configuration:

flow monitor FLEXFLOW
record netflow ipv4 original-input
exporter FLEXFLOW-EXPORT

Verification:

show flow monitor

Flow Monitor FLEXFLOW:
Description:       User defined
Flow Record:       netflow ipv4 original-input
Flow Exporter:     FLEXFLOW-EXPORT
Cache:
Type:              normal
Status:            allocated
Size:              4096 entries / 311316 bytes
Inactive Timeout:  15 secs
Active Timeout:    1800 secs
Update Timeout:    1800 secs

Step 3: Applying Flexible NetFlow to an interface

Description: Activates Flexible NetFlow.

Configuration:

int gi 0/1.25
ip flow monitor FLEXFLOW input

Verification:

show flow interface gi 0/1.25

Interface GigabitEthernet0/1.25
FNF:  monitor:         FLEXFLOW
direction:       Input
traffic(ip):     on

Step 4: View the flow monitor cache

Description: Display the stats of flow data.

Configuration: N/A

Verification:

show flow monitor name FLEXFLOW cache format record

Cache type:                            Normal
Cache size:                              4096
Current entries:                            7
High Watermark:                            10

Flows added:                             1628
Flows aged:                              1621
- Active timeout   (  1800 secs)          3
- Inactive timeout (    15 secs)       1618
- Event aged                              0
- Watermark aged                          0
- Emergency aged                          0

IP TOS:                    0×08
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.22.2
TRNS SOURCE PORT:          42078
TRNS DESTINATION PORT:     35164
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×10
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           963071724
timestamp last:            964314328
counter packets:           493005
counter bytes:             27953356

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.2.151
TRNS SOURCE PORT:          40001
TRNS DESTINATION PORT:     54964
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×1B
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964299824
timestamp last:            964299924
counter packets:           6
counter bytes:             747

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.22.2
TRNS SOURCE PORT:          21
TRNS DESTINATION PORT:     54196
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×10
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964302796
timestamp last:            964302796
counter packets:           1
counter bytes:             52

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.2.151
TRNS SOURCE PORT:          40001
TRNS DESTINATION PORT:     54983
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×1B
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964304136
timestamp last:            964304224
counter packets:           6
counter bytes:             747

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.2.151
TRNS SOURCE PORT:          40001
TRNS DESTINATION PORT:     55002
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×1B
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964308632
timestamp last:            964308736
counter packets:           6
counter bytes:             747

IP TOS:                    0×00
IP PROTOCOL:               17
IPV4 SOURCE ADDRESS:       172.16.2.12
IPV4 DESTINATION ADDRESS:  10.2.2.54
TRNS SOURCE PORT:          161
TRNS DESTINATION PORT:     1045
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×00
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964309744
timestamp last:            964310308
counter packets:           4
counter bytes:             288

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.2.151
TRNS SOURCE PORT:          40001
TRNS DESTINATION PORT:     55021
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×1B
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964313160
timestamp last:            964313276
counter packets:           6
counter bytes:             747

As you can see, the flow cache is starting to capture my content traffic.  Now all I need to do is confirm my NMS is receiving this information.

blank