A while back I installed a router in our client’s facility and managed to establish a L2L firewall VPN connection to this remote device.  On top of the management traffic that is passed, we are securely transferring FTP content between our 2 locations.  The problem with this arrangement is that I’m unable to intrusively monitor the device for application flows.  I can see content exit my office, but can’t really verify whether the remote end is receiving the data or not.

Since the implementation, I’ve been toying around with router configurations on sending SNMP and NetFlow across the VPN so I can monitor the router from my NMS.  SNMP is working fine, but NetFlow doesn’t seem to pass the information across the VPN.  I found an old bug documented on Cisco’s website regarding my problem:

CSCef28662 - Self-generated Netflow packets not encrypted

Problem was reported last summer against 12.1E software, but 12.3 mainline also behaves in the same way. Your case will be the 6th attached to that problem. Currently no DE is assigned to resolve it yet….
I think the best you can do currently is to change the design accordingly.
If you feel that problem is very important for your business - then it makes sense to engage Cisco SE, so they can push DE to implement enhancements.

One workaround for me would be to migrate the termination off the firewall and onto another router with a VTI, but that really isn’t an option for me at this moment.  The other option at this point is Flexible NetFlow, which allows me the ability to export the NetFlow cache across the tunnel to my NMS.  If you want to learn more about the other features available, I recommend reading the white paper.  I’m just going to continue forward with the one feature I need right now.

Step 1: Configuring a Flow Exporter

Description: A flow exporter defines where the flow monitor cache will be delivered (i.e. NetFlow collector).

Configuration:

flow exporter FLEXFLOW-EXPORT
export-protocol netflow-v5 **This command appears in IOS 12.4(22)T.  Default is v9.**
source gi 0/1.25
destination 10.2.2.10
transport udp 2055 **This is the UDP port that my NMS is collecting flows on.**

Verification:

show flow exporter

Flow Exporter FLEXFLOW-EXPORT:
Description:              User defined
Export protocol:          NetFlow Version 5
Transport Configuration:
Destination IP address: 10.2.22.10
Source IP address:      172.16.2.13
Source Interface:       GigabitEthernet0/1.25
Transport Protocol:     UDP
Destination Port:       2055
Source Port:            57409
DSCP:                   0×0
TTL:                    255
Output Features:        Not Used

Step 2: Configuring a Flow Monitor

Description: A flow monitor requires a record to define the contents and layout of its cache entries.  For my example, I’ll be using the default record.

Configuration:

flow monitor FLEXFLOW
record netflow ipv4 original-input
exporter FLEXFLOW-EXPORT

Verification:

show flow monitor

Flow Monitor FLEXFLOW:
Description:       User defined
Flow Record:       netflow ipv4 original-input
Flow Exporter:     FLEXFLOW-EXPORT
Cache:
Type:              normal
Status:            allocated
Size:              4096 entries / 311316 bytes
Inactive Timeout:  15 secs
Active Timeout:    1800 secs
Update Timeout:    1800 secs

Step 3: Applying Flexible NetFlow to an interface

Description: Activates Flexible NetFlow.

Configuration:

int gi 0/1.25
ip flow monitor FLEXFLOW input

Verification:

show flow interface gi 0/1.25

Interface GigabitEthernet0/1.25
FNF:  monitor:         FLEXFLOW
direction:       Input
traffic(ip):     on

Step 4: View the flow monitor cache

Description: Display the stats of flow data.

Configuration: N/A

Verification:

show flow monitor name FLEXFLOW cache format record

Cache type:                            Normal
Cache size:                              4096
Current entries:                            7
High Watermark:                            10

Flows added:                             1628
Flows aged:                              1621
- Active timeout   (  1800 secs)          3
- Inactive timeout (    15 secs)       1618
- Event aged                              0
- Watermark aged                          0
- Emergency aged                          0

IP TOS:                    0×08
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.22.2
TRNS SOURCE PORT:          42078
TRNS DESTINATION PORT:     35164
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×10
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           963071724
timestamp last:            964314328
counter packets:           493005
counter bytes:             27953356

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.2.151
TRNS SOURCE PORT:          40001
TRNS DESTINATION PORT:     54964
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×1B
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964299824
timestamp last:            964299924
counter packets:           6
counter bytes:             747

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.22.2
TRNS SOURCE PORT:          21
TRNS DESTINATION PORT:     54196
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×10
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964302796
timestamp last:            964302796
counter packets:           1
counter bytes:             52

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.2.151
TRNS SOURCE PORT:          40001
TRNS DESTINATION PORT:     54983
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×1B
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964304136
timestamp last:            964304224
counter packets:           6
counter bytes:             747

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.2.151
TRNS SOURCE PORT:          40001
TRNS DESTINATION PORT:     55002
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×1B
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964308632
timestamp last:            964308736
counter packets:           6
counter bytes:             747

IP TOS:                    0×00
IP PROTOCOL:               17
IPV4 SOURCE ADDRESS:       172.16.2.12
IPV4 DESTINATION ADDRESS:  10.2.2.54
TRNS SOURCE PORT:          161
TRNS DESTINATION PORT:     1045
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×00
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964309744
timestamp last:            964310308
counter packets:           4
counter bytes:             288

IP TOS:                    0×00
IP PROTOCOL:               6
IPV4 SOURCE ADDRESS:       172.16.2.10
IPV4 DESTINATION ADDRESS:  10.2.2.151
TRNS SOURCE PORT:          40001
TRNS DESTINATION PORT:     55021
INTERFACE INPUT:           Gi0/1.25
FLOW SAMPLER ID:           0
interface output:          Null
ipv4 next hop address:     0.0.0.0
tcp flags:                 0×1B
ip source as:              0
ip destination as:         0
ipv4 source mask:          /29
ipv4 destination mask:     /0
timestamp first:           964313160
timestamp last:            964313276
counter packets:           6
counter bytes:             747

As you can see, the flow cache is starting to capture my content traffic.  Now all I need to do is confirm my NMS is receiving this information.

Last year I ended up upgrading my Blackjack i607 cell phone to a Blackberry 8820. Basically I did this so it was easier to manage emails from my various contracts. I don’t really have to go into the basic functions of the phone, since many of you may already have one joined at your hip. One of the software packages I would recommend downloading and installing on your phone is called MidpSSH — it’s a free telnet/SSH client that works pretty well and will allow you to manage your equipment. I installed and tested a few others out there, but many of the free versions had too many bugs.

Since my Blackberry was already tied to the client’s Blackberry server to service my email portion, I had the ability to browse and conduct most management functions (i.e. ping checks) over the Internet. When managing a device with this utility, you have to realize you won’t be able to do so with blazing speeds. In fact, it may seem like you are running over a dial-up connection when you first try it out, but in the end it may still prove useful for any quick fix.

Yesterday while I was in transit, I received a call for support to look at a core router. The client was trying to determine the optimal routing path for several servers through traceroute. Unfortunately, the client found that in passing ICMP packets to a certain core router in the path that he was unable to receive the echo-replies. I quickly remembered that there was a DENY_ICMP access-list assigned on the interface that was most likely prohibiting ICMP all together and needed to figure out the quickest way to remove it.

(On a side note, one of my personal goals this year was to minimize my own clutter. One of the ways of achieving that was to reduce the amount of hardware I carried around in my pockets and my backpack, which meant my laptop and the many cables/accessories that came with it will now sit at home.)

To give you some background of the client, they have an external monitoring service co-located in downtown LA. One of the devices in the co-lo is a router that has a VPN tied back to the client office and allows telnet access on the external interface from the Blackberry server IP address.

In order to access the equipment, I opened up the MidpSSH utility on the Blackberry, selected ‘Sessions’, typed in the IP address of the co-lo router, and selected ‘Connect’. After entering my credentials, I was immediately logged into the co-lo router. From this point, I had to reverse telnet (with the /source-interface option) to a distribution switch that was connected to the core router. From the distribution switch I was able reverse SSH into the core router and remove the ACL from the interface.

All in all, it took me only 15 minutes.