Pages

3/22/2020

Monitor Cisco ASA with snmp using powershell

With the entire company working remotely, the bosses want to see an hourly report of ASA connections since our capacity is limited.
  • To find the SNMP OIDs of interest I used a freeware MIB walker and lots of googling in he past 2 days about this and everything else.
  • I already had SNMPGET for the cisco backup script I've been using for years.
  • This was a good opportunity to learn more about using RRDTOOLS.  I downloaded RRD from www.rrdtool.org to use for saving the connection history and graphing it.
  • I also wanted to capture internet circuit bandwidth and utilization.  I have been using the free Solarwinds Realtime Bandwidth Monitor for this to show all the peaks that get averaged out in Orion and others.  I was able to find a powershell script that grabbed a screen shot and adjusted it to only capture the half of the server screen where the internet circuit monitors are open on,
    • It turns out that this requires that I be RDPd to this machine all the time.  So this isn't a long term kind of thing.
  • Using windows task scheduler a batch file runs that executes the powershell to take a screenshot and save it in the working folder.  Then my powershell launches. 
  • Roughly the powershell does the following:
    • create $report and then add the text of HTML BODY and TABLE headers
    • run SNMPGET to pull the svc and webvpn current connection counts from two ASA's we refer to as PRI and SEC.  svc is anyconnect client connections and webvpn is "clientless" which in my case is workspot user sessions.
    • calculate the total sessions on PRI & SEC and grand total.
    • add table rows and table data fields to $report
    • use rrdtool update to save the total to rrd database.
    • use rrdtool graph to create graph and save to working directory
    • add HTML image tags to insert the RRD graph and the screenshot image to $report
    • save $report as HTML file in working directory 
    • use send-mailmessage to e-mail $report to the recipients.  
      • NOTE:  for the images to be included in the message it is necessary to also attach them.
      • ALSO:  to specify more than one attachment or recipient, they need to be entered in quotes separated by commas.  Like:
        • "joe@cool.net", "woodstock@cool.net"
  • Code for screenprint script is below.  Followed by code for main script.

######################

# send-screen.ps1

#
#start-process "PATH TO AN APPLICATION.EXE"
#Start-Sleep -Milliseconds 1000
$File = "\util\asa\internetbw.bmp"
Add-Type -AssemblyName System.Windows.Forms
Add-type -AssemblyName System.Drawing
$Screen = [System.Windows.Forms.SystemInformation]::VirtualScreen
$Width = ($Screen.Width/2+256)
$Height = ($Screen.Height-64)
$Left = ($Screen.Left+800)
$Top = $Screen.Top
$bitmap = New-Object System.Drawing.Bitmap $Width, $Height
$graphic = [System.Drawing.Graphics]::FromImage($bitmap)
$graphic.CopyFromScreen($Left, $Top, 0, 0, $bitmap.Size)
$bitmap.Save($File) 
Write-Output $File
#$SendTo = "joe@cool.net"
#$SMTPServer = "smtp" 
#$EmailFrom = “noreply@cool.net”
#$EmailSubject = “SCREENSHOT”
#$Image = $File
#$Message = new-object Net.Mail.MailMessage
#Add-PSSnapin Microsoft.Exchange.Management.Powershell.Admin -erroraction silentlyContinue
#$attachment = new-object Net.Mail.Attachment($Image)
#$attachment.ContentId = "att"
#$smtp = new-object Net.Mail.SmtpClient($smtpServer)
#$body = ''
#$Message.From = $EmailFrom
#$Message.To.Add($SendTo)
#$Message.Subject = $EmailSubject
#$Message.Body = $body
#$Message.IsBodyHTML = $true
#$Message.Attachments.Add($attachment)
#$smtp.Send($Message)
#$attachment.Dispose()
#
#     END
#######################


##########################################
#
# CONRPT.PS1
#
# ASA Connection Report
#

$community = 'readonly'
$SEC = '10.66.1.16'
$PRI = '10.66.1.6'
$svc = '.1.3.6.1.4.1.9.9.392.1.3.35.0'
$webvpn = '.1.3.6.1.4.1.9.9.392.1.3.38.0'
$outfile = "asa-connect.html"
$logfile = "log.txt"
$rptname = "VPN Connections & Internet Bandwidth Usage"
$recipient = "matt.kunkel@troutman.com","lloyd.petrey@troutman.com"
$today = get-date

############################################################################################
# Report Heading

$report=@'
<STYLE>
BODY{font-family: Verdana, Arial, Helvetica, sans-serif;font-size:12;font-color: #000000}
TABLE{border-width: 2px;padding: 1px;border-style: solid;border-color: black;border-collapse: collapse;} 
TH{border-width: 2px;padding: 4px;border-style: solid;border-color: black;background-color: #dddddd;font-size:16;font-weight:bold}
TD{border-width: 2px;padding: 4px;border-style: solid;border-color: black;background-color: #efefef; font-size:12;font-weight:normal} 
TD.error{border-width: 2px;padding: 4px;border-style: solid;border-color: black;background-color: #ffffff;font face="monospace";font-size:10;font-color: #cccccc}
</STYLE> 
<HTML>
<HEAD> 
<TITLE></TITLE> 
</HEAD> 
<BODY>
'@

$report+="<H2>VPN Connections</H2><H4>"
$report+=$today
$report+="</H4><table><th>Connection </th><th>PRI </th><th>SEC </th></tr>"

############################################################################################




$cmd = "c:\snmp\bin\snmpget -M c:\snmp\mib -O nQ -v 2c -r 2 -t 1000 -c $community $SEC $svc"
$result = invoke-expression $cmd
$output = $result -split "= "
$anyconnectsec = $output[1]
$result=''
$output=''

$cmd = "c:\snmp\bin\snmpget -M c:\snmp\mib -O nQ -v 2c -r 2 -t 1000 -c $community $PRI $svc"
$result = invoke-expression $cmd
$output = $result -split "= "
$anyconnectpri = $output[1]
$result=''
$output=''

$cmd = "c:\snmp\bin\snmpget -M c:\snmp\mib -O nQ -v 2c -r 2 -t 1000 -c $community $PRI $webvpn"
$result = invoke-expression $cmd
$output = $result -split "= "
$workspotpri = $output[1]
$result=''
$output=''

$cmd = "c:\snmp\bin\snmpget -M c:\snmp\mib -O nQ -v 2c -r 2 -t 1000 -c $community $SEC $webvpn"
$result = invoke-expression $cmd
$output = $result -split "= "
$workspotsec = $output[1]
$result=''
$output=''

$totalPRI = [int]$anyconnectpri + [int]$workspotpri
$totalSEC = [int]$anyconnectsec + [int]$workspotsec

$report+="<tr><td>anyconnect</td><td>$anyconnectpri</td><td>$anyconnectsec</td></tr>"
$report+="<tr><td>workspot</td><td>$workspotpri</td><td>$workspotsec</td></tr>"
$report+="<tr><td>total</td><td>$totalPRI</td><td>$totalSEC</td></tr>"

$total = $totalPRI + $totalSEC

$log = "`t`t`t PRI `t SEC `n"
$log+= "anyconnect `t $anyconnectpri `t $anyconnectsec `n"
$log+= "workspot `t $workspotpri `t $workspotsec `n"
$log+= "total `t`t $totalPRI `t $totalSEC `n"
$log+= "$total `n"

$log | out-file $logfile

$now = get-date -date $today -uformat %s
$timestamp = [int]$now

& \rrd\bin\rrdtool update allcon.rrd N:$total 

& & \rrd\bin\rrdtool graph all-week.png --units-exponent 0 --start now-7d --end now DEF:ds1a=allcon.rrd:all:AVERAGE VDEF:ds1max=ds1a`,MAXIMUM LINE3:ds1a#FF0000:"Total Connections = $total" GPRINT:ds1max:"Max for Week=%5.0lf     "

$image=@'
</TABLE><img src="all-week.png"></img>
</img>
'@
$report+=$image
$report+="</BODY></HTML>" 

$report | out-file $outfile 

############################################################################################
#e-mail the report

$messageSubject = $rptname
$smtpServer = "smtp.cool.net"
$smtpFrom = "noreply@cool.net"
$smtpTo = $recipient
$message = $report 
#send-mailmessage -to $smtpTo -cc "joe@cool.net" -from $smtpFrom -subject $messageSubject -body $message -smtpserver $smtpServer -BodyAsHtml
send-mailmessage -to $smtpTo -cc "cool.net" -from $smtpFrom -subject $messageSubject -body $message -smtpserver $smtpServer -BodyAsHtml -attachments "\util\asa\all-week.png","\util\asa\internetbw.bmp"
###


2/21/2020

Multiple mutual redistribution OSPF<>BGP config

Below are router configuration fragments for redistribution between OSPF network and BGP network.


PRI-P2P

!
router ospf 1
 router-id 10.35.1.254
 area 0 authentication
 redistribute bgp 65210 subnets route-map B2O-IN
 network 10.35.1.96 0.0.0.7 area 0
 network 10.35.1.104 0.0.0.7 area 0
 network 192.168.249.254 0.0.0.0 area 0
!
router bgp 65210
 bgp log-neighbor-changes
 network 10.35.1.252 mask 255.255.255.252
 network 192.168.249.254 mask 255.255.255.255
 redistribute ospf 1 route-map O2B-OUT
 neighbor 10.35.1.253 remote-as 65211
 neighbor 10.35.1.253 password 7 xxxxxxxxxxxxxxxx
 neighbor 10.35.1.253 send-community
 neighbor 10.35.1.253 soft-reconfiguration inbound
 neighbor 10.35.1.253 route-map P2P-IN in
 neighbor 10.35.1.253 route-map P2P-OUT out
!
!IP Ranges on BGP network side.  
!
ip prefix-list B2O seq 10 permit 97.0.0.0/11 ge 16
ip prefix-list B2O seq 20 permit 10.64.0.0/11 ge 12
ip prefix-list B2O seq 30 permit 192.168.97.0/24 ge 29
!
!IP Ranges on OSPF network side
!
ip prefix-list O2B seq 10 permit 192.168.249.254/32
ip prefix-list O2B seq 20 permit 10.0.0.0/10 ge 11
ip prefix-list O2B seq 30 permit 10.96.0.0/11 ge 12
ip prefix-list O2B seq 40 permit 10.128.0.0/9 ge 10
!
!Inbound filter - only accept advertisements of networks to be distributed.
!
ip prefix-list P2P-IN-LIST seq 5 deny 0.0.0.0/0
ip prefix-list P2P-IN-LIST seq 10 permit 97.0.0.0/11 ge 16
ip prefix-list P2P-IN-LIST seq 20 permit 10.64.0.0/11 ge 12
ip prefix-list P2P-IN-LIST seq 30 permit 192.168.97.0/24 ge 29
!
!Outbound filter - do not advertise DMVPN endpoint IPs, do not advertise default, permit anything else.
!
ip prefix-list P2P-OUT-LIST seq 10 deny 192.168.255.0/24 ge 25
ip prefix-list P2P-OUT-LIST seq 15 deny 0.0.0.0/0
ip prefix-list P2P-OUT-LIST seq 20 permit 0.0.0.0/0 le 32
!
!
!
route-map B2O-IN deny 5
 match community 400  <== do not distribute BGP routes that have already been distributed.
!
route-map B2O-IN permit 7
 set tag 300   <== tag traffic that will be distributed into OSPF.

!

!
route-map B2O-IN permit 10
 match ip address prefix-list B2O
!
!
!Inbound filter - do not accept advertisements that this router will not redistribute.

!

!
route-map P2P-IN deny 5
 match community 400
!
route-map P2P-IN permit 10
 match ip address prefix-list P2P-IN-LIST
!
route-map P2P-OUT permit 10
 match ip address prefix-list P2P-OUT-LIST
!
route-map O2B-OUT deny 5
 match tag 300
!
route-map O2B-OUT permit 7
 set community 400
!
route-map O2B-OUT permit 10
 match ip address prefix-list O2B
!

PH-L3

!
router ospf 1
 router-id 172.17.1.20
 log-adjacency-changes
 area 0 authentication
 redistribute bgp 65020 metric-type 1 subnets route-map B2O-IN
 network 10.35.244.0 0.0.0.3 area 0
 network 172.17.1.0 0.0.0.255 area 0
!
router bgp 65020
 no synchronization
 bgp router-id 10.35.1.241
 bgp log-neighbor-changes
 network 10.35.1.40 mask 255.255.255.252
 network 192.168.255.20 mask 255.255.255.255
 redistribute ospf 1 route-map O2B-OUT
 neighbor 10.35.1.42 remote-as 10
 neighbor 10.35.1.42 send-community
 neighbor 10.35.1.42 soft-reconfiguration inbound
 neighbor 10.35.1.42 route-map CORE-IN in
 neighbor 10.35.1.42 route-map B2O-DENY out
 neighbor 10.35.1.245 remote-as 3549
 neighbor 10.35.1.245 password 7 XXXXXXXXXXXXXXXXXXX
 neighbor 10.35.1.245 route-map MPLS-FILTER-OUT out
 no auto-summary
!
ip prefix-list B2O seq 10 permit 97.0.0.0/11 ge 16
ip prefix-list B2O seq 90 permit 192.168.97.0/24 ge 29
ip prefix-list B2O seq 100 permit 10.64.0.0/11 ge 12
!
ip prefix-list B2O-DENY seq 10 deny 192.168.255.0/24 ge 25
ip prefix-list B2O-DENY seq 15 deny 0.0.0.0/0
ip prefix-list B2O-DENY seq 20 permit 0.0.0.0/0 le 32
!
ip prefix-list B2O-DENY2 seq 10 permit 192.168.255.0/24 ge 25
ip prefix-list B2O-DENY2 seq 20 permit 0.0.0.0/0
!
ip prefix-list MPLS-OUT seq 10 permit 192.168.255.0/24 ge 25
!
ip prefix-list O2B seq 10 permit 10.10.0.0/16
ip prefix-list O2B seq 20 permit 10.1.0.0/16
ip prefix-list O2B seq 30 permit 10.6.1.0/24
!
!
!
!
route-map B2O-DENY permit 10
 match ip address prefix-list B2O-DENY
!
route-map B2O-IN deny 5
 match community 400
!
route-map B2O-IN deny 6
 match ip address prefix-list B2O-DENY2
!
route-map B2O-IN permit 7
 set tag 300
!
route-map B2O-IN permit 10
 match ip address prefix-list B2O
!
route-map CORE-IN deny 5
 match community 400
!
route-map CORE-IN permit 10
 match ip address prefix-list B2O
!
route-map MPLS-FILTER-OUT deny 10
 match community 400
!
route-map MPLS-FILTER-OUT deny 20
 match ip address prefix-list B2O
!
route-map MPLS-FILTER-OUT permit 30
 match ip address prefix-list MPLS-OUT
!
route-map O2B-OUT deny 5
 match tag 300
!
route-map O2B-OUT permit 7
 set community 400
!
route-map O2B-OUT permit 10
 match ip address prefix-list O2B
!
route-map DENY-OUT deny 5
 match community 400
!

Multiple Mutual Redistribution OSPF <> BGP

Nearing the close of a big project.  Connectivity for two nearly equal sized networks:
First is OSPF overlay on dual service provider MPLS WANs advertising a single IP via BGP.  These IP's are used to establish DMVPN connectivity to hub sites.
The other has a very simplified remote office with just a couple VLANs defaulting to a core switch.  That core switch is a BGP neighbor with the single MPLS service provider router. 

Design Goals

1.     Add point to point connection between new routers PRI-P2P and PH-P2P
2.     Add MPLS to Philadelphia site connected to new router PH-L3
3.     Route user traffic primarily over MPLS
4.     Use point to point connection for communication between specific datacenter systems with policy routing.
5.     If MPLS path fails, user traffic should be routed over point to point circuit automatically.
6.     Redistribute OSPF into BGP on:  PRI-P2P & PH-L3.
7.     Redistribute BGP into OSPF on:  PRI-P2P & PH-L3.
8.     Do not redistribute BGP into OSPF for subnets only used for DMVPN.
9.     Prevent redistribution from creating routing loops.  That is, do not allow redistribution of routes that have been redistributed already by the same or another router.
10.  Avoid asynchronous routes between hosts
11.  Do not redistribute default routes

Things I have learned

I have a fair amount of experience but have never done much redistribution.  I had to learn a lot in order to make this work.  I got help from vendor(s) and colleagues.  Did a lot of googling and reading.  The following is an unorganized list of things that I noticed or that helped me once I realized them.
  • Route redistribution might be better called route import because redistribution into a routing protocol is done in the configuration of the protocol intended to receive routes.  That is, ospf configuration would be added to redistribute "into OSPF".
  • Route redistribution does not put routes into the routing table on the router doing the redistribution.
  • Redistribution isn't really the hard part.  It is about figuring out the routing protocols for troubleshooting and manipulating them to avoid loops and so forth.
    • Route filtering
    • Route tagging
    • In addition to routing table, there will be a database of potential routes--only one of which will go into the routing table.  (show ip bgp)
    • Metric
    • I also assume that redistribution behavior has different rules/behaviors according to what routing protocol is pulling in the routes.
    • BGP commands to show route advertisements sent and received
    • timer adjustments.
  • I cannot found an example on internet of mutual redistribution between OSPF & BGP.
  • BGP doesn't have TAGs.  Uses COMMUNITY.  But the idea is the same.
  • Everything seems to work as desired when a state is achieved where, on the router doing redistribution, the routing table holds OSPF routes for all the "native" OSPF locations and BGP routes for all the "native" BGP routes.
  • To prevent routing loops and achieve the state described above it is important to filter route advertisements in/out in addition to applying tagging to the redistribution config.
  • Use traffic steering to determine primary paths.  i.e. OSPF cost & BGP path length.  Remember that OSPF cost required to make this happen could affect the overall design of the network if costs are involved and some existing OSPF configuration may need adjusted on existing more routers.  In TS case, a bigger OSPF cost is needed for sites that are manually configured with higher cost to prefer the TW path.
  • The routes for the BGP MPLS IP's used for DMVPN must not get into the "overlay" routing table.  That is, if BGP routes for the loopbacks used to establish the DMVPN tunnels get into OSPF then OSPF stops working.  "mid-chain attempting to stack…."
  • BGP
show ip bgp neighbors [IP of neighbor] advertised-routes
show ip bgp neighbors [IP of neighbor] received-routes
show ip bgp

Google Chrome QUIC Protocol

UDP/443

Noticed a lot of UDP/443 traffic?

Check out this article about QUIC experimental protocol that Google is using.


Interesting Chrome hacks:
Chrome Flags

net-internals