Mfcttrf

This is my current setup where i have 40Gbps links between all 4 switches running OSPF using L3 links between them but now i want to double my bandwidth between switches so i am planning to add (dotted links) L3 links and let OSPF load-balance traffic on them, Do you think is there any issue to doing that or this is going to be just fine? ( want second set of eyes )

This is what my ospf config looks on all 4 switches.

interface Ethernet2/10
 no switchport
 mtu 9216
 ip address 192.168.250.9/30
 no ip ospf passive-interface
 ip router ospf 100 area 0.0.0.0
 no shutdown

interface Ethernet2/11
 no switchport
 mtu 9216
 ip address 192.168.250.13/30
 no ip ospf passive-interface
 ip router ospf 100 area 0.0.0.0
 no shutdown

As these are Point-to-Point links, I would consider using the outage to configure each /30 interface with ip ospf network point-to-point. (New and existing links). This reduces the hello and dead timers. This configuration also reduces the need to negotiate a DR and BDR.

Lastly, I would verify the OSPF neighbor states and routing tables, before and after the cutover. You should see ECMP routes after the cutovers and the appropriate neighborships.

There's two ways to do that.

• your proposed way, by adding a second link with its own /30 or /31,
  making sure that OSPF installs multiple equal cost routes in the
  routing table, and let CEF's ECMP (EqualCostMultiPath) forwaring handle the
  packet pushing and the distribution of flows across the set
  of available links. CEF/ECMP uses a different load sharing logic than Port-Channel, and can handle uneven numbers of links a lot better than Port-Channels do. See Ivan Pepelniak's Blog post for reference.




• use L3 Port-Channels: move the IP and routing configuration to a port-channel object (which does not have the "switchport" command), and join the given interfaces to that object. Let Port-Channel load distribution logic handle the distribution of flows.

Your proposed idea is more L3/routing oriented, scaling it might have some issues: You will use lot of /30 or /31s. You can scale in odd numbers, but you'll have to configure a new link and possibly subnet for every scaling step (or go ip unnumbered). ules for Port-Channels apply (e.g. keep number of links in powers of 2).

On the other hand, L3 Port-Channels don't need more IP subnets and don't actually touch your given routing logic. Port-Channel is a bit more "Nexus style", as the whole history of Nexus switches is based on the VPC concept (which does not quite apply here, I'll admit). Scaling of additional links is easier - just add two more, without touching any IP or routing configuration.

ADDON-1: And oh yes, by all means DO follow TDurden's advice to configure point-to-point on the ... ehm.. point-to-point links (really bad pun, I'll admit)

CAVEAT: When using port-channels ensure that you select a load-balancing strategy that fits the expected communication patterns for the given link. When connecting a router to a router (essentially only two MAC addresses on the link), then "Src/Dst MAC" might not have the desired results... For reference, see the Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 9.2(x)

ADDON-2:
On Nexus 9000, with ECMP/CEF, the load-sharing algorithm can be configured to include L4 properties: ip load-sharing address source-destination port source-destination See Configuring Load-Sharing in the Unicast FIB from the Unicast Routing configuration gude.