Category Archives: Networking

Failed to connect to IKEv2 VPN using iPhone USB tethering

Usually I tend to use the iPhone WiFi hotspot feature. But lately, I had to switch to USB tethering, because I had to work a whole workday using the hotspot feature. USB tethering saves battery and the connection was more reliable for me. Please note, that you need to install iTunes to use USB tethering, because the necessary Ethernet driver is only available with iTunes. Without this driver, Windows won’t recorgnize the iPhone as an Ethernet connection.

While using USB tethering I noticed that my IKEv2 VPN connection to my office wasn’t working. I use the native Windows 11 VPN client. At the office operates a WatchGuard T80 firewall with TotalSecurity Subscription. Interestingly the VPN connection was working fine with the WiFi hotspot. I double-checked it with another IKEv2 connection to a customer. This connection showed the same issue. Won’t work with USB tethering, connection was fine using the WiFi hotspot.

Troubleshooting

First things first: The traffic log showed some interesting facts. The connection attempt was recognized by the firewall.

2023-06-24 16:22:24iked(212.117.xx.yy<->91.41.xx.yy)The peer is behind NAT
2023-06-24 16:22:24iked(212.117.xx.yy<->91.41.xx.yy)The local is NOT behind NAT
2023-06-24 16:22:24iked(212.117.xx.yy<->91.41.xx.yy)Processed IKE_SA_INIT request message successfully
2023-06-24 16:22:24iked(212.117.xx.yy<->91.41.xx.yy)'IKE_SA_INIT response' message created successfully. length:496
2023-06-24 16:22:24iked(212.117.xx.yy<->91.41.xx.yy)Sent out IKE_SA_INIT response message (msgId=0) from 212.117.xx.yy:500 to 91.41.xx.yy:64172 for 'WG Default IKEv2 Gateway' gateway endpoint successfully.


2023-06-24 16:22:43iked(212.117.xx.yy<->80.187.xx.yy)The peer is behind NAT
2023-06-24 16:22:43iked(212.117.xx.yy<->80.187.xx.yy)The local is NOT behind NAT
2023-06-24 16:22:43iked(212.117.xx.yy<->80.187.xx.yy)Processed IKE_SA_INIT request message successfully
2023-06-24 16:22:43iked(212.117.xx.yy<->80.187.xx.yy)'IKE_SA_INIT response' message created successfully. length:496
2023-06-24 16:22:43iked(212.117.xx.yy<->80.187.xx.yy)Sent out IKE_SA_INIT response message (msgId=0) from 212.117.xx.yy:500 to 80.187.xx.yy:500 for 'WG Default IKEv2 Gateway' gateway endpoint successfully.

The uppper connection attempt was successfull. You might recorgnize the port used for the destination IP for the IKE_SA_INIT. The lower attempt was using USB tethering and it wasn’t successfull. In this case the connection attempt was made to 500/udp.

This is a Wireshark capture of the unsuccessful connection attempt.

This capture is from a successful attempt.

You will notice the difference after the IKE_AUTH MID=01 Initiator Request (Frame 620 and 1248). The response of the firewall is not received by the client. This behavior often is caused by MTU problems. A quick Google search showed evidence, that USB tethering might behave different from WiFi hotspot.

Solution

Connect your iPhone using an USB-to-Lightning cable. A new Ethernet device should come up. Open an elevated CMD and use the following command to adjust the MTU for the Apple Ethernet device.

PS C:\Users\adm-terlisten> netsh interface ipv4 show interfaces

Idx     Met         MTU          State                Name
---  ----------  ----------  ------------  ---------------------------
  1          75  4294967295  connected     Loopback Pseudo-Interface 1
 10          50        1500  disconnected  WLAN
 19          25        1500  disconnected  LAN-Verbindung* 1
  9          25        1500  disconnected  LAN-Verbindung* 2
 11          25        1500  connected     Ethernet 2
  2          65        1500  disconnected  Bluetooth-Netzwerkverbindung
 20          35        1432  connected     iPhone Hotspot

PS C:\Users\adm-terlisten> netsh interface ipv4 set subinterface "iPhone Hotspot" mtu=1472 store=persistent
OK.

I renamed my Apple Ethernet device, in your case it could something like “Ethernet 4” or similar. That’s it. Enjoy your VPN connection.

Fun fact: Cisco AnyConnect with an IKEv2 connection had no problem to any time, regardless if WiFi hotspot or USB tethering. I encountered the problem only with the native Windows VPN client.

Wartungsfenster Podcast

This posting is ~2 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Ausnahmsweise ein Blogpost in deutscher Sprache. Grund dafür ist, dass Claudia Kühn und ich seit Januar 2022 einen gemeinsamen Podcast rund um den Themenkomplex Datacenter, Cloud und IT ein. Eine lockere Kaminzimmerrunde in der wir entspannt über unseren Job, und alles was damit zu tun hat, plaudern.

Der Podcast erscheint alle zwei Wochen auf den üblichen Kanälen, oder ihr schaut auf der Homepage des Podcasts vorbei. Lasst gernen einen Kommentar/ Feedback da, und gebt uns eine Bewertung auf iTunes.

WatchGuard Network Security Essentials Exam

This posting is ~3 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Yesterday, I passed the first exam of the year. In this case the WatchGuards Network Security Essentials exam. The exam covers basic networking and firewalling skills, as well as the necessary knowledge to configure, manage, and monitor a WatchGuard Firebox. If you were familier with networking and firewalls in general, this exam is a “low hanging fruit”. I had to take it due to partner conditions.

WatchGuards offers a pretty good study guide for this exam which you can get for free.. The exam is delivered by  Kryterion and can be taken in a test center or as online proctored exam.

The closed book exam consists of 70 questions. You have 2h and you need at least 70% to pass the exam. The exam covers six different topics:

  • Network and network security basics
  • Administration and setup
  • Monitoring, logging, and reporting
  • Networking and NAT
  • Policies, proxies, and security services
  • Authentication and VPN

I passed the exam with a some preparation (I’ve only used the study guide). As long as you have experience with WatchGuard firewalls, which is mandartory IMHO, it is sufficient to read the study guide a couple of times.

EAPoL forwarding on NEC VoIP phones

This posting is ~5 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

A customer is running their PCs behind their VoIP phones. Nothing unusual, most VoIP phones I know have an embedded ethernet switch, so that you only need one cable to connect PC and VoIP phone to your network.

Martinelle/ pixabay.com/ Creative Commons CC0

As part of a network security project, my colleague and I implemented IEEE 802.1X port-based Network access control at one of our customers networks. The setup consists of multiple Alcatel-Lucent Enterprise OmniSwitches (6450-P10 and 6860/E) and Aruba ClearPass.

We noticed, that mac-address based authentication worked all the time, but 802.1x fails constantly if the client was connected to a VoIP phone (NEC DT700). The phones does not do any port authentication. We use a device classification rule and User Network Profiles to get them to their correct VLAN. But the connected PCs should do a 802.1x based port authentication.

Wireshark FTW!

We used Wireshark to take a look at the communication. We created a packet trace on a client behind a VoIP phone, and we mirrored the traffic of the port, to which the phone was connected. Our assumption was that the VoIP phones drop the EAP packets from the connected PC.

This is a packet trace from my ThinkPad X250 which was connected to the phone.

Patrick Terlisten/ vcloudnine.de/ Creative Commons CC0

You can see the repeating “Request, Identity” from the switch, and the answer from my laptop (Response, Identity). The destination for the response is a multicast mac-address. But this frame was not captured behind the VoIP phone! It was missing. On the packet trace, that was created my mirroring the switch port to which the phone was connected, the “Request, Identity” was seen, but not the “Response, Identity”. The phone was dropping the EAP packets of my laptop!

RTFM!

The customer called the company who was maintaining the phones. But they did not understood our problem, so they enabled 802.1x on the phones. We disabled this instantly again.

I decided to take a look into the manual of the NEC DT700 and I found a point called “EAPoL forwarding” in the advanced network settings. After enabling this setting, EAP started working instantly.

Patrick Terlisten/ vcloudnine.de/ Creative Commons CC0

This is again a packet trace from my laptop, taken while it was connected to a VoIP phone. As you can see, the last EAP packet is “Success”!

EAPoL forwarding did the trick. :)

Bypass stateful firewall on a Sophos XG

This posting is ~5 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Usually, bypassing a firewall is not the best idea. But sometimes you have to. One case, where you want to bypass a firewall, is asymmetric routing.

MichaelGaida/ pixabay.com/ Creative Commons CC0

What is asymmetric routing? Imagine a scenario with two routers on the same network. One router offeres access to the internet, the other router provides access to other sites with site-2-site VPN tunnels.

Asymmetric Routing

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

Host 1 uses R1 as default gateway. R1 has static routes configured to the networks reachable over the VPN, or it has learned them dynamically using a routing protocol from R2. A packet from host 1 arrives at R1, is routed to R2, and is sent over the VPN tunnel. The answer to this packet arrives at R2, and is sent directly to host 1, because host 1 is the destination. This works because R2 and host 1 are on the same network. This is asymmetric routing, because request and answer go different ways.

In case of routing, this is not a problem. But if R1 is a firewall, this firewall might be stubborn, because it does not see the whole traffic.

Bypass the stateful firewall

I recently had such a setup due to some technical debts. The firewall dropped that “Invalid Traffic”. Fortunately, there is a way to bypass the statefull firewall. You can create advanced firewall rules using the CLI. There is no way to create these rules using the GUI. And this only applies to the Sophos XG (former Cyberoam products).

Login to the device console and select option 4. Then enter on the console the following commands, one per destination:

Console> set advanced-firewall bypass-stateful-firewall-config add source_network 192.168.99.0 source_netmask 255.255.255.0 dest_network 192.168.20.0 dest_netmask 255.255.255.0

Make sure that you have a static or dynamically learned route to the networks. This is not a routing entry, it only tells the firewall what traffic should bypass the stateful firewall.

DOT1X authentication failed on HPE OfficeConnect 1920 switches

This posting is ~5 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

The last two days, I have supported a customer during the implementation of 802.1x. His network consisted of HPE/ Aruba and some HPE Comware switches. Two RADIUS server with appropriate policies was already in place. The configuration and test with the ProVision based switches was pretty simple. The Comware based switches, in this case OfficeConnect 1920, made me more headache.

blickpixel/ pixabay.com/ Creative Commons CC0

The customer had already mac authentication running, so all I had to do, was to enable 802.1x on the desired ports of the OfficeConnect 1920. The laptop, which I used to test the connection, was already configured and worked flawless if I plugged it into a 802.1x enabled port on a ProVision based switch. The OfficeConnect 1920 simply wrote a failure to its log and the authentication failed. The RADIUS server does not logged any failure, so I was quite sure, that the switch caused the problem.

DOT1X/6/DOT1X_AUTH_FAILURE: -IfName=GigabitEthernet1/0/1-UserName=DOM\USERNAME; DOT1X authentication failed

After double-checking all settings using the web interface of the switch, I used the CLI to check some more settings. Unfortunately, the OfficeConnect 1920 is a smart-managed switch and provides only a very, very limited CLI. Fortunately, there is a developer access, enabling the full Comware CLI. You can enable the full CLI by entering

_cmdline-mode on

after logging into the limited CLI. You can find the password using your favorite internet search engine. ;)

Solution

While poking around in the CLI, I stumbled over this option, which is entered in the interface context:

[1920-GigabitEthernet1/0/1] dot1x mandatory-domain RADIUS

RADIUS is the authentication domain, which was used on this switch. The command specifies, that the authentication domain RADIUS has to be for 802.1x authentication requests. Otherwise the switch would use the default authentication domain SYSTEM, which causes, that the switch tries to authenticate the user against the local user database.

I have not found any way to specify this setting using the web GUI! If you know how, of if you can provide additional information about this “issue”, please leave a comment.

Notes about 802.1x and MAC authentication

This posting is ~6 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Open network ports in offices, waiting rooms and entrance halls make me curious. Sometimes I  want to plugin a network cable, just to see if I get an IP address. I know many companies that does not care about network access control. Anybody can plugin any device to the network. When talking with customers about network access control, or port security, I often hear their complains about complexity. It’s too complex to implement, to hard to administrate. But it is not sooo complex. In the easiest setup (with mac authentication), you need a switch, that can act as authenticator, and a authentication server. But IEEE 802.1x is not much more complicated.

A brief overview over IEEE 802.1x

IEEE 802.1X offers authentication and authorization in wired or wireless networks. The supplicant (client) requests access to the network by providing a username/ password, or a digital certificate to the authenticator (switch). The authenticator forwards the provided credentials to the authentication server (mostly RADIUS or DIAMETER). The authentication server verifies the credentials and decides, if the supplicant is allowed to access the network.

802.1x uses the Extensible Authentication Protocol (EAP RFC5247) for authentication. Because EAP is a framework, there are different implementations, like EAP Transport Layer Security (EAP-TLS), or EAP with pre-shared key (EAP-PSK). Because it is only a framework, each protocol, that uses EAP, has to encapsulate it. Typical encapsulations are EAP over LAN (that is what 802.1x uses), RADIUS/ DIAMETER can use also use EAP. Protected EAP (PEAP) encapsulates EAP traffic into a TLS tunnel. PEAP is typically used as a replacement for EAP in EAPOL, or with with RADIUS or DIAMETER.

IEEE 802.1x EAP

Wikipedia/ wikipedia.org/ Public domain image resources

So far nothing special. It’s more a security thing, but an important one, if you ask me. But many customers avoid 802.1x, because of complexity. It’s perfect to keep you out of your own network, if something fails. And not all devices can act as supplicant.

But there is another benefit of 802.1x: RADIUS-Access-Accept messages can be used to dynamically assign VLAN memberships (RADIUS Extensions, RFC6929). To assign a VLAN membership to a port, to which a supplicant is connected, the RADIUS server adds three attributes to the Access-Accept message:

  • Tunnel-Type (VLAN)
  • Tunnel-Medium-Type (802)
  • Tunnel-Private-Group-Id (VLAN ID)

The authenticator uses these attributes to dynamically assign a VLAN to the port, to which the supplicant is connected.

MAC authentication

How does MAC authentication fit into this? If a client does not support 802.1x, the authenticator can use the mac-address of the connected device as username and password. The RADIUS server can use these credentials to authenticate the connected device. If you use a windows-based NAP (Windows Server NPS role), you have to create a user object in your Active Directory or local user database, that uses the mac-address as username and password. Depending on the switch configuration, the format of the username differes (xx:xx:xx:xx:xx:xx or xxxxxx-xxxxxx etc.). It’s a security fail, right? Yes, it is. So please:

  • Use MAC authentication only when needed, and
  • make sure that your authenticator uses PEAP

PEAP uses a TLS tunnel to protect the CHAP messages.

Another important part is your authentication server, mostly a RADIUS or DIAMETER server. Make sure that it is highly available. You should have at least two authentication server. I would not load balance them through a load balancer (Citrix NetScaler etc.). Simply add two authentication servers to your switch configuration. If your authentication server uses a user database, like Microsoft Active Directory, make sure that this database is also highly available. As I said: It is perfect to keep you out of your own network.

Sample config for ArubaOS (HPE ProVision based switches)

Here’s a sample config for a Aruba 2920 switch, running ArubaOS WB.16.04. 802.1x and MAC authentication are configured for the ports 1 to 5. If the authentication failes, VLAN 999 will be assigned to the port. VLAN 999 is used as unauth VLAN, which is used for unauthenticated clients.

radius-server host 192.168.1.10 key "T0p5ecr3t"
radius-server host 192.168.1.11 key "T0p5ecr3t"
aaa authentication port-access eap-radius
aaa authentication mac-based peap-mschapv2
aaa port-access authenticator 1-5
aaa port-access authenticator 1 client-limit 32
aaa port-access authenticator 2 client-limit 32
aaa port-access authenticator 3 client-limit 32
aaa port-access authenticator 4 client-limit 32
aaa port-access authenticator 5 client-limit 32
aaa port-access authenticator active
aaa port-access mac-based 1-5
aaa port-access mac-based 1 addr-limit 32
aaa port-access mac-based 1 unauth-vid 999
aaa port-access mac-based 2 addr-limit 32
aaa port-access mac-based 2 unauth-vid 999
aaa port-access mac-based 3 addr-limit 32
aaa port-access mac-based 3 unauth-vid 999
aaa port-access mac-based 4 addr-limit 32
aaa port-access mac-based 4 unauth-vid 999
aaa port-access mac-based 5 addr-limit 32
aaa port-access mac-based 5 unauth-vid 999

If 802.1x fails, the authenticator, will try MAC authentication. If this fails too, VLAN 999 is assigned to the switch port.

In this case, the client was authenticated by 802.1x.

SW1(config)# show port-access auth client 1 detailed

 Port Access Authenticator Client Status Detailed

  Port-access authenticator activated [No] : Yes
  Allow RADIUS-assigned dynamic (GVRP) VLANs [No] : No
  Dot1x2010 Mode [Disabled] : Disabled                Use LLDP data to authenticate [No] : No

  Client Base Details :
   Port            : 1
   Client Status   : Authenticated         Session Time    : 9 seconds
   Client name     : user@domain           Session Timeout : 0 seconds
   IP              : n/a                   MAC Address     : 643150-7c7c9f

  Access Policy Details :
   COS Map         : Not Defined           In Limit Kbps   : Not Set
   Untagged VLAN   : 2500                  Out Limit Kbps  : Not Set
   Tagged VLANs    : No Tagged VLANs
   Port Mode       : 1000FDx
   RADIUS ACL List : No Radius ACL List

This is the output for MAC authentication.

SW1(config)# show port-access mac-based clients 1 detailed

 Port Access MAC-Based Client Status Detailed

  Client Base Details :
   Port            : 1
   Client Status   : authenticated         Session Time    : 14 seconds
   MAC Address     : 643150-7c7c9f         Session Timeout : 0 seconds
   IP              : n/a

  Access Policy Details :
   COS Map         : Not Defined           In Limit Kbps   : Not Set
   Untagged VLAN   : 1                     Out Limit Kbps  : Not Set
   Tagged VLANs    : No Tagged VLANs
   Port Mode       : 1000FDx               Auth Mode       : User-based
   RADIUS ACL List : No Radius ACL List

In both cases, VLAN 1 was dynamically assigned by RADIUS-Access-Accept messages.

Enable IPv6 SLAAC on HPE OfficeConnect 1920 switches

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

The HPE OfficeConnect 1920 switch series is designed for SMBs. The switch is perfect for small environments, that require features like VLANs, routing or 802.1x. This switch is smart-managed, so it has “only” a web interface and only a limited CLI.

I have two switches in my lab: A 1910-8G and the successor, a 1920-24G. Although the device supports IPv6, it doesn’t support SLAAC (Stateless Address Autoconfiguration) by default. The switch does not send router advertisements (RA). I’m using IPv6 in my lab (Stateless DHCPv6 + SLAAC), so the missing RAs were a problem for me, or at least, annoying. Fortunately you can change the default behaviour.

Enable router advertisements (RA)

To change the default behaviour of the HPE 1920, you have to use the CLI. The CLI is very limited, but there’s a hidden CLI command, which enables access to nearly all available features. If you are familiar with HPEs Comware based switches, you will notice, that the switch is a Comware-based device.

login as: admin
[email protected]'s password:

******************************************************************************
* Copyright (c) 2010-2016 Hewlett Packard Enterprise Development LP          *
* Without the owner's prior written consent,                                 *
* no decompiling or reverse-engineering shall be allowed.                    *
******************************************************************************

<1920-24G>_cmdline-mode on
All commands can be displayed and executed. Continue? [Y/N]y
Please input password: Jinhua1920unauthorized
Warning: Now you enter an all-command mode for developer's testing, some commands may affect operation by wrong use, please carefully use it with our engineer's direction.
<1920-24G>sys
System View: return to User View with Ctrl+Z.

After switching to the system-view, we can change the default behaviour for each VLAN interface. I have multiple VLAN interfaces and each VLAN interface has an IPv4 and an unique local address (ULA) IPv6 address.

[1920-24G]interface Vlan-interface 3
[1920-24G-Vlan-interface3]display this
#
interface Vlan-interface3
 ipv6 address FDDA:28AD:487:3:FFFF:FFFF:FFFF:FFFE/64
 ip address 192.168.200.94 255.255.255.224
 dhcp select relay
 dhcp relay server-select 0
#
return

[1920-24G-Vlan-interface3]undo ipv6 nd ra halt
[1920-24G-Vlan-interface3]ipv6 nd ra prefix FDDA:28AD:487:3::/64 86400 3600
[1920-24G-Vlan-interface3]display this
#
interface Vlan-interface3
 ipv6 nd ra prefix FDDA:28AD:487:3::/64 86400 3600
 undo ipv6 nd ra halt
 ipv6 address FDDA:28AD:487:3:FFFF:FFFF:FFFF:FFFE/64
 ip address 192.168.200.94 255.255.255.224
 dhcp select relay
 dhcp relay server-select 0
#
return
[1920-24G-Vlan-interface3]quit

The first command enables router advertisements. The second command adds the prefix which should be announced. That’s it. Don’t forget to save the changed configuration with “save force”. If you have more than one VLAN interface, enter this command in each VLAN interface context you wish to change.

I’m routing on the edge…

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

In my last post (Routed Port vs. Switch Virtual Interface (SVI)), I have mentioned a consequence of using routed ports to interconnect access and core switches:

You have to route the traffic on the access switches.

Routing on the network access, the edge of the network, is not a question of performance. It is more of a management issue. Depending on the size of your network, and the number of subnets, you have to deal with lots of routes. And think about the effort, if you add, change or remove subnets from your network. This is not what you want to do with static routes. You need a routing protocol.

The experiment of the week

We have a core switch C1, consisting of two independent switches (C1-1 and C1-2) forming a virtual chassis. S1 and S2 are two switches at the network access. This is a core-edge design. There is no distribution layer. Each switch at the network access has two uplinks: One uplink to C1-1 and one uplink to C1-2. The ports on each end of the links are configured as routed ports.

Please ignore the 40 GbE ports (FGE) between C1-1 and C1-2. These ports are used for the Intelligent Resilient Framework (IRF), which is used to create a virtual chassis.

routed_links_1

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

These are the interfaces on the core switch, that are working in route mode. GE1/0/1 and GE2/0/1 are the uplinks to S1, and GE1/0/2 and GE2/0/2 are the uplinks to S2.

[C1]display interface brief
Brief information on interface(s) under route mode:
Link: ADM - administratively down; Stby - standby
Protocol: (s) - spoofing
Interface            Link Protocol Main IP         Description
GE1/0/1              UP   UP       10.0.0.1
GE1/0/2              UP   UP       10.10.0.1
GE2/0/1              UP   UP       10.1.0.1
GE2/0/2              UP   UP       10.11.0.1
InLoop0              UP   UP(s)    --
Loop0                UP   UP(s)    1.1.1.1
MGE0/0/0             DOWN DOWN     --
NULL0                UP   UP(s)    --
REG0                 UP   --       --

These are the interfaces on the access switch S1, that are working in route mode. GE1/0/1 and GE1/0/2 are the uplinks to C1. As you can see, GE1/0/1 on C1 and  GE1/0/1 on S1 belong to the same /30 network. The same applies to GE2/0/1 on C1 and GE1/0/2 on S1. There are also two SVIs, one on VLAN 1 (192.168.1.0/24) and another on VLAN 2 (192.168.2.0/24). These VLANs are used for client connectivity.

[S1]display interface brief
Brief information on interface(s) under route mode:
Link: ADM - administratively down; Stby - standby
Protocol: (s) - spoofing
Interface            Link Protocol Main IP         Description
GE1/0/1              UP   UP       10.0.0.2
GE1/0/2              UP   UP       10.1.0.2
InLoop0              UP   UP(s)    --
Loop0                UP   UP(s)    1.1.1.2
MGE0/0/0             DOWN DOWN     --
NULL0                UP   UP(s)    --
REG0                 UP   --       --
Vlan1                UP   UP       192.168.1.1
Vlan2                UP   UP       192.168.2.1

These are the interfaces on S2, that are working in route mode. GE1/0/1 and GE1/0/2 are the uplinks to C1. The interfaces GE1/0/2 on C1, and  GE1/0/1 on S2 belong to the same /30 network. The same applies to GE2/0/2 on C1 and GE1/0/2 on S2. There are also two SVIs, one on VLAN 1 (192.168.10.0/24) and another on VLAN 2 (192.168.20.0/24).

You might wonder, because the same VLAN IDs are used on both access switches. They don’t care, because there is no layer 2 connectivity between these two switches. The only way from S1 to S2 is over the routed links to the core switch.

[S2]display interface brief
Brief information on interface(s) under route mode:
Link: ADM - administratively down; Stby - standby
Protocol: (s) - spoofing
Interface            Link Protocol Main IP         Description
GE1/0/1              UP   UP       10.10.0.2
GE1/0/2              UP   UP       10.11.0.2
InLoop0              UP   UP(s)    --
Loop0                UP   UP(s)    1.1.1.3
MGE0/0/0             DOWN DOWN     --
NULL0                UP   UP(s)    --
REG0                 UP   --       --
Vlan1                UP   UP       192.168.10.1
Vlan2                UP   UP       192.168.20.1

Now let’s have a look at the Open Shortest Path First (OSPF) routing protocol.

Single Area OSPF

The Open Shortest Path First (OSPF) routing protocol is an interior gateway protocols (IGP), and also a link-state routing protocol. The calculation of the shortest path for each route is based on Dijkstra’s algorithm. I don’t want to annoy you with details. Take a look at the Wikipedia article for OSPF.

The simplest OSPF setup is a “Single Area OSPF”. This is an OSPF configuration, which has only a single area. This is the area 0, or the backbone area.

The configuration on the core switch looks like this:

[C1]ospf 1
[C1-ospf-1]display this
#
ospf 1
 area 0.0.0.0
  network 1.1.1.1 0.0.0.0
  network 10.0.0.0 0.255.255.255
#

The networks, that should be associated with this area, are specified with a wildcard mask. The wildcard mask is the opposite of the subnet mask. The wildcard mask 0.255.255.255 corresponds to the subnet mask 255.0.0.0. Because I have used multiple /30 subnets at the core switch, I can summarize them with a single entry for 10.0.0.0.

The same configuration applies to the access switches S1 and S2.

[S1]ospf 1
[S1-ospf-1]display this
#
ospf 1
 area 0.0.0.0
  network 1.1.1.2 0.0.0.0
  network 10.0.0.0 0.255.255.255
  network 192.168.0.0 0.0.255.255
#
[S2]ospf 1
[S2-ospf-1]display this
#
ospf 1
 area 0.0.0.0
  network 1.1.1.3 0.0.0.0
  network 10.0.0.0 0.255.255.255
  network 192.168.0.0 0.0.255.255
#

With this simple configuration, the switches will exchange their routing information. They will synchronize their link-state databases, and they will be fully adjacent. If a link-state change occurs, OSPF will handle this.

The core switch has two links to each access switch. The router ID represents the access switches. 1.1.1.2 is a loopback interface IP address on S1, 1.1.1.3 is a loopback interface IP address on S2.

<C1>display ospf peer

         OSPF Process 1 with Router ID 1.1.1.1
               Neighbor Brief Information

 Area: 0.0.0.0
 Router ID       Address         Pri Dead-Time  State             Interface
 1.1.1.2         10.0.0.2        1   37         Full/DR           GE1/0/1
 1.1.1.3         10.10.0.2       1   36         Full/DR           GE1/0/2
 1.1.1.2         10.1.0.2        1   38         Full/DR           GE2/0/1
 1.1.1.3         10.11.0.2       1   35         Full/DR           GE2/0/2

The same applies to the access switches, in this case S1. The access switches have also two active links to the core switch.

<S1>dis ospf peer

         OSPF Process 1 with Router ID 1.1.1.2
               Neighbor Brief Information

 Area: 0.0.0.0
 Router ID       Address         Pri Dead-Time  State             Interface
 1.1.1.1         10.0.0.1        1   31         Full/BDR          GE1/0/1
 1.1.1.1         10.1.0.1        1   32         Full/BDR          GE1/0/2

If one of the links fail, the access switch has another working link to the core switch, and OSPF will recalculate the shortest paths, taking the link-state change (link down between core and an access switch) into account.

This is the OSPF routing table of the core switch, based on the example above.

<C1>display ospf routing

         OSPF Process 1 with Router ID 1.1.1.1
                  Routing Table

 Routing for network
 Destination        Cost     Type    NextHop         AdvRouter       Area
 192.168.10.0/24    2        Stub    10.10.0.2       1.1.1.3         0.0.0.0
 192.168.10.0/24    2        Stub    10.11.0.2       1.1.1.3         0.0.0.0
 10.10.0.0/30       1        Transit 0.0.0.0         1.1.1.3         0.0.0.0
 10.11.0.0/30       1        Transit 0.0.0.0         1.1.1.3         0.0.0.0
 10.0.0.0/30        1        Transit 0.0.0.0         1.1.1.2         0.0.0.0
 10.1.0.0/30        1        Transit 0.0.0.0         1.1.1.2         0.0.0.0
 1.1.1.1/32         0        Stub    0.0.0.0         1.1.1.1         0.0.0.0
 1.1.1.2/32         1        Stub    10.0.0.2        1.1.1.2         0.0.0.0
 1.1.1.2/32         1        Stub    10.1.0.2        1.1.1.2         0.0.0.0
 1.1.1.3/32         1        Stub    10.10.0.2       1.1.1.3         0.0.0.0
 1.1.1.3/32         1        Stub    10.11.0.2       1.1.1.3         0.0.0.0
 192.168.20.0/24    2        Stub    10.10.0.2       1.1.1.3         0.0.0.0
 192.168.20.0/24    2        Stub    10.11.0.2       1.1.1.3         0.0.0.0
 192.168.1.0/24     2        Stub    10.0.0.2        1.1.1.2         0.0.0.0
 192.168.1.0/24     2        Stub    10.1.0.2        1.1.1.2         0.0.0.0
 192.168.2.0/24     2        Stub    10.0.0.2        1.1.1.2         0.0.0.0
 192.168.2.0/24     2        Stub    10.1.0.2        1.1.1.2         0.0.0.0

 Total nets: 17
 Intra area: 17  Inter area: 0  ASE: 0  NSSA: 0

What if I add a new subnet on S1? Let’s create a new VLAN and add a SVI to it (VLAN 3 and 192.168.3.1).

[S1]dis int brief
Brief information on interface(s) under route mode:
Link: ADM - administratively down; Stby - standby
Protocol: (s) - spoofing
Interface            Link Protocol Main IP         Description
GE1/0/1              UP   UP       10.0.0.2
GE1/0/2              UP   UP       10.1.0.2
InLoop0              UP   UP(s)    --
Loop0                UP   UP(s)    1.1.1.2
MGE0/0/0             DOWN DOWN     --
NULL0                UP   UP(s)    --
REG0                 UP   --       --
Vlan1                UP   UP       192.168.1.1
Vlan2                UP   UP       192.168.2.1
Vlan3                UP   UP       192.168.3.1

Without touching the OSPF configuration, the core switch C1, and the other access switch S2, added routes to this new subnet.

<C1>display ospf routing

         OSPF Process 1 with Router ID 1.1.1.1
                  Routing Table

 Routing for network
 Destination        Cost     Type    NextHop         AdvRouter       Area
 192.168.3.0/24     2        Stub    10.0.0.2        1.1.1.2         0.0.0.0
 192.168.3.0/24     2        Stub    10.1.0.2        1.1.1.2         0.0.0.0
 192.168.10.0/24    2        Stub    10.10.0.2       1.1.1.3         0.0.0.0
 192.168.10.0/24    2        Stub    10.11.0.2       1.1.1.3         0.0.0.0
 10.10.0.0/30       1        Transit 0.0.0.0         1.1.1.3         0.0.0.0
 10.11.0.0/30       1        Transit 0.0.0.0         1.1.1.3         0.0.0.0
 10.0.0.0/30        1        Transit 0.0.0.0         1.1.1.2         0.0.0.0
 10.1.0.0/30        1        Transit 0.0.0.0         1.1.1.2         0.0.0.0
 1.1.1.1/32         0        Stub    0.0.0.0         1.1.1.1         0.0.0.0
 1.1.1.2/32         1        Stub    10.0.0.2        1.1.1.2         0.0.0.0
 1.1.1.2/32         1        Stub    10.1.0.2        1.1.1.2         0.0.0.0
 1.1.1.3/32         1        Stub    10.10.0.2       1.1.1.3         0.0.0.0
 1.1.1.3/32         1        Stub    10.11.0.2       1.1.1.3         0.0.0.0
 192.168.20.0/24    2        Stub    10.10.0.2       1.1.1.3         0.0.0.0
 192.168.20.0/24    2        Stub    10.11.0.2       1.1.1.3         0.0.0.0
 192.168.1.0/24     2        Stub    10.0.0.2        1.1.1.2         0.0.0.0
 192.168.1.0/24     2        Stub    10.1.0.2        1.1.1.2         0.0.0.0
 192.168.2.0/24     2        Stub    10.0.0.2        1.1.1.2         0.0.0.0
 192.168.2.0/24     2        Stub    10.1.0.2        1.1.1.2         0.0.0.0

 Total nets: 19
 Intra area: 19  Inter area: 0  ASE: 0  NSSA: 0
<S2>display ospf routing

         OSPF Process 1 with Router ID 1.1.1.3
                  Routing Table

 Routing for network
 Destination        Cost     Type    NextHop         AdvRouter       Area
 192.168.3.0/24     3        Stub    10.10.0.1       1.1.1.2         0.0.0.0
 192.168.3.0/24     3        Stub    10.11.0.1       1.1.1.2         0.0.0.0
 192.168.10.0/24    1        Stub    0.0.0.0         1.1.1.3         0.0.0.0
 10.10.0.0/30       1        Transit 0.0.0.0         1.1.1.3         0.0.0.0
 10.11.0.0/30       1        Transit 0.0.0.0         1.1.1.3         0.0.0.0
 10.0.0.0/30        2        Transit 10.10.0.1       1.1.1.2         0.0.0.0
 10.0.0.0/30        2        Transit 10.11.0.1       1.1.1.2         0.0.0.0
 10.1.0.0/30        2        Transit 10.10.0.1       1.1.1.2         0.0.0.0
 10.1.0.0/30        2        Transit 10.11.0.1       1.1.1.2         0.0.0.0
 1.1.1.1/32         1        Stub    10.10.0.1       1.1.1.1         0.0.0.0
 1.1.1.1/32         1        Stub    10.11.0.1       1.1.1.1         0.0.0.0
 1.1.1.2/32         2        Stub    10.10.0.1       1.1.1.2         0.0.0.0
 1.1.1.2/32         2        Stub    10.11.0.1       1.1.1.2         0.0.0.0
 1.1.1.3/32         0        Stub    0.0.0.0         1.1.1.3         0.0.0.0
 192.168.20.0/24    1        Stub    0.0.0.0         1.1.1.3         0.0.0.0
 192.168.1.0/24     3        Stub    10.10.0.1       1.1.1.2         0.0.0.0
 192.168.1.0/24     3        Stub    10.11.0.1       1.1.1.2         0.0.0.0
 192.168.2.0/24     3        Stub    10.10.0.1       1.1.1.2         0.0.0.0
 192.168.2.0/24     3        Stub    10.11.0.1       1.1.1.2         0.0.0.0

 Total nets: 19
 Intra area: 19  Inter area: 0  ASE: 0  NSSA: 0

Pretty cool, isn’t it?

Any downsides?

This is only an example with a single core switch and two access switches. OSPF can be pretty complex, if the size of the network increases. The Dijkstra’s algorithm can be really CPU intensive, and the size of the link-state databases (LSDB) increase with adding more routers and networks. For this reason, larger networks have to be divided into separate areas. It depends on the network size and the CPU/ memory performance of your switches/ routers, but a common practice is a maximum of up to 50 switches/ routers per area. If you have unstable links, the area should be smaller, because each link-state change is flooded to all neighbors and consumes CPU time.

You need a good subnet design, otherwise you have to touch your OSPF configuration too often. You should be able to summarize subnets.

Conclusion

Routing at the network access is nothing for small networks. There are better designs for small networks. But if your network has a decent size, routing at the edge of the network can offer some benefits. Instead of working with SVIs and small transfer VLANs, a routed port is more simple to implement. Routed links can also have a shorter convergence delay, and you can reduce the usage of Spanning Tree Protocol to a minimum.

Routed Port vs. Switch Virtual Interface (SVI)

This posting is ~7 years years old. You should keep this in mind. IT is a short living business. This information might be outdated.

Many years ago, networks consisted of repeaters, bridges and router. Switches are the successors of the bridges. A switch is nothing else than a multiport bridge, and a traditional switch doesn’t know how to pass traffic to a different broadcast domains (VLANs). Passing traffic between different broadcast domains, is a job for a router. A router has an IP interface in each broadcast domain, and the IP interface is used by the clients in the broadcast domain as a gateway.

Switch Virtual Interface

A Switch Virtual Interface, or SVI, is exactly this: An virtual IP interface in a broadcast domain (or VLAN). It’s used by the connected clients in the broadcast domain to send traffic to other broadcast domains.

This is how a SVI is created on HPE Comware 7. It’s similar to other vendors.

[C1]interface Vlan-interface 1
[C1-Vlan-interface1]ip address 10.100.100.1 30
[C1-Vlan-interface1]display this
#
interface Vlan-interface1
 ip address 10.100.100.1 255.255.255.252
#

At least one port is assigned to this VLAN, and as soon as at least one port of this VLAN is online, the SVI is also reachable.

What happens, if you connect two switches with a cable? The broadcast domain spans both switches. Layer 2 traffic is transmitted between the switches. And what would happen if you connect a second cable between the same two switches? As long as you are running Spanning Tree Protocol (STP), or another loop detection mechanism, nothing would happen. But one of the two connection would be blocked. No traffic would be able to pass over this connection. If you want to use multiple, active connections between switches, you have to use Link Aggregation Groups (LAG), or things like Multiple Spanning Tree Protocol (MSTP) and Per VLAN Spanning Tree (PVST).

Routers don’t know this. Multiple connections between the same two routers can’t form a loop. Loops and STP (an some other crappy layer 2 stuff) are legacies of the bridges, still alive in modern switches. Loops are a typical “bridge problem”.

Routed Ports

Some switches offer a way, to change the operation mode of a switch port. After changing this operation mode, a switch port doesn’t act like a bridge port anymore. It’s acting like the port of a router, that only handles layer 3 traffic.

This is again a HPE Comware 7 example. I know that Cisco and Alcatel Lucent Enterprise also offer routed ports.

This is a normal switch port. Please note the “port link-mode bridge”.

[C1]int XGE1/0/49
[C1-Ten-GigabitEthernet1/0/49]display this
#
interface Ten-GigabitEthernet1/0/49
 port link-mode bridge
 combo enable fiber
#

To “convert” a switch into a routed port, simply change the link-mode of the port.

[C1-Ten-GigabitEthernet1/0/49]port link-mode route
[C1-Ten-GigabitEthernet1/0/49]ip address 10.10.10.1 30
[C1-Ten-GigabitEthernet1/0/49]display this
#
interface Ten-GigabitEthernet1/0/49
 port link-mode route
 combo enable fiber
 ip address 10.10.10.1 255.255.255.252
#

As you can see, you can now assign an IP address directly to the port.

Example

Let’s try to make this clear with an example. C1-1 and C1-2 are two HPE Comware based switched, configured as an IRF stack (virtual chassis). These two switches form the core switch C1. S1 and S2 are two access switches, also HPE Comware based. Each access switch has two uplinks: One uplink to C1-1 and another uplink to C1-2, the two chassis that form C1. The 40 GbE Ports between C1-1 and C1-2 are used for IRF. Please ignore them.

The uplinks between the switches, all ports are Gigabit Ethernet (GE) ports, are configured as routed ports.

Without routed ports, the uplinks must be configured as a LAG, or STP would block one of the two uplinks between the core switches and the access switch. But because routed ports are used, no loop is formed. Most layer 2 traffic can’t pass the routed ports (broadcasts, multicasts etc.)

routed_links_1

Patrick Terlisten/ www.vcloudnine.de/ Creative Commons CC0

THe Link Layer Discovery Protocol (LLDP) traffic can pass the routed port. This is what the core switch (C1) “sees” over LLDP.

[C1]display lldp neighbor-information list
Chassis ID : * -- -- Nearest nontpmr bridge neighbor
             # -- -- Nearest customer bridge neighbor
             Default -- -- Nearest bridge neighbor
System Name          Local Interface Chassis ID      Port ID
S1                   GE1/0/1         34ce-d3a9-0300  GigabitEthernet1/0/1
S2                   GE1/0/2         34cf-0690-0400  GigabitEthernet1/0/1
S1                   GE2/0/1         34ce-d3a9-0300  GigabitEthernet1/0/2
S2                   GE2/0/2         34cf-0690-0400  GigabitEthernet1/0/2

Each routed port as an IP address assigned. The same applies to the routed ports on the access switches. Each uplink pair (core to access) uses a /30 subnet.

As you can see, the interfaces working in bridge mode start counting at GE1/0/3.

[C1]display interfaces brief
Brief information on interface(s) under route mode:
Link: ADM - administratively down; Stby - standby
Protocol: (s) - spoofing
Interface            Link Protocol Main IP         Description
GE1/0/1              UP   UP       10.0.0.1
GE1/0/2              UP   UP       10.10.0.1
GE2/0/1              UP   UP       10.1.0.1
GE2/0/2              UP   UP       10.11.0.1
InLoop0              UP   UP(s)    --
Loop0                UP   UP(s)    1.1.1.1
MGE0/0/0             DOWN DOWN     --
NULL0                UP   UP(s)    --
REG0                 UP   --       --

Brief information on interface(s) under bridge mode:
Link: ADM - administratively down; Stby - standby
Speed or Duplex: (a)/A - auto; H - half; F - full
Type: A - access; T - trunk; H - hybrid
Interface            Link Speed   Duplex Type PVID Description
FGE1/0/53            UP   40G     F(a)   --   --
FGE1/0/54            UP   40G     F(a)   --   --
FGE2/0/53            UP   40G     F(a)   --   --
FGE2/0/54            UP   40G     F(a)   --   --
GE1/0/3              DOWN auto    A      A    1
....
....

The same applies to STP. The ports, that were configured as routed ports, are not listed in the output. STP is not active on these ports.

[C1]display stp
-------[CIST Global Info][Mode MSTP]-------
 Bridge ID           : 0.34a9-6908-0100
 Bridge times        : Hello 2s MaxAge 20s FwdDelay 15s MaxHops 20
 Root ID/ERPC        : 0.34a9-6908-0100, 0
 RegRoot ID/IRPC     : 0.34a9-6908-0100, 0
 RootPort ID         : 0.0
 BPDU-Protection     : Disabled
 Bridge Config-
 Digest-Snooping     : Disabled
 Root type           : Primary root
 TC or TCN received  : 0
 Time since last TC  : 0 days 0h:27m:23s

----[Port4(GigabitEthernet1/0/3)][DOWN]----
 Port protocol       : Enabled
 Port role           : Disabled Port
 Port ID             : 128.4
 Port cost(Legacy)   : Config=auto, Active=200000
 Desg.bridge/port    : 0.34a9-6908-0100, 128.4
 Port edged          : Config=disabled, Active=disabled
 Point-to-Point      : Config=auto, Active=false
 Transmit limit      : 10 packets/hello-time
 TC-Restriction      : Disabled
 Role-Restriction    : Disabled
 Protection type     : Config=none, Active=none
 MST BPDU format     : Config=auto, Active=802.1s
 Port Config-
 Digest-Snooping     : Disabled
 Rapid transition    : False
 Num of VLANs mapped : 1
 Port times          : Hello 2s MaxAge 20s FwdDelay 15s MsgAge 0s RemHops 20
 BPDU sent           : 0
          TCN: 0, Config: 0, RST: 0, MST: 0
 BPDU received       : 0
          TCN: 0, Config: 0, RST: 0, MST: 0

What are the implications?

The example shows redundant links between access and core switches. There are no loops, but there’s also no layer 2 connectivity. VLANs are only located on the access switches. There are no VLANs spanning multiple switches. What does this mean? How can a client on S1 reach a server on S2? The answer is simple: You have to route the traffic on the access switches. But that’s a topic for another blog post.