Andy,
That's a lot of info but here goes:
Hardware = own design ARM CPU board with three Infinion DSL chips.
system = Linux 2.6.12 + patches for ARM processor (AT91RM9200)
from maxim.org.za + driver module for Infineon DSL chipset created by
us.
Quagga = Quagga 0.99.3
We do not have ip on these boxes only ifconfig supplied by busybox.
When the link is connected ifconfig shows:
dsl0 Link encap:Point-Point Protocol
inet addr:10.164.56.1 P-t-P:10.164.56.5 Mask:255.255.255.255
UP PINTPINT RUNNING NARP MTU:1500 Metric:1
RX packets:2211 errors:103835 dropped:21768 overruns:0
frame:82067
TX packets:755 errors:4010 dropped:0 overruns:0 carrier:4010
collisions:0 txqueuelen:100
RX bytes:131570 (128.4 KiB) TX bytes:52930 (51.6 KiB)
Interrupt:14
When a cable is pulled out ifconfig shows:
dsl0 Link encap:Point-Point Protocol
inet addr:10.164.56.1 P-t-P:10.164.56.5 Mask:255.255.255.255
UP PINTPINT NARP MTU:1500 Metric:1
RX packets:2236 errors:219371 dropped:46026 overruns:0
frame:173345
TX packets:764 errors:8215 dropped:0 overruns:0 carrier:8215
collisions:0 txqueuelen:100
RX bytes:133020 (129.9 KiB) TX bytes:53560 (52.3 KiB)
Interrupt:14
As we see "RUNNING" has disappeared as expected. It does reappear when
the cable is reconnected. We also have an SNMP agent that reports
correct Admin and status as far as I can tell.
When cable is connected Zebra "show interface" reports:
Interface dsl0 is up, line protocol is up
index 3 metric 1 mtu 1500 <UP,PINTPINT,RUNNING,NARP>
inet 10.164.56.1/32 pointopoint 10.164.56.5
2284 input packets (0 multicast), 135868 bytes, 46026 dropped
219371 input errors, 0 length, 0 overrun, 0 CRC, 173345 frame
0 fifo, 0 missed
782 output packets, 54788 bytes, 0 dropped
8215 output errors, 0 aborted, 8215 carrier, 0 fifo, 0 heartbeat
0 window, 0 collisions
When disconnected:
Interface dsl0 is up, line protocol is down
index 3 metric 1 mtu 1500 <UP,PINTPINT,NARP>
inet 10.164.56.1/32 pointopoint 10.164.56.5
2315 input packets (0 multicast), 137653 bytes, 69958 dropped
332852 input errors, 0 length, 0 overrun, 0 CRC, 262894 frame
0 fifo, 0 missed
793 output packets, 55558 bytes, 0 dropped
12283 output errors, 0 aborted, 12283 carrier, 0 fifo, 0 heartbeat
0 window, 0 collisions
When connected SPF "show ip ospf interface" reports:
dsl0 is up
ifindex 3, MTU 1500 bytes, BW 0 Kbit <UP,PINTPINT,RUNNING,NARP>
Internet Address 10.164.56.1/32, Peer 10.164.56.5, Area 0.0.0.0
MTU mismatch detection:enabled
Router ID 10.164.56.1, Network Type PINTPINT, Cost: 1
Transmit Delay is 1 sec, State Point-To-Point, Priority 1
No designated router on this network
No backup designated router on this network
Multicast group memberships: SPFAllRouters
Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit
5
Hello due in 9.298s
Neighbor Count is 1, Adjacent neighbor count is 1
And when the cable is out:
dsl0 is down
ifindex 3, MTU 1500 bytes, BW 0 Kbit <UP,PINTPINT,NARP>
SPF is enabled, but not running on this interface
I have now have the "link-detect" option in zebra.conf which did change
the results somewhat and produce the situation above. In addition I do
now get an alternative route to a neighbor when a cable is pulled.
Trouble is the original direct route is still in the kernel and seems to
be selected by the kernel instead of the working alternative.
For example I now have three routers on my desk connected in a ring:
R1(dsl0) =R2(dsl1)
R2(dsl0) =R3(dsl1)
R3(dsl0) =R1(dsl1)
With all cables in R1 shows routes:
# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
10.164.56.9 * 255.255.255.255 UH 0 0 0
dsl1
10.164.56.1 10.164.56.5 255.255.255.255 UGH 2 0 0
dsl0
10.164.56.5 * 255.255.255.255 UH 0 0 0
dsl0
10.164.56.0 * 255.255.252.0 U 0 0 0
eth0
Now I pull the cable between 10.164.56.1 and 10.164.56.9. After a while
I have routes:
~ # route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
10.164.56.9 * 255.255.255.255 UH 0 0 0
dsl1
10.164.56.9 10.164.56.5 255.255.255.255 UGH 2 0 0
dsl0
10.164.56.1 10.164.56.5 255.255.255.255 UGH 2 0 0
dsl0
10.164.56.5 * 255.255.255.255 UH 0 0 0
dsl0
10.164.56.0 * 255.255.252.0 U 0 0 0
eth0
Here we see a new route has appeared to 10.164.56.9 via 10.164.56.5 but
the original direct route is still there and seems to be selected by the
kernel, even if it is not running, as I can no longer reach 10.164.56.9
Phew ! Hope that helps pin this down.
At this point I'm not sure if quagga/ospf is even supposed to cope with
this situation.
Regards,
Michael.
Ystavallisin terveisin / Best Regards
Michael Rychlik
Swarco Technology
Niittylanpolku 16
FIN-00620 Helsinki
Tel: +358 20 741 0325
Mobile: +358 40 822 8586
Fax: +358 9 777 3103
E-mail: Michael.Rychlik (AT) swarco (DOT) com
Web: http://www.swarco.com
This message contains information that may be privileged or confidential and is the property of Swarco. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute or use this message or any part thereof. If you receive this message in error, please notify the sender immediately by reply e-mail and delete all copies of this message. Thank you for your cooperation.
Message
From: Andrew J. Schorr [mailto:aschorr (AT) telemetry-investments (DOT) com]
Sent: 8. toukokuuta 2006 17:29
To: Rychlik, Michael - FI
Cc: quagga-users (AT) lists (DOT) quagga.net
Subject: Re: [quagga-users 6851] ospf over unnumbered DSL interfaces.
Mon, May 08, 2006 at 10:54:33AM +0200, Rychlik, Michael - FI wrote:
So far SPF does successfully distribute routes around our "loop". The
problem comes when a link goes down (Cable out). SPF then correctly
reroutes around the broken link for all routers except for the ones on
each end of the disconnected link. Seems neighbors will always route
to
each other across the direct interface between them even if is not
running.
Which operating system are you using, and which version of quagga?
Is the /S detecting the link failure? If linux, what's the output
of "/sbin/ip addr ls" before and after the link failure?
Similarly, what's the output of zebra "show interface" and
ospfd "show ip ospf interface"? Also, have you tried turning
on link-detect for the interfaces in your zebra config?
Regards,
Andy
Quagga-users mailing list
Quagga-users (AT) lists (DOT) quagga.net

Tue, May 09, 2006 at 04:26:53PM +0200, Rychlik, Michael - FI wrote:
At this point I'm not sure if quagga/ospf is even supposed to cope with
this situation.

To summarize, it seems to me that the kernel is correctly detecting when the
cable is pulled by dropping the RUNNING flag. And when you turn on link-detect
in zebra, quagga now understands that the interface is not operational when the
RUNNING flag is absent. So it seems like quagga is now doing the right thing
in populating a new route. But the problem seems to be that the kernel is not
removing the directly connected route when the cable is pulled. I'm guessing
the kernel ought to do that (not quagga), but perhaps others have better
insight

Regards,
Andy

Quagga-users mailing list
Quagga-users (AT) lists (DOT) quagga.net

Michael,

see below.

Rychlik, Michael - FI wrote:
Andy,

That's a lot of info but here goes:

Hardware = own design ARM CPU board with three Infinion DSL chips.

system = Linux 2.6.12 + patches for ARM processor (AT91RM9200)
from maxim.org.za + driver module for Infineon DSL chipset created by
us.

Quagga = Quagga 0.99.3

We do not have ip on these boxes only ifconfig supplied by busybox.

When the link is connected ifconfig shows:

dsl0 Link encap:Point-Point Protocol
inet addr:10.164.56.1 P-t-P:10.164.56.5 Mask:255.255.255.255
UP PINTPINT RUNNING NARP MTU:1500 Metric:1
RX packets:2211 errors:103835 dropped:21768 overruns:0
frame:82067
TX packets:755 errors:4010 dropped:0 overruns:0 carrier:4010
collisions:0 txqueuelen:100
RX bytes:131570 (128.4 KiB) TX bytes:52930 (51.6 KiB)
Interrupt:14

When a cable is pulled out ifconfig shows:

dsl0 Link encap:Point-Point Protocol
inet addr:10.164.56.1 P-t-P:10.164.56.5 Mask:255.255.255.255
UP PINTPINT NARP MTU:1500 Metric:1
RX packets:2236 errors:219371 dropped:46026 overruns:0
frame:173345
TX packets:764 errors:8215 dropped:0 overruns:0 carrier:8215
collisions:0 txqueuelen:100
RX bytes:133020 (129.9 KiB) TX bytes:53560 (52.3 KiB)
Interrupt:14

As we see "RUNNING" has disappeared as expected. It does reappear when
the cable is reconnected. We also have an SNMP agent that reports
correct Admin and status as far as I can tell.

When cable is connected Zebra "show interface" reports:

Interface dsl0 is up, line protocol is up
index 3 metric 1 mtu 1500 <UP,PINTPINT,RUNNING,NARP>
inet 10.164.56.1/32 pointopoint 10.164.56.5
2284 input packets (0 multicast), 135868 bytes, 46026 dropped
219371 input errors, 0 length, 0 overrun, 0 CRC, 173345 frame
0 fifo, 0 missed
782 output packets, 54788 bytes, 0 dropped
8215 output errors, 0 aborted, 8215 carrier, 0 fifo, 0 heartbeat
0 window, 0 collisions

When disconnected:

Interface dsl0 is up, line protocol is down
index 3 metric 1 mtu 1500 <UP,PINTPINT,NARP>
inet 10.164.56.1/32 pointopoint 10.164.56.5
2315 input packets (0 multicast), 137653 bytes, 69958 dropped
332852 input errors, 0 length, 0 overrun, 0 CRC, 262894 frame
0 fifo, 0 missed
793 output packets, 55558 bytes, 0 dropped
12283 output errors, 0 aborted, 12283 carrier, 0 fifo, 0 heartbeat
0 window, 0 collisions

When connected SPF "show ip ospf interface" reports:

dsl0 is up
ifindex 3, MTU 1500 bytes, BW 0 Kbit <UP,PINTPINT,RUNNING,NARP>
Internet Address 10.164.56.1/32, Peer 10.164.56.5, Area 0.0.0.0
MTU mismatch detection:enabled
Router ID 10.164.56.1, Network Type PINTPINT, Cost: 1
Transmit Delay is 1 sec, State Point-To-Point, Priority 1
No designated router on this network
No backup designated router on this network
Multicast group memberships: SPFAllRouters
Timer intervals configured, Hello 10s, Dead 40s, Wait 40s, Retransmit
5
Hello due in 9.298s
Neighbor Count is 1, Adjacent neighbor count is 1

And when the cable is out:

dsl0 is down
ifindex 3, MTU 1500 bytes, BW 0 Kbit <UP,PINTPINT,NARP>
SPF is enabled, but not running on this interface

I have now have the "link-detect" option in zebra.conf which did change
the results somewhat and produce the situation above. In addition I do
now get an alternative route to a neighbor when a cable is pulled.
Trouble is the original direct route is still in the kernel and seems to
be selected by the kernel instead of the working alternative.

For example I now have three routers on my desk connected in a ring:

R1(dsl0) =R2(dsl1)
R2(dsl0) =R3(dsl1)
R3(dsl0) =R1(dsl1)

With all cables in R1 shows routes:

# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
10.164.56.9 * 255.255.255.255 UH 0 0 0
dsl1
10.164.56.1 10.164.56.5 255.255.255.255 UGH 2 0 0
dsl0
10.164.56.5 * 255.255.255.255 UH 0 0 0
dsl0
10.164.56.0 * 255.255.252.0 U 0 0 0
eth0

Now I pull the cable between 10.164.56.1 and 10.164.56.9. After a while
I have routes:

~ # route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
10.164.56.9 * 255.255.255.255 UH 0 0 0
dsl1
10.164.56.9 10.164.56.5 255.255.255.255 UGH 2 0 0
dsl0
10.164.56.1 10.164.56.5 255.255.255.255 UGH 2 0 0
dsl0
10.164.56.5 * 255.255.255.255 UH 0 0 0
dsl0
10.164.56.0 * 255.255.252.0 U 0 0 0
eth0

Here we see a new route has appeared to 10.164.56.9 via 10.164.56.5 but
the original direct route is still there and seems to be selected by the
kernel, even if it is not running, as I can no longer reach 10.164.56.9

Please, can you provide the output of show ip route and show ip ospf
database too before and after pulling down the cable ?

Phew ! Hope that helps pin this down.

At this point I'm not sure if quagga/ospf is even supposed to cope with
this situation.

It could be, I have seen a bug like that we fixed some time agos, but
there were 2 levels of bugs: kernel and Quagga.

Regards,
Vincent

Regards,

Michael.

Ystavallisin terveisin / Best Regards

Michael Rychlik

Swarco Technology
Niittylanpolku 16
FIN-00620 Helsinki

Tel: +358 20 741 0325
Mobile: +358 40 822 8586
Fax: +358 9 777 3103
E-mail: Michael.Rychlik (AT) swarco (DOT) com
Web: http://www.swarco.com

This message contains information that may be privileged or confidential and is the property of Swarco. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute or use this message or any part thereof. If you receive this message in error, please notify the sender immediately by reply e-mail and delete all copies of this message. Thank you for your cooperation.

Message

From: Andrew J. Schorr [mailto:aschorr (AT) telemetry-investments (DOT) com]
Sent: 8. toukokuuta 2006 17:29
To: Rychlik, Michael - FI
Cc: quagga-users (AT) lists (DOT) quagga.net
Subject: Re: [quagga-users 6851] ospf over unnumbered DSL interfaces.

Mon, May 08, 2006 at 10:54:33AM +0200, Rychlik, Michael - FI wrote:

>>So far SPF does successfully distribute routes around our "loop". The
>>problem comes when a link goes down (Cable out). SPF then correctly
>>reroutes around the broken link for all routers except for the ones on
>>each end of the disconnected link. Seems neighbors will always route

to

>>each other across the direct interface between them even if is not
>>running.

Which operating system are you using, and which version of quagga?
Is the /S detecting the link failure? If linux, what's the output
of "/sbin/ip addr ls" before and after the link failure?
Similarly, what's the output of zebra "show interface" and
ospfd "show ip ospf interface"? Also, have you tried turning
on link-detect for the interfaces in your zebra config?

Regards,
Andy

Quagga-users mailing list
Quagga-users (AT) lists (DOT) quagga.net

Quagga-users mailing list
Quagga-users (AT) lists (DOT) quagga.net