OpenBSD Commonly Encountered Problems

Using 21143-based NICs

Unlike most of my OpenBSD Frequently Encountered Problems articles, I have no hands-on experience with these cards. So, I have relied upon the knowledge and wisdom of Henning Brauer, a frequent and valued contributor to the OpenBSD (and other) mail lists. Henning also has the added qualification of having coded support for two PHY drivers used by some 21143-based NICs, and being a serious user of 21143-based NICs in serious production environments.

History

The 21143 and its relatives have an "interesting" history. Originally developed by Digital Equipment Corp., what happened next is somewhat unclear to me, however it looks like (perhaps as part of DEC's purchase by Compaq), parts of the company were spun off -- one of those parts was the silicon fabrication facilities, which were (for the most part?) purchased by Intel, and rights to the production of DEC's complete NICs may have gone to Cabletron (I do not consider my sources for this to be authoritative). For quite some time, I was under the impression that the DEC Ethernet chips were "gone" -- never to be seen again. Then, I became aware of a whole new breed of NIC which utilized the 21143. Links to AUTHORITATIVE sources would be appreciated, and quoting anything in this paragraph as evidence to anything would be quite foolish..

Somewhere along the way, the DEC 21x4x chip ended up being a commonly "cloned" chip. Unlike other "cloned" Ethernet devices, such as the NE2000, the compatibility of the DEC 21x4x clones is imperfect at best. A better word might be "similar" than "clones", similar in the sense that if you have a working driver for one type of chip, another can probably be supported with minor tweaks to the driver, but blind compatibility is not to be expected.. These devices range in performance from junk to quite good. You are probably wondering "Which are good? Which are junk?". I don't have an answer at this time.

Using

The history of the DEC chips has complicated their use. This is unfortunate, as many people have reported that the real 21143 is a VERY good chip.

OpenBSD has not one but TWO drivers which support 21x4x-like chips, 'de' and 'dc', and trying to discern the difference between them from the man pages was not overly productive (at least for me).

Differences between the two drivers:

de(4) is more in the style of a Linux driver -- bus interface, main chip and PHY are all in one monolithic driver. 'de' was written for the 10Mbps 21x4x chips, and does not perform overly well for the 100Mbps chips. This driver works with EVERY DEC 21x4x chip, not all the "similars", but doesn't always work well when it works.

dc(4) is a modular driver was written from scratch for the 21143 and similar chips. Bus interfaces modules are separated for PCI and CardBus, as are the various PHY options.

At the moment, the 'de' driver is loaded with higher priority than the 'dc' driver -- this means that if your board COULD be handled by the 'de', it will be..even if 'dc' would do better. This means if you need or wish to use the 'dc' driver instead, you will need to disable the 'de' driver. It was hoped this would change for OpenBSD 3.0, but it doesn't appear to have.

This is done by User Kernel Configuration (UKC), either temporarily at boot, or permanently using the config command.

To make a temporary change, suitable for installation or to get yourself to where you can do a permanent change, when the system first boots, it will display a prompt that looks like:
    boot>
At this prompt, enter 'boot -c':
    boot> boot -c
This will load the kernel, but give you the UKC prompt before you get too far in the boot.
    UKC>
At this point, you can give the "disable de" command:
    UKC>disable de
    UKC>quit
and the boot will resume, this time without the 'de'.

To make this change permanent, you have two options:
1) Build a new kernel with de disabled.
2) Disable the de driver permanently in the kernel:

# config -ef /bsd
OpenBSD 2.9-current (RAID) #1: Thu Jun 28 09:47:40 EDT 2001
njh@devl:/usr/src/sys/arch/i386/compile/RAID
Enter 'help' for information
ukc> disable de*
57 de* disabled
ukc> quit
Saving modified kernel.
#

Ta-da! No more de.

NOTE! If you installed a system using the 'de' driver and decide you would rather use the 'dc' driver, you will have a number of things you will need to change, most likely:

/etc/hostname.de?
/etc/dhcpd.interfaces
/etc/ipf.rules
/etc/ipnat.rules

Do a 'grep de[0123] /etc/*' to look for any other files you need to change.

Quirks:

Multiple 21143's: The dc driver, as shipped in OpenBSD 2.9 release, does not properly handle multiple 21143 chips in one system. This problem has been fixed in -current. The system would crash upon configuring the second instance of the 21143, whether it be on a separate NIC or a multi-port card. If -current isn't an option for you (and this is understandable), Henning reports that simply using revision 1.15 of sys/dev/pci/if_dc_pci.c from the source tree on your -stable or -release system will resolve the problem. Obviously, this requires rebuilding the kernel, which may require a temporary NIC (or using the de driver) if you have a multi-port NIC in the machine.
Multi-port cards: If you have a multi-port NIC, such as the Dlink DFE-570TX, you will find a curious quirk of the dc driver when used with multi-port cards: all the ports are given the same MAC address. This is technically not wrong, as any fan of Sun computers will assure you, it us just not what many switches are expecting. This will not cause any problem unless multiple ports are plugged into the same switch, which is an unusual condition. As of the time of this writing, a Problem Report has been filed, but it has not been fixed.
de driver has bad performance for 21143: The 'de' driver does handle the 21143, but poorly. Henning reports a top throughput of 2Mbps -- well under what one would expect from a very cheap and very bad 10Mbps card, not a quality 100Mbps card! The problem is very noticeable, and the solution is to disable the de driver, as detailed above.
PHY Incompatibility: Some 21143 cards may be recognized by the 'de' driver, but have a PHY device not in the monolithic de driver, even though OpenBSD might well have a modular driver compatible with the card's PHY. If the PHY isn't properly set up, the NIC won't work, even though it is recognized. Solution, again, is disable the 'de' driver, use 'dc' and its modular PHY support.

Is this too much work to go through? Well, to quote Henning in one of his messages to me on the subject, "This chip and the cards based on it are too great not to be used due to a silly driver bug." If you need a good card for a good price, the challenges are worth the trouble. Further, the 21143 is used in many (most?) Quad-port NICs (four complete 10/100 Ethernet ports on one card), and this is a good reason to look at these chips.

Quad-port Ethernet cards

A common question on the lists is "Is the <brand> <model> four-port NIC supported by OpenBSD?" followed closely by "O.k., how?" A reasonable question, as they seem like a fairly exotic beast. However, they really aren't quite as strange as they might first appear. They normally consist of a PCI-PCI bridge (a circuit which links two separate PCI busses), and the circuitry for four complete NICs. PCI-PCI bridge circuits are typically handled in mostly the same way, so that isn't a problem, and then, it just appears to be four separate NICs to the computer -- one DLink 570TX will appear to be four 'dc' devices, dc0, dc1, dc2 and dc3 (See Henning's dmesg). That's it. Nothing to it. The only problem (assuming you install the fix to that annoying crash problem) is the current 'dc' driver gives all the NICs the same MAC address, but again, this should cause no problem for MOST implementations.

Footnotes

PHY: "Physical Layer Device". How exactly they went from "Physical Layer Device" to "PHY", and why it is in all caps, no explanation, other than the computer industry's love of TLAs (Three Letter Abbreviations/Acronyms). Anyway, this is the portion of the NIC which goes between the MII (Media Independent Interface) output of the NIC's main chip and the "Magnetics Module" (basically, a high-frequency isolation transformer), it is responsible for shaping and conditioning the signal from the wire (or more accurately, the Magnetics Module) and presenting the NIC's main chip with the info it needs. The importance of the PHY is not to be underestimated -- if you have perfect wires, perfect hub, and perfect NICs all over your network, it may not matter much, but a quality PHY may well save you from many bad incoming packets in a real-world network.

Relatives: Some relatives of the 21143 card (collectively refered to as the 21x4x):
21040, 21041: 10Mbps chips. Run fine with the de driver.
21140, 21142, 21143: 10/100Mbps chips. Of these, only the 21143 is currently in production, and the older ones did not seem to see wide circulation. This family carried the name "Tulip" when it was labeled by DEC, it appears that since Intel has taken production over, the "Tulip" name has been lost officially, but many of us still call it the "DEC Tulip chip"

Cloned: How did other companies "clone" the 21143 without legal problems? Again, links to authoritative sources would be appreciated. I would guess these "clones" got away with it by being sloppy on their duplication. On the other hand, there is plenty of precident for making software and hardware compatable duplicates of successful products, on the other hand, there are also plenty of examples of legal action over this. A related question is why would they "clone" the chip if it is only going to be sorta-compatable?

Monolithic: A driver consisting of the chip driver, the bus interface driver and the PHY driver in one.

Modular: A driver which only supports the chip to the MII point, and uses a separate PHY driver. This potentially not only reduces effort and code size, but also makes it possible that new cards can be supported automatically if they use an already supported chip and and PHY, just in a new combination.

Quad-port: DEC apparently provided a "Reference Design" for a quad-port PCI NIC. These "Reference Designs" are intended to be used to show the proper utilization of a chip, in order to help its acceptance. This reference design was good enough that a number of companies utilize this design in their four-port cards, including Compushack and DLink. Makes sense -- why reinvent the wheel when the chip manufacturer has done the R&D work for you?

dmseg: Henning Brauer has provided me a part of the dmesg from one of his machines which has TWO quad-port Compushack cards in it:

ppb1 at pci0 dev 8 function 0 "DEC DECchip 21152 PCI-PCI" rev 0x03
pci2 at ppb1 bus 2
dc0 at pci2 dev 0 function 0 "DEC DECchip 21142/3" rev 0x41: irq 10 address 00:00:cb:26:11:bc
icsphy0 at dc0 phy 5: ICS1892 10/100 media interface, rev. 0
dc1 at pci2 dev 4 function 0 "DEC DECchip 21142/3" rev 0x41: irq 10 address 00:00:cb:26:11:bc
icsphy1 at dc1 phy 5: ICS1892 10/100 media interface, rev. 0
dc2 at pci2 dev 8 function 0 "DEC DECchip 21142/3" rev 0x41: irq 10 address 00:00:cb:26:11:bc
icsphy2 at dc2 phy 5: ICS1892 10/100 media interface, rev. 0
dc3 at pci2 dev 12 function 0 "DEC DECchip 21142/3" rev 0x41: irq 10 address 00:00:cb:26:11:bc
icsphy3 at dc3 phy 5: ICS1892 10/100 media interface, rev. 0
ppb2 at pci0 dev 9 function 0 "DEC DECchip 21152 PCI-PCI" rev 0x03
pci3 at ppb2 bus 3
dc4 at pci3 dev 0 function 0 "DEC DECchip 21142/3" rev 0x41: irq 11 address 00:00:cb:26:11:cc
icsphy4 at dc4 phy 5: ICS1892 10/100 media interface, rev. 0
dc5 at pci3 dev 4 function 0 "DEC DECchip 21142/3" rev 0x41: irq 11 address 00:00:cb:26:11:cc
icsphy5 at dc5 phy 5: ICS1892 10/100 media interface, rev. 0
dc6 at pci3 dev 8 function 0 "DEC DECchip 21142/3" rev 0x41: irq 11 address 00:00:cb:26:11:cc
icsphy6 at dc6 phy 5: ICS1892 10/100 media interface, rev. 0
dc7 at pci3 dev 12 function 0 "DEC DECchip 21142/3" rev 0x41: irq 11 address 00:00:cb:26:11:cc
icsphy7 at dc7 phy 5: ICS1892 10/100 media interface, rev. 0

You can see quite a few things of interest here -- first entry is the PCI-PCI bridge. The next four 'dc' devices are on the first card. Note that all the ports share the same interrupt (something that is quite permissible in a PCI environment), and have the same MAC address. You then see another PCI-PCI bridge and another four ports. This machine has at least three PCI busses on it -- the one on the main board, and one on each of the two four-port NICs. Eight network ports. Very nice.

Back to OpenBSD Commonly Encountered Problems page
Holland Consulting home page
Contact Holland Consulting

since September 10, 2001

(C)opyright 2000, Nick Holland, Holland Consulting