Recently, I was looking for information on what Ethernet cards will work with Nemo drivers for link aggregation/trunking. It was a long search and took me a while to figure out what is supported and what product/technologies should be used on what hardware. So, here are my notes for configuring trunking on various Solaris 10 systems.
The first thing you should do before even begin working on trunking on a Solaris 10 machine is to make sure it has latest patches.
Next, check what kind of Ethernet driver do you have. There are several ways you can find out. Here is one way:
dladm show-link
Look for the first word for each line. If you see any of the following driver types, then you can use Nemo drivers: bge, e1000g, xge, nge, rge, ixgb
If you have qfe, ge, or ce, then you should use Sun Trunking 1.3. See also support matrix.
Let’s say that you have Nemo supported drivers, then here are the steps to setup link aggregation:
1. Identify which ports you want to aggregate. dladm show-link should give you an idea what are available. I’ll assume you want to aggregate bge0 and bge1.
2. Run the following command to create an aggregation interface:
dladm create-aggr -d bge0 -d bge1 1
If you get an error, it is most likly you either attempting to use Nemo drivers on a not supported Ethernet driver or you do not have the latest patches for Solaris 10.
3. Plumb the interface. See example below:
ifconfig aggr1 plub 192.168.1.200 up
4. Check the configuration:
dladm show-aggr ifconfig -a
If you decided that Sun Trunking 1.3 is what you need, then follow the steps below:
1. Download the binary.
2. Read the Installation Guide
3. After unzip the binary from step #1, follow the installation steps stated in page 46 (printed on the left corner on the page or page 56 according to your Acrobat reader) in the Installation guide.
4. Install the appripriate patches for the driver you use. See page 44 (printed on the page or 54 according to Acrobat reader) in the Installation Guide.
5. Set local-mac-address as root on the command-line:
eeprom local-mac-address?=true
6.Reboot
7. Make sure the interfacs that you want aggregate isn’t plumb. If not, use the following command to unplumb:
ifconfig ce0 unplumb
8 Create an aggregated interface
/etc/opt/SUNWconn/bin/nettr -setup 0 device ce members=0,1
This should create an aggregated interface for use with switches (MAC based policy) If you want to use Round Robin for back to back connection, use the following command instead:
/etc/opt/SUNWconn/bin/nettr -setup 0 device ce members=0,1 policy=2
See Page 29 and 32 for further information about other policies.
9. Plumb the interface.
ifconfig ce0 plumb 192.168.1.200 up
10. Check config:
/etc/opt/SUNWconn/bin/nettr -conf
Life will be easier once you get clearview with Solaris.
Dave's Blogs 
Thanks for the nice write up.
RE: Step 8 – Create an aggregated interface
“This should create an aggregated interface for use with switches…”
I’m unsure what needs to be done (if anything) on the switch side to get Sun Trunking 1.3 working.
Do you know if a consumer-grade NetGear GigE switch will work with Sun Trunking? I have a Sun Blade 100 with a QFE card and want to trunk all 4 ports to an unmanaged NetGear GigE switch.
Thanks.
I didn’t have to do anything on the switch to get trunking to work. Switches typically work below TCP/IP layer (MAC), so I would give this a try and see what happens.
It should read “dladm” not “dladmin”, afaik.
Cheers,
Hans
Hi Hans,
Thanks for the correction.
Thanks for the info. It was short, easy to read and to the point. I was able to get it up and running on 6 – x4600’s and 2 4200’s.
You welcome. Good to know that these notes also worked for you.
Thanks for the info. It’s been very helpful. One thing I would like to add to the note is to make sure the interface(s) are unplumbed before add the device to the aggregate.
Connie
Hi Connie,
Thank you for the note.
Hi Dave,
Is there any option to use to select that you want to create a “fat pipe” I have one one 4-port e1000g card and I am trying to use three ports as one one fast link between another machine.
I have both machines setup the same way but it still takes two minutes to transmit 10 GB.
It appears that the e1000g interfaces cannot do Round Robin in Solaris 10. From what I see, Sun Trunking software does not support this interface and dladm does not do Round Robin. I have two servers connected point to point and I can’t configure the x4500 to do Round Robin, only a hash can be used which is not very useful in my configuration.
How i want to download nemo drivers as my server is currently using nge interface?
Two things missing:
1. Explanation of what role plays LACP at switch level, anything to configure? it is not clear based on the text
2. Why nettr was used instead of dladm ? explanation of nettr vs. dladm is missing
I’ve been wanting to set up teaming for a while. I have a V480 running Solaris 10, a cisco 3550-12T (supports LACP), one x86 machine running OpenSolaris, and one x64 machine running OpenSolaris. I can’t seem to find any information on whether all the NICs support this.
V480 – 2 bge
OS1 – 1 rge, 1 nge
OS2 – 1 e1000g, 1 rge
Can you tell me if they will work?