hjazz6 Posted January 12, 2009 Report Share Posted January 12, 2009 Hi, I have an IBM Bladecenter HS21 with 3 blade servers. All 3 blade servers were initially installed with RedHat Linux 4. I then installed ixMOS on one of the blade servers, and Mandriva Linux was also installed (over RedHat) automatically along with ixMOS. Now, I have some network problems. The other 2 servers with RedHat can ping each other, but it appears that the server with ixMOS/Mandriva Linux is isolated, that is, I can't ping the other 2 servers from it, and I can't ping it from the other 2 servers. Does anyone have any ideas on what is wrong and how I can solve this problem? One more thing, my installation of Mandriva Linux has no GUI, so I have to do everything using the command line interface. Thank you. Quote Link to comment Share on other sites More sharing options...
ianw1974 Posted January 12, 2009 Report Share Posted January 12, 2009 Do you have an IP address assigned on both of these machines? Please post the output of: ifconfig -a so that we can see if a network interface is up and running. You should be seeing an eth0 interface configured with an IP address. If you only see lo, then that is the loopback interface, and so could mean you don't have any networking. What version of Mandriva Linux did you install? Or version of ixMOS (never heard of that one)? Quote Link to comment Share on other sites More sharing options...
stv4rn0 Posted January 12, 2009 Report Share Posted January 12, 2009 I can't ping the other 2 servers from it, and I can't ping it from the other 2 servers. The default route (for outgoing) and firewall settings (for incoming traffic) may be worth checking. Quote Link to comment Share on other sites More sharing options...
hjazz6 Posted January 12, 2009 Author Report Share Posted January 12, 2009 I'm not at the machines now, so I can't give the exact ifconfig information. But I have to manually assign the IP addresses for all 3 servers: Mandriva server: 192.168.168.202 RedHat #1: 192.168.168.201 RedHat #2: 192.168.168.203 Using ifconfig, I do see the eth0 interface with the IP address (after I've set them) for all three servers. I don't know what version of Mandriva Linux it was, and the ixMOS is a software for traffic analysis. The version of ixMOS is 4.1.3, and the Mandriva installation came from the ixMOS installation disc. The default route (for outgoing) and firewall settings (for incoming traffic) may be worth checking. How do I check these? Quote Link to comment Share on other sites More sharing options...
ianw1974 Posted January 12, 2009 Report Share Posted January 12, 2009 The easiest way of checking the route is: route -n this only works as root, so if you're a regular user, then: netstat -nr will do the same. Also, since they are both in the same IP range, it won't be a routing issue - so just make sure they are in the same network as each other and not separated by VLAN's or something else. Also try: ifconfig eth0 up in case the interface is down. For the Mandriva release: cat /etc/mandriva-release and post the results so we know what version you have. Quote Link to comment Share on other sites More sharing options...
hjazz6 Posted January 13, 2009 Author Report Share Posted January 13, 2009 (edited) The easiest way of checking the route is: route -n I have the following output: Mandriva (192.168.168.202): Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.168.0 0.0.0.0 255.255.255.0 U 5 0 0 eth0 169.254.0.0 0.0.0.0 255.255.0.0 U 5 0 0 eth0 0.0.0.0 192.168.168.1 0.0.0.0 UG 5 0 0 eth0 RedHat #1 (192.168.168.201): Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.168.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 RedHat #2 (192.168.168.203): Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.168.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 0.0.0.0 192.168.168.1 0.0.0.0 UG 0 0 0 eth0 For the Mandriva release: cat /etc/mandriva-release and post the results so we know what version you have. I got "x86_64 smp" One more thing I found out. I looked at /etc/sysconfig/network-scripts/ifcfg-eth0 of the Mandriva server, which contains the following: DEVICE = eth0 BOOTPROTO=static IPADDR=192.168.168.202 NETMASK=255.255.255.0 GATEWAY=192.168.168.1 ONBOOT=yes METRIC=5 MII_NOT_SUPPORTED=yes USERCTL=no DNS1=192.168.168.2 RESOLV_MODS=yes LINK_DETECTION_DELAY=6 IPV6INIT=no IPV6to4INIT=no and noticed that if the line MII_NOT_SUPPORTED equals "yes", then the server has the IP address 192.168.168.202 upon startup, but if MII_NOT_SUPPORTED=no, then the server has no IP address upon startup, and I have to manually assign one. Also, when I typed "mii-tool" on the Mandriva server (regardless of MII_NOT_SUPPORTED=yes or =no), I get the following: SIOCGMIIPHY on 'eth0' failed: Operation not supported SIOCGMIIPHY on 'eth1' failed: Operation not supported SIOCGMIIPHY on 'eth2' failed: Resource temporarily unavailable SIOCGMIIPHY on 'eth3' failed: Resource temporarily unavailable no MII interfaces found But on the 2 RedHat servers, which didn't contain the MII_NOT_SUPPORTED lines, I get the following output: eth0: negotiated 10baseT-HD, link ok SIOCGMIIPHY on 'eth1' failed: Resource temporarily unavailable SIOCGMIIPHY on 'eth2' failed: Operation not supported SIOCGMIIPHY on 'eth3' failed: Operation not supported I ran the ethtool program and got the following output. output of "ethtool -t etho" on Mandriva server The test result is FAIL The test extra info: nvram test (online) 0 link test (online) 1 register test (offline) 0 memory test (offline) 0 loopback test (offline) 0 interrupt test (offline) 0 output of "ethtool -t etho" on both RedHat servers The test result is PASS The test extra info: register test (offline) 0 memory test (offline) 0 loopback test (offline) 0 nvram test (online) 0 interrupt test (online) 0 link test (online) 0 output of "ethtool eth0" on Mandriva server: Settings for eth0: Supported ports: [ FIBRE ] Supported link modes: 1000baseT/Half 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 1000baseT/Half 1000baseT/Full Advertised auto-negotiation: Yes Speed: Unknown! (65535) Duplex: Unknown! (255) Port: FIBRE PHYAD: 1 Transceiver: internal Auto-negotiation: on Supports Wake-on: g Wake-on: d Current message level: 0x000000ff (255) Link detected: no output of "ethtool eth0" on RedHat servers: Settings for eth0: Supported ports: [ FIBRE ] Supported link modes: 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 1000baseT/Full Advertised auto-negotiation: Yes Speed: 1000 Mb/s Duplex: Full Port: FIBRE PHYAD: 2 Transceiver: internal Auto-negotiation: on Supports Wake-on: g Wake-on: g Link detected: yes Does this mean that there are some problems with the eth0 interface of the Mandriva server? Thank you. Edited January 13, 2009 by hjazz6 Quote Link to comment Share on other sites More sharing options...
ianw1974 Posted January 13, 2009 Report Share Posted January 13, 2009 MII doesn't seem to be supported on your network card, which is why you get an IP address assigned when it's set to yes than compared to no. Please post the contents of the /etc/sysconfig/network-scripts/ifcfg-eth0 from the Red Hat machines so we can compare them. Incidently, take a copy of the Mandriva one, and make it look similar to the Red Hat one by deleting the lines in the Mandriva one that don't exist in the Red Hat one. Then see if you're networking works OK. I still don't know which version of Mandriva you have, and unless you can find out from wherever you got it from, it could even be that you have an old version and it simply doesn't like your hardware. I would suggest that if you don't know what version it is, to download the latest version available from Mandriva's website. It will be difficult for us to troubleshoot if you can't verify what version it is. Alternatively, please do: rpm -qa from the Mandriva machine, and this will list the versioning information in the rpm package name normally along the lines of mdv2008.1 or mdv2009.0 and so on. Normally /etc/mandriva-release would have had the Mandriva version, but since yours doesn't display this I can't even be sure you have a pure Mandriva and not something that has been hacked around with. You could also try: cat /etc/mandrake-release but if it gets the same result, then I don't think it's a normal copy of Mandriva. Either way, I suggest you download the latest version from Mandriva's website if you're not anywhere near the 2009 version. Quote Link to comment Share on other sites More sharing options...
hjazz6 Posted January 13, 2009 Author Report Share Posted January 13, 2009 MII doesn't seem to be supported on your network card, which is why you get an IP address assigned when it's set to yes than compared to no. How could this be when all 3 servers reside on the same BladeCenter? The only difference between the servers is that one runs Mandriva and the other 2 runs RedHat. Please post the contents of the /etc/sysconfig/network-scripts/ifcfg-eth0 from the Red Hat machines so we can compare them. Incidently, take a copy of the Mandriva one, and make it look similar to the Red Hat one by deleting the lines in the Mandriva one that don't exist in the Red Hat one. Then see if you're networking works OK. I've tried making the Mandriva server's ifcfg-eth0 similar to RedHat #2, but it didn't help. Contents of /etc/sysconfig/network-scripts/ifcfg-eth0 on RedHat #1: DEVICE=eth0 BOOTPROTO=none HWADDR=00:21:5E:22:03:78 ONBOOT=yes TYPE=Ethernet NETMASK=255.255.255.0 IPADDR=192.168.168.201 USERCTL=no IPV6INIT=no PEERDNS=yes Contents of /etc/sysconfig/network-scripts/ifcfg-eth0 on RedHat #2: DEVICE=eth0 BOOTPROTO=none HWADDR=00:21:5E:22:03:A2 ONBOOT=yes TYPE=Ethernet NETMASK=255.255.255.0 IPADDR=192.168.168.203 USERCTL=no IPV6INIT=no PEERDNS=yes GATEWAY=192.168.168.1 Does it matter that the HW address of the Mandriva server (00:1A:64:DD:0E:C6) is quite different to the 2 RedHat servers? I still don't know which version of Mandriva you have, and unless you can find out from wherever you got it from, it could even be that you have an old version and it simply doesn't like your hardware. I would suggest that if you don't know what version it is, to download the latest version available from Mandriva's website. It will be difficult for us to troubleshoot if you can't verify what version it is. Alternatively, please do: rpm -qa from the Mandriva machine, and this will list the versioning information in the rpm package name normally along the lines of mdv2008.1 or mdv2009.0 and so on. I did this and got the following 3 entries with "mandriva" in it. rpm-mandriva-setup-1.63-1mdv2008.0 mandriva-release-Free-2008.0-5mdv2008.0 mandriva-release-common-2008.0-5mdv2008.0 cat /etc/mandrake-release gives me the same x86_64 smp result. Quote Link to comment Share on other sites More sharing options...
stv4rn0 Posted January 13, 2009 Report Share Posted January 13, 2009 Settings for eth0: Supported ports: [ FIBRE ] Supported link modes: 1000baseT/Half 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 1000baseT/Half 1000baseT/Full Advertised auto-negotiation: Yes Speed: Unknown! (65535) Duplex: Unknown! (255) Port: FIBRE PHYAD: 1 Transceiver: internal Auto-negotiation: on Supports Wake-on: g Wake-on: d Current message level: 0x000000ff (255) Link detected: no looks like your Ethernet interface is down; the command (the output of which you didn't post so far) $ ifconfig eth0 should tell you if it's up or not Quote Link to comment Share on other sites More sharing options...
ianw1974 Posted January 13, 2009 Report Share Posted January 13, 2009 How could this be when all 3 servers reside on the same BladeCenter? The only difference between the servers is that one runs Mandriva and the other 2 runs RedHat. The Red Hat configs don't have it noted in the config file, and so maybe the default is for MII to be disabled, and why they have an IP address. I don't know, just a guess..... The other problem could be that Mandriva doesn't have the correct module for your network card, and doesn't work properly. Does it matter that the HW address of the Mandriva server (00:1A:64:DD:0E:C6) is quite different to the 2 RedHat servers? No it shouldn't do as the hardware address is supposed to be different. I did this and got the following 3 entries with "mandriva" in it. rpm-mandriva-setup-1.63-1mdv2008.0 mandriva-release-Free-2008.0-5mdv2008.0 mandriva-release-common-2008.0-5mdv2008.0 cat /etc/mandrake-release gives me the same x86_64 smp result. you have Mandriva 2008.0. This should be OK, since it's relatively new, but will soon be unsupported by Mandriva for updates, etc. I'm surprised though that RHEL4 is working, since it uses a much older kernel than Mandriva 2008.0. Then again, maybe Mandriva is not certified to work on your particular hardware. Please provide the output from lspci for each of the machines, so I can see what network card has been detected in the RHEL/Mandriva installations. Did you also do the command I asked earlier? ifconfig eth0 up in case, as mentioned by the last post, that your interface is simply down and not usable. Quote Link to comment Share on other sites More sharing options...
hjazz6 Posted January 14, 2009 Author Report Share Posted January 14, 2009 looks like your Ethernet interface is down; the command (the output of which you didn't post so far)$ ifconfig eth0 should tell you if it's up or not eth0 is always up after I boot the server. The other problem could be that Mandriva doesn't have the correct module for your network card, and doesn't work properly. You think this could be an OS problem? Would it help if I reinstall RedHat on this server, since it appears that the RedHat servers have no problems? Please provide the output from lspci for each of the machines, so I can see what network card has been detected in the RHEL/Mandriva installations. I got a "command not found" message when using lspci on the Mandriva server. The portions of the lspci output that I think are relevant: RedHat #1: 03:00:0 PCI bridge: Broadcom: Unknown device 0103 (rev c3) 04:00:0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (rev 12) 05:00:0 PCI bridge: Broadcom: Unknown device 0103 (rev c3) 06:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (rev 12) 0c:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10) 0c:01.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10) RedHat #2: 03:00:0 PCI bridge: Broadcom: Unknown device 0103 (rev c3) 04:00:0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (rev 12) 05:00:0 PCI bridge: Broadcom: Unknown device 0103 (rev c3) 06:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (rev 12) 08:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10) 08:01.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S Gigabit Ethernet (rev 10) I also did a lsmod on all 3 servers, which gave me a whole list of modules. The one that looked familar was On RedHat #1 and #2: Module Size Used by bnx2 181512 0 On Mandriva: Module Size Used by bnx2 160408 0 Also, looking through /var/log/messages of all 3 servers, I have (relevant portions) RedHat #1 eth0: Broadcom NetXtreme II BCM5708 1000Base-SX(B2) PCI-X 64-bit 133MHz found at mem da000000, IRQ 209, node addr 00:21:5e:22:03:78 eth1: Broadcom NetXtreme II BCM5708 1000Base-SX(B2) PCI-X 64-bit 133MHz found at mem d8000000, IRQ 193, node addr 00:21:5e:22:03:7a eth2: Tigon3[partno(BCM95704s) rev2100 PHY(serdes)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:1a:64:dd:0e:a2 eth2: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Spilt[0] WireSpeed[0] TS0cap[1] eth2: dma_rwctrl[769f4000] eth3: Tigon3[partno(BCM95704s) rev2100 PHY(serdes)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:1a:64:dd:0e:a3 eth3: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Spilt[0] WireSpeed[0] TS0cap[1] eth3: dma_rwctrl[769f4000] [...] bnx2: eth0: using MSI bnx2:eth0 NIC SerDesLink is Up, 1000Mbps full duplex RedHat #2 eth0: Broadcom NetXtreme II BCM5708 1000Base-SX(B2) PCI-X 64-bit 133MHz found at mem da000000, IRQ 255, node addr 00:21:5e:22:03:a2 eth1: Broadcom NetXtreme II BCM5708 1000Base-SX(B2) PCI-X 64-bit 133MHz found at mem d8000000, IRQ 193, node addr 00:21:5e:22:03:a4 eth2: Tigon3[partno(BCM95704s) rev2100 PHY(serdes)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:1a:64:dd:09:a8 eth2: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Spilt[0] WireSpeed[0] TS0cap[1] eth2: dma_rwctrl[769f4000] eth3: Tigon3[partno(BCM95704s) rev2100 PHY(serdes)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:1a:64:dd:09:a9 eth3: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Spilt[0] WireSpeed[0] TS0cap[1] eth3: dma_rwctrl[769f4000] [...] bnx2: eth0: using MSI bnx2:eth0 NIC SerDesLink is Up, 1000Mbps full duplex Mandriva Broadcom NetXtreme II Gigabit Ethernet Driver bx2 v1.5.11 (June 4, 2007) eth0: Broadcom NetXtreme II BCM5708 1000Base-SX(B2) PCI-X 64-bit 133MHz found at mem da000000, IRQ 17, node addr 00215e220366 eth1: Broadcom NetXtreme II BCM5708 1000Base-SX(B2) PCI-X 64-bit 133MHz found at mem d8000000, IRQ 19, node addr 00215e220368 eth0: Tigon3[partno(BCM95704s) rev2100 PHY(serdes)] (PCIX:133MHz:64-bit) 1000Base-SX Ethernet 00:1a:64:dd:0e:c6 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] WireSpeed[0] TS0cap[1] eth0: dma_rwctrl[769f4000] dma_mask[64-bit] eth1: Tigon3[partno(BCM95704s) rev2100 PHY(serdes)] (PCIX:133MHz:64-bit) 1000Base-SX Ethernet 00:1a:64:dd:0e:c7 eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] WireSpeed[0] TS0cap[1] eth1: dma_rwctrl[769f4000] dma_mask[64-bit] [...] ntpd[3758]: precision = 1.000 usec ntpd[3758]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled ntpd[3758]: Listening on interface #1 wildcard, ::#123 Disabled ntpd[3758]: Listening on interface #2 lo, ::1#123 Enabled ntpd[3758]: Listening on interface #3 lo, 127.0.0.1#123 Enabled ntpd[3758]: Listening on interface #4 eth0, 192.168.168.202#123 Enabled ntpd[3758]: Kernel time sync status 0040 So it appears that eth0 on both RedHat servers use Broadcom NetXtreme II , and the Mandriva server used that initially, but then overwrote that and used Tigon3 instead. The HWaddr from ifconfig confirms this. Could this be the problem? Quote Link to comment Share on other sites More sharing options...
ianw1974 Posted January 14, 2009 Report Share Posted January 14, 2009 You can check to see if the module is the same on Mandriva by checking if it's the bnx2 or tg3 module that's been loaded. Now, what you could do, if it's using bnx2, is: modprobe -r tg3 modprobe bnx2 then check if you have an eth0 interface, or any eth device, and then configure with IP address and see if you get a connection. The lspci command could have failed if there was no path for the executable. To find it do: which lspci or: whereis lspci however, it should work if you were logged in as root, or gained superuser privileges by using su first. If nothing after that, install the pciutils package as it might not be installed, and why lspci is not there. Quote Link to comment Share on other sites More sharing options...
hjazz6 Posted January 15, 2009 Author Report Share Posted January 15, 2009 Turns out that there are 2 modules in the Bladecenter, BCM5708 and BCM95704. Both RedHat servers used BCM5708 for eth0 and eth1, and BCM95704 for eth2 and eth3. However, during the boot process, the Mandriva server used BCM95704 for eth0 and eth1 instead. After editing udev's persistent rules, I finally got the servers to ping each other. ianw1974 and stv4rn0, thank you so much for your help! Quote Link to comment Share on other sites More sharing options...
ianw1974 Posted January 15, 2009 Report Share Posted January 15, 2009 Nice one, glad you got it sorted Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.