Multi homed host, arp flux and source base routing

If you have a linux box and multiple interfaces on the same subnet you may notice that all traffic is really going through the first interface. This is not a problem unless if you want to differentiate services per interface. For example I have a host with 2 interfaces on subnet 192.168.1.0:

eth0 : 192.168.1.1
eth1 : 192.168.1.2

Now on eth0 I want to run smb and on eth1 Alfresco's smb service.

After making sure that each service binds on a specific interface:

i) Alfresco (3.4b):

File: webapps/alfresco/WEB-INF/classes/alfresc o/subsystems/fileServers/default/file-servers.properties

cifs.enabled=true
cifs.serverName={Your hostname on that interface}
cifs.domain={a workgroup}
cifs.broadcast=192.168.1.255
cifs.bindto=192.168.1.2
cifs.tcpipSMB.port=4450
cifs.netBIOSSMB.sessionPort=1390
cifs.netBIOSSMB.namePort=1370
cifs.netBIOSSMB.datagramPort=1380

Obviously you need to adjust some values to your specific needs. If you like me run tomcat as an unprivileged user you should set higher than 1024 ports and use iptables to redirect:

iptables -t nat -F PREROUTING
iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 445 -j REDIRECT --to-ports 4450
iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 139 -j REDIRECT --to-ports 1390
iptables -t nat -A PREROUTING -i eth1 -p udp -m udp --dport 137 -j REDIRECT --to-ports 1370
iptables -t nat -A PREROUTING -i eth1 -p udp -m udp --dport 138 -j REDIRECT --to-ports 1380

(You should also add rules for INPUT if the default policy is not ACCEPT)

ii) Samba

File: /etc/samba/smb.conf

Add to global:

bind interfaces only = yes
interfaces = lo eth0

Now restarting nmb, smb and tomcat/alfresco will apply all changes. Unfortunately this may not be enough. You have to make sure that traffic for interface eth1 is delivered to it. Or more precisely that outgoing traffic is sent through eth1 when the source is 192.168.1.2

The solution is source based routing:

1) Create a table alias, here called surlatable:

echo 200 surlatable >> /etc/iproute2/rt_tables

2) Traffic with source 192.168.1.2 should be routed according with the newly created table:

ip rule add from 192.168.1.2 table surelatable

3) Local and (optionally) Default routes

ip route add 192.168.1.0/24 via 192.168.1.2 dev eth1 table surelatable
ip route add default via {GATEWAY} dev eth1 table surelatable

4) Apply changes

ip route flush cache

You may want to make these changes permanent by adding the commands in /sbin/if-up.local script (for fedora) when eth1 comes up.

Lastly fix the ARP FLUX issue by editing/adding to /etc/sysctl.conf:

net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1

Modern kernels support these flags and are compiled with Advanced routing features. You can apply these without rebooting:

echo 2 > /proc/sys/net/ipv4/conf/default/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/default/arp_ignore

Now you can verify that everything is working. Make sure to clear the arp cache of the computer you will use as a client. Then ping both interfaces and check the cache. Each address should have the correct MAC address listed.

For more details on advanced routing and much more, don't miss Linux Advanced Routing & Traffic Control HOWTO

I am sorry firefox

I have always been using Firefox for anything that had to do with browsing (apart from some websites that only work with IE). My habit is to leave it running for days and after a point it does become slow. I thought it had to do with its infamous memory leaks and did not believe the official response that caching is raising memory consumption. I was wrong.

I have started using Chrome which does look fast (reality is, it is faster for the most part) when you are not using it :-) Today I woke up and Chrome was almost unresponsive. Here is a screen shot of task manager:

I am really sorry Firefox :-)

ICH5 sound issues with Fedora

I had an Albatron mobo running FC10. All was well and it could handle multiple services running headless for years.

Then I needed to run mpg123 24/7 and I found out how problematic the ICH5 ALC655 chip was with linux. I could get sound however at random instances the audio would go off. At first I though mpg123 was the problem so I ran it under supervise. At a point I came across MPD and I ditched mpg123 for it. I discovered that the same thing happened to mpd. Only I also discovered that audio would come back after a random duration of time.

To cut a long story short I assumed the alsa driver intel8x0 was responsible. Somewhere in the net a mention of a kernel patch solving the issues on FC12 made me preupgrade to FC12. After a few hours of readjusting everything to work I came to find that the sound issues were still there. I activated PulseAudio, a sound server which uses the Alsa drivers and can sit on top of Alsa. As a side note PulseAudio is the default in FC12 however since I preupgraded from FC10 it was not. Then tried mpd using PulseAudio, Alsa (via PulseAudio) with no luck. Oh linux has a wonderful world of sound software which takes forever to configure and understand. Then I read someone suggesting (the older and not really open) OSS as a solution to all your problems. I installed it to find out the problem was still there! Only this time the kernel said:

Jan 26 22:56:12 dryad kernel: irq 17: nobody cared (try booting with the "irqpoll" option)
Jan 26 22:56:12 dryad kernel: Pid: 0, comm: swapper Tainted: P 2.6.32.26-175.fc12.i686 #1
Jan 26 22:56:12 dryad kernel: Call Trace:
Jan 26 22:56:12 dryad kernel: [] __report_bad_irq+0x33/0x74
Jan 26 22:56:12 dryad kernel: [] note_interrupt+0xfa/0x153
Jan 26 22:56:12 dryad kernel: [] handle_fasteoi_irq+0x83/0xa2
Jan 26 22:56:12 dryad kernel: [] handle_irq+0x40/0x4d
Jan 26 22:56:12 dryad kernel: [] do_IRQ+0x46/0x9a
Jan 26 22:56:12 dryad kernel: [] common_interrupt+0x30/0x38
Jan 26 22:56:12 dryad kernel: [] ? mwait_idle+0x61/0x6c
Jan 26 22:56:12 dryad kernel: [] cpu_idle+0x96/0xb2
Jan 26 22:56:12 dryad kernel: [] rest_init+0x58/0x5a
Jan 26 22:56:12 dryad kernel: [] start_kernel+0x33c/0x341
Jan 26 22:56:12 dryad kernel: [] i386_start_kernel+0x97/0x9e
Jan 26 22:56:12 dryad kernel: handlers:
Jan 26 22:56:12 dryad kernel: [] (osscore_intr+0x0/0x4a [osscore])
Jan 26 22:56:12 dryad kernel: Disabling IRQ #17
Jan 26 22:56:12 dryad kernel: osscore: Output timed out on audio engine 1/'Intel ICH5 (24D5) (vmix)' (count=39919616)
Jan 26 22:56:12 dryad kernel: 157856 fifo errors were detected
Jan 26 23:05:20 dryad kernel: 1401 fifo errors were detected
Jan 26 23:07:22 dryad kernel: usbcore: deregistering interface driver oss_usb
Jan 26 23:07:22 dryad kernel: oss_ich 0000:00:1f.5: PCI INT B disabled

As I visualize it that means that the driver was eventually waiting for a write, the IRQ was set however somehow was not serviced by the appropriate handler. A kernel issue perhaps. The result was the blocking call was blocking the driver all the way up to the application which initiated the call.

I did not try the irqpolloption yet. I will do so after I try noapic first. I did try noirqdebug which does not affect the issue.

In the meantime I am trying one of these super cheap usb sound dongles. It does not work really:

Bus 002 Device 002: ID 1130:f211 Tenx Technology, Inc. audio headset

Next thing to try is a YMF724 PCI card found on storage.

UPDATE: YMF724 worked out of the box (mpd cpu dropped around 4% when no streaming). I decided to stop pursuing a solution with the onboard chip. Does not worth it.

alsa-lib-1.0.23
kernel 2.6.32.26-175

alternatives

The alternatives utility automates managing multiple programs with the same function.

For instance if you want to use qmail instead of the default sendmail, install qmail and then issue:

alternatives --install /usr/sbin/sendmail mta /var/qmail/bin/sendmail 50 --slave /usr/lib/sendmail mta-sendmail /var/qmail/bin/sendmail

There are more slave links to explore, but I think it works for the most part.

Then run alternatives --config mta to verify that qmail is the preferred choice.

To go back you can simply run the later command again and make a new choice.