Ticket #17 (closed defect: fixed)

Opened 14 years ago

Last modified 14 years ago

MTU of TUN device can't be increased on IONs

Reported by: iskra Owned by: iskra
Priority: major Milestone:
Component: ZeptoOS Version:
Keywords: Cc:


There seems to be a restriction on the ION kernel regarding the MTU size of the TUN device (which we use for IP forwarding). On the CN, we run a newer kernel ( and can increase that MTU using "ifconfig" to 65535 without any problems. With the older (2.6.16-based) kernel on the IONs, the same ifconfig command returns an error, and we are stuck with the default 1500. As a result, the bandwidth of IP-forwarding infrastructure between CNs and IONs (over the tree network) is much lower than between two CNs (over torus). The ION kernel is capable of higher MTU values on other interfaces (eth0 uses 9000), and a quick diff of the TUN device source between CN and ION kernels does not turn up anything obvious, so it's a bit of a mystery what the problem is.

Change History

comment:1 Changed 14 years ago by iskra

  • Owner changed from zepto team to iskra
  • Status changed from new to assigned

Update: this seems to be working now, however, the performance is not good. It's actually more stable to keep the MTU at 1500, at least it's stable then. Tried disabling flow control, but it didn't improve the stability. Why is the performance so much better between CNs? Could it be due to flat memory? We should try using flat memory for this!

comment:2 Changed 14 years ago by iskra

  • Status changed from assigned to closed
  • Resolution set to fixed

The performance problems have been resolved. The fix was to uncomment the old pthread_yield code in the packet sending/receiving loop. Without the yield, the scheduler could preempt a thread with an IP packet for up to 10ms, even if other cores were available. This could be observed by pinging a CN: it would normally take under 0.2ms, but sometimes over 10ms.

Note: See TracTickets for help on using tickets.