Overview

You have an appliance that has been upgraded to 9.4.2 patch 1 and you noticed that everything is very slow with a lot of dropped packets. The appliance feels non-responsive.

The Generic Receive Offload article has a great explanation of what GRO is and how it works.

Solution

For a Software appliance running in any Hypervisor

Ensure that you are using Intel E1000 or VMXnet3 drivers for the adapters. If that doesn't fix the issue, please proceed with these steps:

This solution works until an upgrade or a downgrade since the ifupdown script gets rewritten.

1. Access Kerio Control's Shell Using SSH. Please copy-paste all the code of each step as a single line.
2. Verify the current state of the generic-receive-offload by running this line:
   hwinfo --network --short | tail -n +2 | awk '{print $1}' | xargs -i sh -c 'echo {}; ethtool -k {} | grep generic-receive'
3. Execute this line in SSH It disables GRO (generic receive offload) on all the ethernet interfaces (if you skip this step, make sure to reboot the appliance after the last step):
   hwinfo --network --short | tail -n +2 | awk '{print $1}' | xargs -I {} ethtool -K {} gro off
4. If that fixes the issue, to make the solution permanent, type the following line (and hit Enter) to allow making modifications to the filesystem and add the GRO reset upon all interface changes, and then make the filesystem read-only again:
   mount -o rw,remount /; printf "#/bin/bash\n\nhwinfo --network --short | tail -n +2 | awk '{print \$1}' | xargs -I {} ethtool -K {} gro off" > /usr/bin/gro_fix; chmod 0755 /usr/bin/gro_fix; if ! grep -q "gro_fix" "/usr/bin/ifupdown"; then printf "\n\nsetsid /usr/bin/gro_fix &2>1 $\n" >> /usr/bin/ifupdown; fi; mount -o ro,remount /

For a Hardware appliance:

This solution works until an upgrade or a downgrade since the ifupdown script gets rewritten.

First, reboot the device and don't run a SpeedTest as that makes the issue appear. Then follow the steps below to potentially resolve the problem:

1. Access Kerio Control's Shell Using SSH. Please copy-paste all the code of each step as a single line.
2. Verify the current state of the generic-receive-offload by running this line:
   hwinfo --network --short | tail -n +2 | awk '{print $1}' | xargs -i sh -c 'echo {}; ethtool -k {} | grep generic-receive'
3. Execute this line in SSH It enables GRO (generic receive offload) on all the ethernet interfaces and disables GRO for the software loopback, kvnet and VLAN interfaces (if you skip this step, make sure to reboot the appliance after the last step):
   hwinfo --network --short | tail -n +2 | awk '{print $1}' | grep -v '\.' | xargs -I {} ethtool -K {} gro on; hwinfo --network --short | tail -n +2 | awk '{print $1}' | grep '\.' | xargs -I {} ethtool -K {} gro off; ethtool -K lo gro off; ethtool -K kvnet gro off
4. If that fixes the issue, to make the solution permanent, type the following line (and hit Enter) to allow making modifications to the filesystem and add the GRO reset upon all interface changes, and then make the filesystem read-only again:
   mount -o rw,remount /; printf "#/bin/bash\n\nhwinfo --network --short | tail -n +2 | awk '{print $1}' | grep -v '\.' | xargs -I {} ethtool -K {} gro on\nhwinfo --network --short | tail -n +2 | awk '{print $1}' | grep '\.' | xargs -I {} ethtool -K {} gro off\nethtool -K lo gro off\nethtool -K kvnet gro off\n" > /usr/bin/gro_fix; chmod 0755 /usr/bin/gro_fix; if ! grep -q "gro_fix" "/usr/bin/ifupdown"; then printf "\n\nsetsid /usr/bin/gro_fix &2>1 $\n" >> /usr/bin/ifupdown; fi; mount -o ro,remount /

If you use VLANs a lot, please use these two lines instead of the ones in steps 3 and 4:

hwinfo --network --short | tail -n +2 | awk '{print $1}' | xargs -I {} ethtool -K {} gro on; ethtool -K lo gro off; ethtool -K kvnet gro off 

Permanent solution:

mount -o rw,remount /; printf "#/bin/bash\n\nhwinfo --network --short | tail -n +2 | awk '{print $1}' | xargs -I {} ethtool -K {} gro on\nethtool -K lo gro off\nethtool -K kvnet gro off\n" > /usr/bin/gro_fix; chmod 0755 /usr/bin/gro_fix; if ! grep -q "gro_fix" "/usr/bin/ifupdown"; then printf "\n\nsetsid /usr/bin/gro_fix &2>1 $\n" >> /usr/bin/ifupdown; fi; mount -o ro,remount /