A New York customer had recently purchased another company, so they added that company's email domain to their Kerio Connect mail server. A few users were moved in one direction or another and everything seemed to be going well.
As the newly purchased company needed some hardware upgrades, new Windows 7 machines were purchased for several of their users. That's where the trouble started..
It has to be the domain
The new machines could not seem to synchronize with Kerio.
For those who aren't familiar with Kerio, let me explain that in addition to Webmail, POP3 and IMAP, Kerio supports a MAPI configuration which gives Outlook most of the important features it would have using Microsoft Exchange. This requires the installation of a Kerio Outlook Connector (KOC) on client machines. After installation, the KOC attempts to bring down message headers from the server - synchronizing its view of email with the servers. It was this process that was failing.
IMAP has a similar synchronization and both it and Webmail performed badly on these new machines also, although not quite so poorly as the KOC did.
The situation was described by Rick, their IT guy, to me over the phone and then he followed up with this email (I added comments and explanations in [brackets]):
Given: Both users are at[the new company], and work fine on Windows
XP with Office 2003 SP3 with the Outlook connector
Both users have Windows 7 Pro, and Office 2010.
I have Windows 7 Pro with Office 2010 with -0- problems
The ONLY thing I can think of is the Domain. I have no domain.
So, I asked [tech person at the new company] to set up a new machine,
but don't join the domain.. Then try the KOC setup. NO PROBLEMS!
Then she joined the machine to the domain, (and had to setup
KOC again, because it's a new profile on the computer.) and
BAM! PROBLEM setting up KOC!
Let's pause for a moment, shall we?
This HAS to be a domain policy. Rick said so himself here and I said the same thing.
One small wrinkle, though: the old XP machines are in the domain and they worked fine. Moreover, everyone at the new company swore up and down that there were no policies applied - the domain really was only to help protect a few shared resources from unauthorized users
So, we bumped it up to Kerio Support. They looked at logs and didn't have much to add:
Thank you for the log information.
It appears as if there is network connectivity issues as soon as
you join the domain. the debug log shows it unable to locate items
(404 not found errros) and RPC server unavailable. Something is
blocking these machines as soon as they join the domain.
Can you check the windows event viewer for any sign here of
issues? other warning messages.
What Antivirus is used on these machines. Are these the only win7
platforms on the domain that are having this issue? are there win7's
that are working?
I have to be honest, that I have not seen this type of issue with
domain based win7 boxes, since we use them ourselves and many other
clients/customers are running win7 without this type of issue. It
has to be something about this domain that is causing the issue.
So, really, it HAS to be the domain, right?
Microsoft was summoned. At first, they thought it was Outlook. Then of course, they suspected Kerio, but had to agree that Kerio worked correctly before the machines joined the domain. Rick also showed them that Zimbra IMAP had the same problems Kerio exhibited.
Meanwhile, Kerio second tier support confirmed what we thought we were seeing:
The development team writes that requests from KOFF to Connect are
timeouting when downloading partially synchronized message.
I wondered about the Internet link at the new company. It's only a T1; were we overloading it? But that made no sense because, again, the trouble only showed up when the machines joined the domain and that sent me back to domain policy and there was none.
Meanwhile, Microsoft had decided that this was indeed a domain related network issue and bumped the case upward in their support chain. A week later, they fixed it.
Presented as it is, without understanding
As we all said, this had to be a domain policy, and Microsoft pointed at "Remote Installation Services". There are four options in that policy:
- Automatic Setup
- Custom Setup
- Restart Setup
Each option can be set to Enabled, Disabled and Not Configured. In the murky past, someone had set Custom, Restart, and Tools to "Disabled". Microsoft had Rick set them back to "Not Configured".
I have to bring this train to a screeching halt for just a minute. This Remote Installation Services is for installing OS images on PXE capable systems. Therefore, unless you actually are installing operating systems, it would seem reasonable to assume that the most this service would be doing is listening for BOOTP requests - and it would only do that if it were enabled. Moreover, given Microsoft's explanation of "Not Configured" (see image below), that is no different from "Disabled" in this case.
However, Microsoft code is full of strange things. While being prevented from listening for BOOTP packets, perhaps the PDC gets bored and decides to echo random thoughts back out on the wire, interfering with our synchronization?
Perhaps Windows 7, feeling humiliated by the looming shadow of Win 8, desires an OS transplant, and, noticing that this option is specifically disabled, cries out in pain, throws itself on the floor and trashes around in tantrum? Whatever: if Microsoft tells you to change a setting, you change the setting - even if they admit they have no idea why it matters.
But wait - there's more!
Microsoft also had Rick do this:
We had to issue the following commands at the command prompt (AS ADMINISTRATOR!)
[on the Win 7 machines]
netsh interface tcp set global rss=disabled
netsh interface tcp set global autotuninglevel=disabled
netsh int ip set global taskoffload=disabled
netsh interface tcp set global chimney=disabled
When complete, the command :
netsh int tcp show global
should show everything disabled except for NetDMA State
That makes more sense. This has to do with smarty-pants NIC cards that can take on TCP/IP processing that the OS would normally handle. If you google for "Win 7 SNP", you'll find articles that recommend similar actions for similar problems. Interestingly, this stuff goes back to Server 2003 - I found this "update to turn off default SNP features", which leads off with an explanation of all the bad networking trouble this can cause:
This makes sense - apparently Microsoft coders and the NIC card coders aren't communicating well and the result can be a mess like this. I can't imagine how this has anything to do with Remote Installation Services, but so what?
But why does it turn up only after joining the domain? Does the domain change the Win 7 TCP/IP stack? Darned if I know.. I did some Googling but couldn't zero in on anything helpful. Apparently something happens - because Rick reports success:
And this my friends provides us with a Windows 7 computer on a Server
2003 Domain that operates quickly, smoothly and most enjoyably.
Rick explained in a phone conversation that they tried doing just the SNP, but without changing the domain policy, it did not help. We all agree (even Microsoft, apparently) that it makes no sense - when KOC is talking to the mail server, the DC isn't even involved, but there it is just the same: both these actions were needed to fix the problem.
So, two weeks after the initial problem, and with much wailing and gnashing of teeth, the problem is solved. Happy workers are now reading email on their brand spanking new Windows 7 machines. I'm sure Rick is sleeping better and those of us who work in the field should remember this SNP stuff for future reference.
If this had been happening BEFORE joining the domain, the first thing I would have tried is adding a different NIC. That's a simple and quick test for driver weirdness, but because the silly things worked outside of the domain, that thought never entered my head. It's good to keep in mind that for this kind of troubleshooting, you actually want a no frills, basic, simple NIC that Windows has known how to handle for years - no smarty pants hardware need apply. Had I been wise enough to do that, the finger would likely have pointed at the NICS instantly and we either would have got to this point faster or would have just bought a box of new NICS and gone around the problem. But I did not.
But that's life, isn't it? We get led astray at times and it can be hard to resist that. Had I suggested trying different NICs, I probably would have met resistance because it worked until they joined the domain. So, maybe it doesn't matter, but I wish I had tried just the same.
Got something to add? Send me email.
Increase ad revenue 50-250% with Ezoic
More Articles by Anthony Lawrence
Find me on Google+
© 2015-02-07 Anthony Lawrence