IIS SMTP Server SmtpOutboundCommandSupportOptions Bug

The Simple Mail Transport Protocol (or SMTP for short) Server has been a part of Internet Information Services (IIS) since the days of Windows NT Server.  Over the years, little about it has changed, nor is any big revamp expected by Microsoft.  This is underscored by the fact that the SMTP Server was not made a part of IIS 7, instead remaining available in Windows Server 2008 has an IIS 6 legacy feature.  And for the most part, this is fine.  Although totally useless as an inbound mail server (except for Windows Server 2003, when Microsoft added a matching POP3 server), it excels in one area that keeps it firmly a part of the Windows Server feature set, even though IIS has moved on to be fully integrated with the Microsoft .NET framework.  It's a capable relay for outbound email create by web applications.

The SMTP Server is a smart mail server, meaning that it can deliver messages directly to the recipient's mail server.  Which allows web applications to send out email directly from the server without having to relay through an intermediate mail system.  (Of course, there is a trick to this, and I'll cover the proper configuration the IIS SMTP Server in a future post.)  But while the SMTP Server hasn't changed much over the years, how mail servers communicate with other did when Enhanced SMTP (ESMTP) was introduced to extend the SMTP command set.  It has eclipsed the older SMTP command set and ESMTP is now used by the vast majority of all mail servers.  In fact, the large public email services like AOL, Gmail, Hotmail (or Outlook.com), and Yahoo won't even talk to a mail server that doesn't speak ESMTP.

And here lies a big problem for most IIS SMTP Servers.  Although the SMTP Server supports ESMTP, it is disabled by default!  Thus when the SMTP Server tries to deliver email to recipients at one of the large public email services, the connection is simply closed without any failure notice.  The SMTP Server keeps trying to deliver the email, each time being rebuffed by the large public email services, until it finally gives up and moves the message to the Badmail folder.

When I encountered this a couple of years back, I was trying to figure out why email relayed through the SMTP Server wasn't getting to Gmail and Yahoo recipients.  Mail was being delivered to other mail systems just fine, just not the big players, which represented about half of the intended recipients.  I finally stumbled upon an obscure TechNet article that partially (partially!) documented the IIS metabase setting called SmtpOutboundCommandSupportOptions.  This a bit-mapped value, and by setting or clearing various bits, you can turn off or on support for various ESMTP commands when making outbound connections.  (There is a similar setting called SmtpInboundCommandSupportOptions which does the same thing for inbound connections.)

According to the documentation, the value of the SmtpOutboundCommandSupportOptions setting defaults to 7697601, which turns on support for all the ESMTP commands for outbound connections.  But the default value as listed in the official Microsoft documentation is totally incorrect.  The real default value is 7!  Which means that ESMTP support for outgoing connections is disabled, so trying to communicate with the large public email services is a completely futile exercise.  By when I changed the setting to officially document default value of 7697601, the SMTP Server was now able to deliver email to the large public email services without problems, and the outbound mail queue was suddenly empty.

So how do you change the value of the SmtpOutboundCommandSupportOptions setting?  It's actually very simple.  It can done using the ADSUTIL.VBS script or by editing the metabase directly.

Using The ADSUTIL.VBS Script

ADSUTIL.VBS is a VBscript utility created by Microsoft for working with IIS metabase.  It must be run from a Command-Line Prompt windows using CSCRIPT.  It is normally find in the folder C:\inetpub\AdminScripts\, and on Windows Server 2008, requires that the "IIS 6 Scripting Tools" feature be installed.  Some hosting companies move this to a different folder in their Windows Server builds, which was the case for Windows Server 2003 at Peer 1, where it's locating the D:\AdminScripts folder (we stopped moving it in Windows Server 2008).  Once you've located the ADSUTIL.VBS script, open a Command Prompt windows, change to the directory where the script is located, and run the following command:

cscript adsutil.vbs SET SmtpSvc/SmtpOutboundCommandSupportOptions 7697601

which will return the following message:

SmtpOutboundCommandSupportOptions: (Long) 7697601

In order for the change to take effect, you need to restart the "Simple Mail Transport Protocol" service.

Editing The IIS Metabase

Should the ADSUTIL.VBS script not be available, you can edit the IIS metabase directly using Notepad.  NOTE: You will need to stop IIS in order to save the changes made in Notepad.

  1. Stop the IIS services using the command:

  2. Open the file C:\WINDOWS\system32\inetsrv\MetaBase.xml in Notepad.
  3. Under the section IIsSmtpService Location = "/LM/SmtpSvc", find the property SmtpOutboundCommandSupportOptions. Since this property appears only once in the metabase, the quickest way to locate it is Notepad's Find utility (Edit –> Find).
  4. Change the property so it reads:

  5. Save the file.
  6. Restart the IIS services using the command:


And that's all there is to it.

Posted in Windows.
All information in this blog is provided "AS IS" with no warranties and confers no rights.
The opinions expressed in this blog are mine alone and do not represent those of my employer.
Powered By PEER 1 Managed Hosting