I spent most of the afternoon tracking down a problem with my web server. Performance came to a standstill. So I went into Task manager and found the MailService.exe process was maxing out resources. I rebooted, and found that it would gradually start taking more and more resources until the system became virtually unusable. Within 20 minutes after a reboot of the system, it would be hogging most of the memory, and max out the CPU to 100%. I also looked through the event Application logs. Searched for ‘mail’. Found errors “System.OutOfMemoryException: Exception of type ‘System.OutOfMemoryException’ was thrown.”.

I then Googled on ‘mailservice.exe memory System.OutOfMemoryException’ and found this post on SmarterTools Forums – MailService.exe memory reporting issue in 6.5. Many users seem to be experiencing this problem. A second post from the SmarterTools Forums – HELP! SmarterMail.exe keeps failing every few hours also indicates that this is a Microsoft .Net issue. They refer to the prior post stating the fix is to re-install Microsoft .Net. I’ve already attempted this procedure before. See my prior post 2010-04-23 Notes for the day.

The posts above refer to using the ASP.NET IIS Registration Tool (aspnet_regiis.exe) to fix the memory issue. Another article from SmarterTools – Troubleshooting – ASP.NET and IIS Issues FAQ also discusses using this tool.

So it all seemed logical to me that SmarterMail had developed a bug of some sort that needed fixing. At the bottom of this post, I’ve included all the steps I undertook in an attempt to cure the servers ills. Feel free to read it if you’re in need of some serious sleep! The bottom line was that I had never properly configured my SmarterMail to prevent spammers from moving in. By the time I figured this out, I had well over 48,000 messages in the outgoing spool! I feel like a chump indeed. How long has this been going on I wonder? No wonder the sever was crawling on its knees! Poor thing. Anyway, with the help of the SmarterMail Help online web site (look under Additional Topics | Locking Down Your Server), here are the steps I undertook to lock down the server:

Steps to immediately stop sending and receiving spam

If you’re in my situation, you’ll want to stop the spamming as soon as possible. Here’s the quick step to get rid of them:

1. Log in to SmarterMail as admin
2. Navigate to Manage | Services and shut down the SMTP and Spool services.
3. Navigate to Settings | General Settings | Spool (remember the spool path)
4. Open Control Panel | Administrative Tools | Services
5. Find the SmarterMail Service and Stop the Service
6. Open Windows Explorer and navigate to spool path
7. Rename Spool to Spool.Old and create a new folder Spool
8. Restart the SmarterMail Service
9. Log in to SmarterMail as admin
10. Navigate to Manage | Spool | All Messages (none should be present)
11. Navigate to Manage | Services and start the SMTP and Spool services.

Steps taken to lock down the SmarterMail server

1. Log in to SmarterMail as admin
2. Navigate to Settings | Protocol Settings | SMTP In and change Allow Relay to Nobody.
3. Navigate to Manage | All Domains | Select larrybrouwer.com | Edit | Features and de-select ‘Enable Catch-Alls’.
4. Navigate to Manage | All Domains | Select larrybrouwer.com | Edit | Technical and choose ‘Do not auto-respond to Spam Level Low and above’ for Auto-Responder Exclusions, choose ‘Do not forward Spam Level Low and above’, check Require SMTP Authentication, check Enable once per day per sender autoresponder restriction, click Save.
5. Navigate to Manage | All Domains | Select larrybrouwer.com | Edit | Limits and set Disk Space to 10, Users 5, User Aliases 10, Mailing Lists 10, Pop Retrieval Accounts 5, Max Message Size 1000, Recipients per Message 10, click Save.
6. Navigate to Manage | All Domains | Select larrybrouwer.com | Edit | Throttling and set Outgoing Messages per Hour 5 and click Enabled, Outgoing Bandwidth per Hour 1 and click Enabled, Bounced Recieved per Hour 1 and click Enabled, click Save.

Some of the throttling settings may be ridiculously low for you. But for me, they are perfectly fine since I’m the only one using this email, and I’m rarely sending or receiving from this account. Adjust them accordingly. That’s it!

Now as promised, here’s the tedious notes of my afternoon wrestling with this problem running down the wrong trails!

So, I’ve decided to follow the SmarterTools Troubleshooting article (see top of post).

Open a command prompt as Administrator (right click on cmd and choose run as Administrator).

From command prompt, run: echo %SYSTEMROOT% and open Windows Explorer and navigate to %SYSTEMROOT%\Microsoft.NET\Framework.

This Microsoft Article – How to determine .NET Framework versions installed goes into great detail about what to look for and how to check whether your .NET Frameworks are up to date or not.

Following this article, navigate to %SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\ right click on Mscorlib.dll and select properties. Then open the Details tab. The product version shows what Service Pack if any has been applied. In my case, it’s version 2.0.50727.4200, and SP2 would be 2.0.50727.3053. So, it would appear that I need up apply .NET 2.0 SP2. I ran Windows Update, and it is up to date. So I manually downloaded Microsoft .NET Framework 2.0 Service Pack 2 and attempted to install it. It popped up a message box stating “This product is not supported on Vista Operating System.”. I spent considerable amount of time Googling in an attempt to find out if there is a version for Server 2008. I finally read the Overview section in SP2 download page, and it states that the updates for Vista and Windows 2008 are deployed via the full .NET Framework 3.5 SP1.

I then proceeded to re-apply the .NET Framework 3.5 SP1, which I had previously downloaded on 4/23/2010. Upon running dotnetfx35.exe, I chose the repair option. I then continued with the installation instructions by applying Microsoft Update KB959209 – An update for the .NET Framework 3.5 SP1. The Microsoft .NET Framework 3.5 Family Update for Server 2008 x86 was then installed. There are 3 updates to install, they must be installed in the proper sequence as described (Be sure to read the instructions carefully, the list below is not in the correct installation order). I did not run this update the last time. The first two (2) updates did not apply to this server. The third update installed successfully. I then rebooted the server.

After the server rebooted, I looked at the MailService.exe in the Task Manager, and it is still taking +50% CPU and memory usage is expanding rapidly. I then opened up Windows Update and looked for new updates. It found nothing. I then went back to the SmarterTools – Troubleshooting – ASP.NET and IIS Issues FAQ and continued. I opened a cmd window with Administrator privileges. I then executed:

%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis –i

I then rebooted the server. The same thing is still happening.

Back to Googling and found SmarterTools Forums – mailservice.exe 100% cpu usage. Looks like the problems are rampant with this product. I decided to upgrade to the latest version. Downloaded and installed SmarterMail v6.8.3750. I then rebooted the server. Again the same thing is still happening.

I then followed Configure IIS7 with Windows 2008. First, click on Start | Program Files | Smarter Tools | SmarterMail | Configure SmarterMail Web Server. Click Stop. Change Startup Mode to Disabled and Apply then click Close. Next open IIS7 Console Manager. Right click on Application Pools and choose “Add Application Pool”. Name the pool SmarterMail. Set Managed Pipeline Mode to Classic and click Ok. Next Right click on Sites and choose Add Web Site … Name it SmarterMail. Application Pool is same name. Set the Physical path to: C:\Program Files\SmarterTools\SmarterMail\MRS. Set the Port to 9998. Host name: www.larrybrouwer.com. This was successful, however, it does not fix my problem with the MailService.exe.