While installing some demo software today, I had to create a self-signed SSL certificate for IIS7. This is a relatively easy thing to do within the IIS7 Manager. However, the certificate that IIS7 generates is only for the machine that it is running on. I need a certificate that has my custom web site domain name on it. As it turns out, Microsoft has eliminated this capability within IIS7, so the easiest way to accomplish this is to use the selfssl tool from IIS6.

It took me a while to figure out, but after some Googling (with much trial and error), here’s what I came up with.

I found a post, Self-Signed Certificates on IIS7, that I used to create and install the certificate.

First, I opened IIS7 Manager, and found the Web Site ID for my web application.

Second, I downloaded, IIS 6 Resources Kit Tools, and installed just the selfssl tool.

Third, I opened an Administrator command prompt and navigated to the selfssl location: C:\Program Files (x86)\IIS Resources\SelfSSL

Fourth, I executed the following command:

selfssl /N:CN=www.larrybrouwer.com /V:10 95 /S:6 /P:8443

That completes the configuration on the Server.

Now, on the client, open Internet Explorer with Administrator privileges.

Second, navigate to the secured site. Bypass the security warnings.

Third, view the certificate and Install it to the “Trusted Root Certification Authorities”, “Local Computer”.

Done!

So far, I’ve installed three (3) open source shopping carts; PrestaShop, OpenCart, and TomatoCart. All of these packages have been developed with PHP and MySQL technologies. This morning I’ve decided to see if there are any open source ecommerce packages written in a Microsoft .NET environment.

My first search lead me to the Microsoft Web Gallery. From there, I found nopCommerce. It looks like a reasonable package, so I’ve decided to install it and see what it looks like.

I wanted to first look at the documentation, however, it’s not free! I decided to download the product and install it anyway. I have two (2) choices for installing. The first way is to use the Microsoft Web Platform Installer. The second way is to manually download and install it. I’m choosing the latter, as I want to be sure to get the complete package with source code.

I then navigated to the nopCommerce download page, downloaded the latest release (v1.90 at the time of this writing) and extracted the Visual Studio solution to my development workstation. I opened the solution with Visual Studio, and realized that I didn’t “unblock” the downloaded zip file prior to extracting the solution to my hard drive. So I deleted the solution files, used Microsoft file explorer to change the properties to “Unblock” the zip file prior to extracting the files. I then re-extracted the files to my development environment and re-opened project with Visual Studio 2010. Much better now! No security warnings.

Next, I rebuilt the application and after verifying that it executed a clean build, I took the contents of the “NopCommerceStore” folder and copied to the “www\nopCommerce” folder on my development Web Server. I then added the new application using IIS7 Manager, and binded it to port 8004. I then opened the server’s firewall port.

I browsed to the nopCommerce Web Site, and immediately got the following error:

Server Error in ‘/’ Application.

---

Configuration Error

Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately.

Line 26: 			<error statusCode="404" redirect="filenotfound.htm"/>
Line 27: 		</customErrors>
Line 28: 		<compilation debug="true" targetFramework="4.0">
Line 29: 			<expressionBuilders>
Line 30: 				<add expressionPrefix="NopResources" type="NopSolutions.NopCommerce.BusinessLogic.Localization.NopResourceExpressionBuilder, Nop.BusinessLogic"/>

My initial thoughts were that the Application Pool wasn’t set to the proper .NET Framework, so I used IIS7 Manager and changed it from .NET Framework v2.0.50727 to v4.0.30319. I then restarted the Application Pool, and the application is now loading fine.

I followed the installation instructions, and created a new Microsoft SQL Server schema and user. The only problem I encountered here was that I forgot to include the database instance name in the SQL Server name field. The format is “SERVER\INSTANCENAME”. Other than that, the installation process was a breeze.

After the installation was complete, I was able to browse to the home page and log into the application. From there, I was able to get into the administration section and begin configuring it.

I next renamed the existing “robots.txt” to “robots.txt.orig” and replaced it with the following to prevent spidering of my demo site:

# go away
User-agent: *
Disallow: /

I’ll keep updating this post as I continue exploring nopCommerce.

You’re welcome to visit my nopCommerce demo site here. It will be up and running while I’m developing at my desk during the day. If you would like help with your installation, just get in contact with me. I’m always happy to help out!

Updates:

One of the first modification that I did was to download the Five colour theme pack for front-end themes. To install the themes, simply extract the files from the download and drop them into the “APP_Themes” folder.

I’m continuing my evaluation of open source ecommerce packages by installing TomatoCart. TomatoCart was chosen as the second runner-up of the 2010 Open Source Awards as published by Packt Publishing.

I began my TomatoCart installation by reviewing the tomato Cart documentation WiKi. The wiki has many uncompleted links, but I managed to find the TomatoCart User Guide, which covered the basics needed to get the software installed.

I downloaded the current version, TomatoCart v1.1.3, and saved it to my “install\tomatocart” folder. I then created a new “tomatocart” folder under my “www” root folder and copied the contents of the “gtrunk2” folder from the downloaded file there.

I then went into Windows IIS7 Manager and created a new Web Site at port 8003. I also added an additional binding for my local IP address of 192.10.11.21. Lastly, I opened my firewall port at 8003.

I then browsed to the site’s install URL, 192.10.11.21:8003/install, and followed the installation procedure. I soon realized that I needed to create a new MySQL database schema and user, so I logged into MySQL Workbench, and created both. I make sure that the user only has privileges to it’s own schema, and that it can only log in locally to the server. I gave the user total rights to it’s “tomatocart” schema.

I continued with the installation, entering all the database information requested. I chose the MySQL – InnoDB (Transaction-Safe) database type. Upon hitting the “Continue” button, the import ran for a few seconds and then bombed with the following error:

There was a problem importing the database. The following error had occured:

1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘type = innodb’ at line 1; alter table toc_weight_classes_rules type = innodb

Please verify the connection parameters and try again.

I then logged into MySQL Workbench and entered the “alter table” SQL statement. It has a syntax error. Upon looking up the correct syntax, it should be “alter table toc_weight_classes_rules engine = innodb;”. I executed the code, and it works fine.

I next tracked down the offending code that produced the bug by searching for it in the installation folder using the grepwin explorer tool. I located the problem code in the “install\rpc.php” file. I changed the code from “type = innodb” to “engine = innodb”, saved the file, then dropped the “tomatocart” schema and re-created it. I then re-ran the installation, and the tables loaded fine.

I continued with the installation and completed it with no additional problems. I next renamed the “install” directory to prevent it from being run again.

I next renamed the existing “robots.txt” to “robots.txt.orig” and replaced it with the following to prevent spidering of my demo site:

# go away
User-agent: *
Disallow: /

I’ve completed the installation, and now I need to begin configuring it. So, apart from the minor bug, it was a pretty easy installation.

I’ll keep updating this post as I continue working with TomatoCart.

As always, you’re welcome to view my TomatoCart demo site here. It will be available while I’m working during the day. If you should need any help with your installation, send me an email. I’d be happy to help you out! Have fun!

Update on 3/2/2011

While attempting to add a new product with a “variant”, I got the following error message:

“Error: There was a problem performing the action.”

Upon further research, I discovered that “STRICT_TRANS_TABLES” needed to be removed from the sql-mode parameter in the my.ini startup file.

Here’s a list of free stuff I’ve installed on my development workstation. I will keep updating this post as I go along.

FileZilla

CC Cleaner

Defraggler

Notepad++

Jing

Screenhunter

Navicat MySQL Lite

SQLyog

WinMerge

Fiddler

This morning, I’m working on installing open source ecommerce packages on my Windows Server 2008 box. This is mainly for experimental and learning purposes.

The first package I choose to install is PrestaShop. PrestaShop was chosen as the winner of the 2010 Open Source Awards as published by Packt Publishing.

My first step was to read through the most current PrestaShop User Guide. The most current stable version as of this writing is PrestaShop v.1.3.5. PrestaShop uses a MySql Database, so my first task is to install the current release of MySql (5.5.8) on my Windows 2008 server.

Since I’m not currently using MySql on a daily basis, I also made sure all of my workstation components (i.e. workbench, connector, etc.) are also up to date.

I ran into a glitch installing MySql. The Configuration Wizard has an identified bug in it as reported to the MySql Bug database as, Bug #59038 mysql.user.authentication_string column causes configuration wizard to fail. To overcome this bug, I executed the following command from the MySql Workbench:

use mysql;
alter table
    user change authentication_string authentication_string text null;

I then clicked on the re-try for the Wizard and it completed normally. After this, I ran the following commands:

use mysql;
update user set authentication_string = '' where authentication_string is null;
alter table user change authentication_string authentication_string text not null;

<?phpecho md5("xae0u0c9kzEGgPFJWgGtgJa8PG1STZubGmeI57482nPwmzTkEowdbBth"."admin");?> 

"C:\Program Files (x86)\PHP\php.exe" -f .\password.php

# go awayUser-agent: *Disallow: /

I spent most of the afternoon tracking down a problem with my web server. Performance came to a standstill. So I went into Task manager and found the MailService.exe process was maxing out resources. I rebooted, and found that it would gradually start taking more and more resources until the system became virtually unusable. Within 20 minutes after a reboot of the system, it would be hogging most of the memory, and max out the CPU to 100%. I also looked through the event Application logs. Searched for ‘mail’. Found errors “System.OutOfMemoryException: Exception of type ‘System.OutOfMemoryException’ was thrown.”.

I then Googled on ‘mailservice.exe memory System.OutOfMemoryException’ and found this post on SmarterTools Forums – MailService.exe memory reporting issue in 6.5. Many users seem to be experiencing this problem. A second post from the SmarterTools Forums – HELP! SmarterMail.exe keeps failing every few hours also indicates that this is a Microsoft .Net issue. They refer to the prior post stating the fix is to re-install Microsoft .Net. I’ve already attempted this procedure before. See my prior post 2010-04-23 Notes for the day.

The posts above refer to using the ASP.NET IIS Registration Tool (aspnet_regiis.exe) to fix the memory issue. Another article from SmarterTools – Troubleshooting – ASP.NET and IIS Issues FAQ also discusses using this tool.

So it all seemed logical to me that SmarterMail had developed a bug of some sort that needed fixing. At the bottom of this post, I’ve included all the steps I undertook in an attempt to cure the servers ills. Feel free to read it if you’re in need of some serious sleep! The bottom line was that I had never properly configured my SmarterMail to prevent spammers from moving in. By the time I figured this out, I had well over 48,000 messages in the outgoing spool! I feel like a chump indeed. How long has this been going on I wonder? No wonder the sever was crawling on its knees! Poor thing. Anyway, with the help of the SmarterMail Help online web site (look under Additional Topics | Locking Down Your Server), here are the steps I undertook to lock down the server:

Steps to immediately stop sending and receiving spam

If you’re in my situation, you’ll want to stop the spamming as soon as possible. Here’s the quick step to get rid of them:

1. Log in to SmarterMail as admin
2. Navigate to Manage | Services and shut down the SMTP and Spool services.
3. Navigate to Settings | General Settings | Spool (remember the spool path)
4. Open Control Panel | Administrative Tools | Services
5. Find the SmarterMail Service and Stop the Service
6. Open Windows Explorer and navigate to spool path
7. Rename Spool to Spool.Old and create a new folder Spool
8. Restart the SmarterMail Service
9. Log in to SmarterMail as admin
10. Navigate to Manage | Spool | All Messages (none should be present)
11. Navigate to Manage | Services and start the SMTP and Spool services.

Steps taken to lock down the SmarterMail server

1. Log in to SmarterMail as admin
2. Navigate to Settings | Protocol Settings | SMTP In and change Allow Relay to Nobody.
3. Navigate to Manage | All Domains | Select larrybrouwer.com | Edit | Features and de-select ‘Enable Catch-Alls’.
4. Navigate to Manage | All Domains | Select larrybrouwer.com | Edit | Technical and choose ‘Do not auto-respond to Spam Level Low and above’ for Auto-Responder Exclusions, choose ‘Do not forward Spam Level Low and above’, check Require SMTP Authentication, check Enable once per day per sender autoresponder restriction, click Save.
5. Navigate to Manage | All Domains | Select larrybrouwer.com | Edit | Limits and set Disk Space to 10, Users 5, User Aliases 10, Mailing Lists 10, Pop Retrieval Accounts 5, Max Message Size 1000, Recipients per Message 10, click Save.
6. Navigate to Manage | All Domains | Select larrybrouwer.com | Edit | Throttling and set Outgoing Messages per Hour 5 and click Enabled, Outgoing Bandwidth per Hour 1 and click Enabled, Bounced Recieved per Hour 1 and click Enabled, click Save.

Some of the throttling settings may be ridiculously low for you. But for me, they are perfectly fine since I’m the only one using this email, and I’m rarely sending or receiving from this account. Adjust them accordingly. That’s it!

Now as promised, here’s the tedious notes of my afternoon wrestling with this problem running down the wrong trails!

So, I’ve decided to follow the SmarterTools Troubleshooting article (see top of post).

Open a command prompt as Administrator (right click on cmd and choose run as Administrator).

From command prompt, run: echo %SYSTEMROOT% and open Windows Explorer and navigate to %SYSTEMROOT%\Microsoft.NET\Framework.

This Microsoft Article – How to determine .NET Framework versions installed goes into great detail about what to look for and how to check whether your .NET Frameworks are up to date or not.

Following this article, navigate to %SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\ right click on Mscorlib.dll and select properties. Then open the Details tab. The product version shows what Service Pack if any has been applied. In my case, it’s version 2.0.50727.4200, and SP2 would be 2.0.50727.3053. So, it would appear that I need up apply .NET 2.0 SP2. I ran Windows Update, and it is up to date. So I manually downloaded Microsoft .NET Framework 2.0 Service Pack 2 and attempted to install it. It popped up a message box stating “This product is not supported on Vista Operating System.”. I spent considerable amount of time Googling in an attempt to find out if there is a version for Server 2008. I finally read the Overview section in SP2 download page, and it states that the updates for Vista and Windows 2008 are deployed via the full .NET Framework 3.5 SP1.

I then proceeded to re-apply the .NET Framework 3.5 SP1, which I had previously downloaded on 4/23/2010. Upon running dotnetfx35.exe, I chose the repair option. I then continued with the installation instructions by applying Microsoft Update KB959209 – An update for the .NET Framework 3.5 SP1. The Microsoft .NET Framework 3.5 Family Update for Server 2008 x86 was then installed. There are 3 updates to install, they must be installed in the proper sequence as described (Be sure to read the instructions carefully, the list below is not in the correct installation order). I did not run this update the last time. The first two (2) updates did not apply to this server. The third update installed successfully. I then rebooted the server.

After the server rebooted, I looked at the MailService.exe in the Task Manager, and it is still taking +50% CPU and memory usage is expanding rapidly. I then opened up Windows Update and looked for new updates. It found nothing. I then went back to the SmarterTools – Troubleshooting – ASP.NET and IIS Issues FAQ and continued. I opened a cmd window with Administrator privileges. I then executed:

%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis –i

I then rebooted the server. The same thing is still happening.

Back to Googling and found SmarterTools Forums – mailservice.exe 100% cpu usage. Looks like the problems are rampant with this product. I decided to upgrade to the latest version. Downloaded and installed SmarterMail v6.8.3750. I then rebooted the server. Again the same thing is still happening.

I then followed Configure IIS7 with Windows 2008. First, click on Start | Program Files | Smarter Tools | SmarterMail | Configure SmarterMail Web Server. Click Stop. Change Startup Mode to Disabled and Apply then click Close. Next open IIS7 Console Manager. Right click on Application Pools and choose “Add Application Pool”. Name the pool SmarterMail. Set Managed Pipeline Mode to Classic and click Ok. Next Right click on Sites and choose Add Web Site … Name it SmarterMail. Application Pool is same name. Set the Physical path to: C:\Program Files\SmarterTools\SmarterMail\MRS. Set the Port to 9998. Host name: www.larrybrouwer.com. This was successful, however, it does not fix my problem with the MailService.exe.

This morning, my Outlook inbox has messages with no ‘From’, no ‘Subject’, and nothing in the ‘body’. So, I’m thinking that the .pst file has been corrupted at some point. I Googled on ‘Outlook 2007 repair tool’ and came up with this Microsoft Support article "How to use the Inbox Repair Tools to recover e-mail messages in Outlook 2002, Outlook 2003, and Outlook 2007". I followed the instructions to repair my outlook .pst file as follows:

Open windows explorer and navigate to the folder where my outlook files are stored. Normally, this would be in: C:\users\username\AppData\Local\Microsoft\Outlook, however, I’ve moved them to: My Documents\Exchange.

Check the file sizes. In my case, I have a whopper of a file! It’s 3.9GB. I’m thinking this could be part of my problems. The Microsoft article discusses file size max is 2GB for prior versions of Outlook. I need to find out what the file maximum is for Outlook 2007. I ran a Google search on ‘Outlook 2007 maximum pst file size and found an article from About.com Do Outlook PST Files Have a Size Limit?. This article says that beginning with Outlook 2003, pst file format capable of storing Unicode data, so it has no theoretical size limit. There is a practical limit of 20GB, however.

The file limits are now configurable, as I found out by reading Microsoft Support article "How to configure the size limit for both (.pst) and (.ost) files in Outlook 2007 and in Outlook 2003. I now followed the instructions in this article to determine my current file size settings. They are configured in the registry. There are four (4) registry entries: MaxFileSize, WarnFileSize, MaxLargeFileSize, WarnLargeFileSize. I opened regedit and tried to navigated to: HKEY_CURRENT_USER\Software\Policies\Microsoft\Office\12.0\Outlook\PST, however, I could not navigate beyond Microsoft. Nothing was found. I then Googled on ‘HKEY_CURRENT_USER\Software\Policies\Microsoft\Office\12.0\Outlook\PST values not found’ and found a post discussing this here. Based on this post, it looks like there is no need to mess with the registry as the default settings are at 20GB for the Unicode file format. I did not research any further as to why I don’t have the registry keys.

I’ve been using Outlook a very long time, and I’m wondering what my current file format is. Perhaps the format is still ANSI with a 2GB limit? So I Googled on ‘determining pst file format ansi unicode’ and found a MSOutlook.info article "Am I in ANSI or UNICODE format?". Following the instructions, I opened Outlook and navigated to File | Data File Management. I then double clicked on the pst file in question and looked at the Format field. It reads ‘Personal Folders File’. According to this article, that means my pst file is in UNICODE format. If it would have read ‘Personal FOlders File (97 – 2002)’, then it would have been in ANSI format. Looks like my file size of 3.9GB is fine.

Continuing on with the Outlook PST repair procedure.

I exit Outlook. Open Windows Explorer. I need to make sure I can see hidden files. I’m running Windows 7, so in Windows Explorer, click Organize | Folder and seach options. Open View tab at the top. Make sure ‘Show hidden files, folders, and drives’ is selected. Click Apply. Click OK.

Locate the ‘Scanpst.exe’ program by either navigating to it in Windows Explorer to: C:\Program Files (x86)\Microsoft Office\Office12 or by clicking Start and entering ‘Scanpst.exe’ in the Search programs and files box. Run the program. Browse to the pst file and click Open. Click Options. Check Append to log if you don’t want the log file replaced. Click Ok. Click Start.

After completion, The Microsoft Office Outlook 2007 Inbox Repair Tool found errors in my .pst file. It reads “Errors were found in this file. To repair these errors, click Repair. Folders found in this file: 767 Items found in this file: 40699”. I clicked on Details and the message reads: “Internal errors were found in this file. They must be repaired for this file to work correctly.”. I clicked Ok. I checked ‘Make a backup of scanned file before repairing’. I then clicked Repair.

The repair procedure took a considerable amount of time. Well over 20 minutes. Upon completion, a message box appears “Repair complete”. I clicked Ok. I then went back into Outlook. The blank emails are still present in my InBox. I do not see any evidence of any recovered emails.

The Microsoft support article proceeds to step 2, a discussion about how to recover the repaired items. Following along with the article, I open the Personal Folders of the one I repaired. The article says I should have recovered folders: Calendar, Contacts, Deleted Items, Inbox, Journal, Notes, Outbox, Sent Items, Tasks. I see none of these. However, at this point, I realize that the blank emails are actually coming in from my Office Live account from the Outlook Connector which is a different profile altogether. So I went back into my Data Files to see which file is storing these emails, and it says Filename “Not Available”.

The resolution of my blank emails

All the above information is nice to know, but it still hasn’t solved my original problems. A solution is about to finally appear. So I perform a Google search on ‘officelive outlook 2007 blank emails’ and find a post about this topic, Outlook getting blank emails. No solutions are found here.

However, the solution finally is found in another post Outlook Connector downloading blank emails that states I need to disable the email integration with my anti-virus software.

I’m currently using Norton provided from Comcast. I opened up Norton Security Suite. Navigated to Settings | Antispam and clicked off the Antispam feature. Clicked Apply then Back. Then clicked on AntiVirus | Scans and Risks. I then clicked off the Email Scan feature. Clicked Apply then selected ‘Permanently’ to stop it from scanning emails altogether. Clicked Apply then Back then Close then exited the program.

I then restarted Outlook. As expected, the blank emails are still there after doing a Send/Receive. I then clicked on Outlook Connector | Mail Settings. It took me to the OfficeLive Website. I logged into my account at Microsoft OfficeLive.

Re-Downloading all the blank emails from OfficeLive

**After trying to find some settings that might help, I went into my OfficeLive InBox. I then checked all my emails, and selected Mark as Unread. I then went back into Outlook and clicked Send/Receive, and all my emails re-downloaded! Cool! Now all my blank emails are gone. Unfortunately, I deleted many blank emails from my inbox prior to all this, and now they did not re-download, and even worse, I have no idea what they are/were. Since the majority of them are not important, I’m not pursuing this any further.

In this post, I just want to keep track of what IIS7 extensions I’ve installed on my development web server, and some brief notes about them. The extensions are located at http://www.iis.net/.

1. Search Engine Optimization Toolkit.

2. Web Platform Installer. This allows for installation of Web Platform Extensions, New Web Applications (i.e. BlogEngine.Net), and Beta Extensions from within the IIS manager. You should really check out the samples. They are also located at: http://www.microsoft.com/web/gallery

3. Advanced logging.

4. Admin Pack 1.0. Originally, I was doing some googling on reporting tools for IIS7 and came across a post for an open source report framework that is an extension to IIS. In the post it said that the reporting tools had been added to the Admin Pack. However, once I installed the Admin Pack, I could not find the reports anywhere. Upon further digging, I found some posts stating that the Reporting extension had been pulled at release, but was still in the Admin Pack RC. I could not find anything to suggest that a report extension is forthcoming.

5. URL Rewrite 2.0 (2010-04-05).

6. Microsoft .NET Framework 4 (2010-04-29)

7. Windows Powershell 2.0 (2010-04-29)

8. Visual Web Developer 2010 Express (2010-04-29)

9. Database Manager 1.0 (2010-04-29)

10. ASP.NET MVC 2 (2010-04-29)

11. Windows Cache Extension for PHP 5.2 (2010-04-29)

12. Update for IIS 7.0 FastCGI (KB980363) (2010-04-29)

13. PHP 5.2.13 (2010-04-29)

14. WordPress (2010-04-29)

15. MySQL Connector/Net 5.2 (Dependency) (2010-04-29)

16. MySQL Windows Essential 5.1 (2010-04-29)

17. Hotfix for IIS DB Manager (2010-04-29)

Today, I decided that I needed to see a little bit more about the traffic hitting my website. So I set off attempting to find a tool that will help me do just that. After doing some Googling, I ran across a Microsoft utility called Log Parser. So I downloaded and installed it. A great post to read about Log Parser is by Steve Bunting. Another post by Jeff Atwood here. Microsoft Technet has an article about how Log Parser works here. Log Parser is not officially a supported Microsoft tool, the forum site can be found here. Another article from Symantec discusses the fields for parsing and some examples here. I played around with Log Parser some and got some rudimentary results. But since I didn’t want to recreate the wheel, I decided to keep searching.

After Googling on “IIS7 Statistics”, I stumbled upon AWStats. I decided to install it. The documentation for installing on Windows Server 2008 is pretty weak. I found a  great guide to installing AWStats here. The steps can be summarized as follows:

1. Install ActivePerl here.
2. Install AWStats here. Be sure to change the default installation path to: “C:\AWStats”. If you let it go into ‘Program Files’, you’ll run into trouble in step 5 below as the path with ‘Program Files’ has a space. Trust me, I tried it. I’m sure there’s a work-around, (please post your comments about a solution!) but after spending 30 minutes with various quotes, etc. I gave just up and moved ahead!
3. Create a new default configuration file.
4. Edit the default configuration file with notepad. Run notepad as Administrator.
5. Change entry: LogFile="/var/log/httpd/mylog.log" to: LogFile=”c:\awstats\tools\logresolvemerge.pl c:\inetpub\logs\LogFiles\W3SVC3\*.log |” **Make sure double quotes are not formatted type, ***Be sure to get the right site number at the end of ‘W3SVC#’
6. Change LogFormat = 2 (I had already changed the logformat in IIS7 to W3C when I set up the site, so no additional changes were needed to the LogFormat)
7. Verify SiteDomain and HostAliases vars.
8. Open Command Prompt (cmd) with Administrative Privileges and navigate to “c:\AWStats\wwwroot\cgi-bin” and run “awstats.pl -config=www.yourdomain.com” (don’t add the prefix “awstats” or suffix “conf” it will add it for you)

So, at this point, I’m getting an error about the format of the log file. It doesn’t match the format it’s expecting. My second attempt at generating the stats database was to use the LogFormat string as follows from my guiding post above:

LogFormat=”%time2 %other %method %url %other %query %other %host %other %code % %other %other %bytesd”

This also produced the same type of error. I then opened up IIS7 Manager and navigated to the Webserver Node, selected logging. I then chose the ‘Select Fields’ option. I noticed here, that there are many more fields selected than what is defined in the LogFormat string I’m providing. Here is the list for fields currently being logged and a screen shot showing them as well:

Date, Time, Client IP Address, User Name, Service Name, Server Name, Server IP Address, Server Port, Method, URI Stem, URI Query, Protocol Status, Protocol Substatus, Win32 Status, Bytes Sent, Bytes Received, Time Taken, Protocol Version, Host, User Agent, Cookie, Referer

My next task is to see if I can manually determine the actual format of the log file. So I Googled on “W3SVC IIS7 file format” and got a nice explanation of file formats here. I navigated to “C:\inetput\logs\LogFiles\W3SVC3” and opened up the current log file. I immediately noticed that there are #Fields: statements in the file. Here is a sample line from the log file:

#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken

I then compared the commented #Fields with actual content in the log file, and they appear to match. I then cross-referenced the following list of fields from the awstats config file:

#   %host             Client hostname or IP address (or Sender host for mail log)
#   %host_r           Receiver hostname or IP address (for mail log)
#   %lognamequot      Authenticated login/user with format: "john"
#   %logname          Authenticated login/user with format: john here
#   %time1            Date and time with format: [dd/mon/yyyy:hh:mm:ss +0000] or [dd/mon/yyyy:hh:mm:ss]
#   %time2            Date and time with format: yyyy-mm-dd hh:mm:ss
#   %time3            Date and time with format: Mon dd hh:mm:ss or Mon dd hh:mm:ss yyyy
#   %time4            Date and time with unix timestamp format: dddddddddd
#   %methodurl        Method and URL with format: "GET /index.html HTTP/x.x"
#   %methodurlnoprot  Method and URL with format: "GET /index.html"
#   %method           Method with format: GET
#   %url              URL only with format: /index.html
#   %query            Query string (used by URLWithQuery option)
#   %code             Return code status (with format for web log: 999)
#   %bytesd           Size of document in bytes
#   %refererquot      Referer page with format: "http://from.com/from.htm"
#   %referer          Referer page with format: http://from.com/from.htm
#   %uabracket        User agent with format: [Mozilla/4.0 (compatible, ...)]
#   %uaquot           User agent with format: "Mozilla/4.0 (compatible, …)"
#   %ua               User agent with format: Mozilla/4.0_(compatible…)
#   %gzipin           mod_gzip compression input bytes: In:XXX
#   %gzipout          mod_gzip compression output bytes & ratio: Out:YYY:ZZpct.
#   %gzipratio        mod_gzip compression ratio: ZZpct.
#   %deflateratio     mod_deflate compression ratio with format: (ZZ)
#   %email            EMail sender (for mail log)
#   %email_r          EMail receiver (for mail log)
#   %virtualname      Web sever virtual hostname. Use this tag when same log
#                     contains data of several virtual web servers. AWStats
#                     will discard records not in SiteDomain nor HostAliases
#   %cluster          If log file is provided from several computers (merged by
#                     logresolvemerge.pl), use this to define cluster id field.
#   %extraX           Another field that you plan to use for building a
#                     personalized report with ExtraSection feature (See later).
#   If your log format has some fields not included in this list, use:
#   %other            Means another not used field
#   %otherquot        Means another not used double quoted field

After mapping, here is my new LogFormat file:

LogFormat=”%time2 %other %other %host_r %method %url %query %other %logname %host %other %ua %other %referer %other %code %other %other %bytesd %other %other”

A couple notes here about why step 8, the statistics build, didn’t work. First, in step 5, I was using “W3SVC1” instead of “W3SVC3”. The number at the end indicates the site. The site I’m using is site 3. Second, I selected additional fields to log several days ago. This changed the format of the file from its original state. So, the build program had problems recognizing the older record formats. To solve this, I moved all of the old log files out into another directory so that only the files with the newest format are being used in the database.

I now continued on, repeating from step 8 above:

1. Open Command Prompt (cmd) with Administrative Privileges and navigate to “c:\AWStats\wwwroot\cgi-bin” and run “awstats.pl -config=www.yourdomain.com” **If it runs successfully, you will get Phase 1 …, Phase 2 …, Jumped lines …, Parsed lines …, …, Found 804 new qualified records.
2. Run “awstats.pl -config=www.yourdomain.com –output –staticlinks > ..\yourdomain_com_stats.html” **remember to use “..\” to build the html file in directory above.

At this point, I used Windows explorer and navigated to “C:\AWStats\wwwroot\” and clicked on the generated html file. I expected something magical to appear, but was disappointed to see it display:

WARNING: LastLine parameter in history file is ‘20100426184511′ so in future. May be you need to correct manually the line LastLine in some awstats*.www.larrybrouwer.com.conf files. Create/Update database for config "C:\AWStats\wwwroot\cgi-bin/awstats.www.larrybrouwer.com.conf" by AWStats version 6.95 (build 1.943) From data in log file "c:\awstats\tools\logresolvemerge.pl c:\inetpub\logs\LogFiles\W3SVC3\*.log |"… Phase 1 : First bypass old records, searching new record… Searching new records from beginning of log file… Jumped lines in file: 0 Parsed lines in file: 804 Found 0 dropped records, Found 0 corrupted records, Found 804 old records, Found 0 new qualified records.

I then Googled “AWStats Warning LastLine parameter in history file” and found numerous posts, one of which is here. It appears that the IIS7 outputs timestamps in GMT, and my server is on US Central Time (GMT-0600), so I need to correct for that as indicated here. There is a parameter in the AWStats config file:

LoadPlugin=”timezone –6”

I tried –6 first, but got the same result. A Wikipedia explanation here is useful. Daylight savings is the culprit, so I changed the parameter to “–7”, and it worked! I’ve temporarily put it at http://www.larrybrouwer.com:81/stats/larrybrouwer_com_stats.html for viewing. Sadly, clicking on this link won’t work since DNN is re-writing the URL. Bummer! It’ll be changing the link in the near future anyway when I integrate it into my DotNetNuke site.

Additional Steps to work on:

1. Set it up to view from my DNN website
2. Have it automatically produce new stats
3. Have it automatically compensate for Daylight Savings

(see my post, Adding AWStats to DotNetNuke)

One final note… While trying to integrate AWStats into DNN, I stumbled upon the logging features in DNN. Logging is not turned on by default now as is outlined in this post here. Per this post, to enable logging, as administrator, navigate to: Site Settings \ Advanced Settings \ Host Settings \ Site Log History and change it to a positive number representing the number of days you wish to have logs kept.

To view the logs, click on Admin then select Site Log in the Basic Features section.

Update on 4/27/2010:

I’ve noticed that the gifs aren’t showing up in the viewed pages. So I went poking around through the installation document that comes with the software. It’s located at c:\AWStats\docs\awstats_setup.html. It probably would have been best to follow these instructions first as it describes in detail my current problems. It also discusses how to set up the log file just for AWStats use (i.e. disabling all the unnecessary fields).

So I’m going to fix my installation for IIS7 using the documentation that came with the software.

1. Configure IIS7 to create logs in the “Extended W3C log format”. I’m leaving my log file formats the way they are. Since I’ve already re-mapped the fields, no need to change them again.
2. Copy the contents of AWStats cgi-bin folder to your server’s cgi-bin directory. In my case I have two websites running. So I created new cgi-bin dirs for each site.
3. Copy the icon directory and sub-directory contents to the target website root folders. Again, I copied the icon folder to my two sites.
4. Create a configuration file. I’m using the one I’ve already configured.
5. Edit your configuration file. LogFile values remain the same; LogType=”W”, stays the same; LogFormat stays the same; Change the DirIcons parameter to reflect the relative path of icon directory. In my case, I don’t have to change the location since I’ve now moved the icon dir to the proper location off the root directory.

Now the icons are showing up fine.

Looking more closely at the AWStats report, I’ve noticed that there links to other reports that aren’t there. Doing some more looking at the documentation, I see that there are many more reports that can be generated. A quick way to build up all the reports is to use awstats_buildstaticpages.pl. Here’s the script I created:

I’ve just configured AWStats to work with IIS7 on my web server (see my post, IIS7 Statistics Reporting with Log Parser and AWStats), and now I would like to integrate it into my DotNetNuke (DNN) website.

Here’s an outline of what I would like to do:

1. Be able to log into my DNN web site as an authenticated user and navigate to a web statistics page. I don’t want anonymous users to view my stats.
2. Create a script that runs in the background to keep the statistics up to date and have it re-generate the html page for the stats.
3. See if I can programmatically adjust the LoadPlugin=”timezone –7” for US Central daylight savings or standard time. I really don’t want to have to remember this 2x per year.

So the first thing I did was to open the IIS7 Manager and navigate to my DNN website. I then Right Clicked on it and selected ‘Add Virtual Directory…’. I then choose ‘Statistics’ for my Alias, and navigated to the correct directory path of the AWStats webroot or ‘C:\AWStats\wwwroot’. I then opened up IE and navigated to my web site at http://www.larrybrouwer.com/stats/larrybrouwer_com_stats.html, and it displayed just fine.

My first approach to add the statistics html page was to first add a new page to the DNN website logged in as the site administrator, and then have it permanently redirect as a URL to the statistics page. At first, this seemed to work fine. The only problem was when I tried to restrict access to the file, it didn’t have any effect. In other words, I could enter the URL directly into the browser, and I could open the statistics page.

After much fiddling around with various settings, I decided to try another approach. This time, I again created a new page and deleted the default HTML module on the page. I then added an IFrame Module and set the Location to the URL for the statistics page. I set the Width and Height to 100%, checked the Auto height, disabled Scrolling and Border by setting both of them to ‘No’. This seemed to work much better. Now, when I directly enter the URL for the DNN Statistics page, it redirects me to the login screen! Pretty cool.

There’s only one catch… And that is, I can still access the statistics page by directly entering the URL of the actual page. After about 2 hours of Googling and fiddling around with Web.config settings, I came up with the solution. I went into IIS7 Manager, and selected the Statistics virtual directory. I then clicked on the Authorization Rules. I then clicked ‘Add Deny Rule…’. I then selected ‘All anonymous users’ to Deny access to anyone who isn’t a user with a valid id. So I think I’ve got it! This action resulted in a Web.config file being generated in the Statistics virtual directory (or c:\AWStats\wwwroot). Here’s the contents of the Web.config file:

This concludes step 1! On to the next. The second step should be pretty straight forward. All I think I need to do is create a script with essentially two (2) steps in it. First, regenerate the AWStats, then regenerate the html for it. Here’s the script I came up with:

I then added the script to the Task Scheduler. I initially set it up to run every 10 minutes.

The next day (4/27/2010)

I discovered today that I’m still having issues with access security on the direct URL’s. So I decided to install AWStats directly in the DotNetNuke root folder. To me, this makes a lot more sense now, and should help me solve some problems with using virtual directories, etc. I’ll tackle that problem later today, hopefully.

I also fixed a problem with the icons not displaying correctly (see my other post for more details here).

I also updated my script to generate all the reports automatically:

I also changed DNN to allow for uploading of .html files. Host / Host Settings / Advanced Settings / Other Settings / Allowable File Extensions which in an attempt to fix my file security issues. More on that later.