C windows system32 logfiles httperr

Your web application is throwing some sort of error or invalid response. Do you know how to troubleshoot IIS or ASP.NET errors on your servers? Luckily, Windows & ASP.NET provide several different logs where failed requests are logged.

You are probably familiar with normal IIS logs, but there are some other places to look if you are looking for more detailed error messages or can’t find anything in your IIS log files.

1. Standard IIS Logs

Standard IIS logs will include every single web request that flows through your IIS site. Via IIS Manager you can verify that your IIS logs are enabled and where they are being written to. 

You should find your logs in folders that are named by your W3SVC site ID numbers.

By default each logged request in your IIS log will include several key fields including the URL, querystring, and error codes via the status, substatus and win32 status. These status codes can help identify the actual error in more detail.

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
2016-09-13 21:45:10 ::1 GET /webapp2 - 80 - ::1 Mozilla/5.0 - 500 0 0 5502
2016-09-13 21:45:10 ::1 GET /favicon.ico - 80 - ::1 Mozilla/5.0 http://localhost/webapp2 404 0 2 4

The “sc-status” and “sc-substatus” fields are the standard HTTP status code of 200 for OK, 404, 500 for errors, etc. 

The “sc-win32-status” can provide more details that you won’t know unless you lookup the code. They are basic Win32 error codes.

2. Can’t find your request in the IIS log? HTTPERR is your IIS error log

Every single web request should show in your IIS log. If it doesn’t, it is possible that the request never made it to IIS, or IIS wasn’t running. Also, be sure that your IIS logging is enabled.

Incoming requests to your server first route through HTTP.SYS before being handed to IIS. These type of errors get logged in HTTPERR. Common errors are 400 Bad Request, timeouts, 503 Service Unavailable and similar types of issues. The built in error messages and error codes from HTTP.SYS are usually very detailed.

Where are the HTTPERR error logs?
C:\Windows\System32\LogFiles\HTTPERR

3. Look for ASP.NET exceptions in Windows Event Viewer

By default ASP.NET will log unhandled 500 level exceptions to the Windows Application EventLog. This is handled by the ASP.NET Health Monitoring feature. You can control settings for it via system.web/healthMonitoring in your web.config file.

Very few people realize that the number of errors written to the Application EventLog is rate limited. So you may not find your error! By default it will only log the same type of error once a minute. You can also disable writing any errors to the Application EventLog. 

Still can’t find your exception?

Depending on if you are using WebForms, MVC, Web API, WCF or other frameworks, you may have issues with ASP.NET not writing any errors at all to ASP.NET due to compatibility issues with the health monitoring feature. 

4. Enable Failed Request Tracing for an advanced IIS error log

Failed request tracing (FRT) is probably one of the least used features in IIS. It provides robust IIS logging and works as a great IIS error log. FRT is enabled in IIS Manager and can be configured via rules for all requests, slow requests, or just certain response status codes.

The only problem with FRT is it is insanely detailed. It tracks every detail and every step of the IIS pipeline. You can spend a lot of time trying to decipher a single request.

This post has been republished via RSS; it originally appeared at: IIS Support Blog articles.

In many cases, when troubleshooting web applications hosted in IIS, we start by collecting the basics for investigation:

• The configuration of IIS and underlying HTTP.SYS, of Asp.Net and of the application, and system info.
• The logs and traces from IIS, HTTP.SYS, Windows Events. If we’re lucky, we have FREBs, Failed Request Traces.
• Problem description, to help us figure out how and where to focus during investigation.

I intend to detail in a later article why we look into those files, what we expect to see there. And my colleagues are working on a tool, a PowerShell script, to automate the data collection. But until then…

Start collecting

Many of the files are only accessible to the Administrators on the investigated machine.

So open an administrative command-line console if you’re going to use the commands below. Then…

First, create a folder where all collected files will be placed:

MKDIR C:\MsLogs
C:
CD \MsLogs

Collect HTTP.SYS configuration

netsh http show cacheparam > C:\MsLogs\netsh_http_show_cacheparam.txt
netsh http show cachestate > C:\MsLogs\netsh_http_show_cachestate.txt
netsh http show iplisten > C:\MsLogs\netsh_http_show_iplisten.txt
netsh http show servicestate > C:\MsLogs\netsh_http_show_servicestate.txt
netsh http show setting > C:\MsLogs\netsh_http_show_setting.txt
netsh http show sslcert > C:\MsLogs\netsh_http_show_sslcert.txt
netsh http show timeout > C:\MsLogs\netsh_http_show_timeout.txt
netsh http show urlacl > C:\MsLogs\netsh_http_show_urlacl.txt

Master IIS configuration — applicationHost.config

Default location is:

C:\Windows\System32\inetsrv\config\

Note: It’s better if you can send the whole folder C:\Windows\System32\inetsrv\config\.

Example:

XCOPY %systemroot%\System32\inetsrv\config\*.* C:\MsLogs\ /E

IIS Log files

Only send logs of the site containing the problematic application. Default location:

C:\inetpub\logs\LogFiles\W3SVC[siteID]\

Determine the Site ID from IIS Manager console, selecting the Sites node.

Only include latest files, if too many logs are there; we only need the investigated timeframe. For instance, logs in the last 10 days.

Example:

MKDIR C:\MsLogs\IIS-logs
ROBOCOPY C:\inetpub\logs\LogFiles\W3SVC1\ C:\MsLogs\IIS-logs\ /MAXAGE:10

FREBs, Failed Request Traces…

…if any were collected at all, related to the issue being reported.

Only send traces for the site containing the problematic application. Default location:

C:\inetpub\logs\FailedReqLogFiles\W3SVC[siteID]\

Determine the Site ID from IIS Manager console, selecting the Sites node.

Example:

MKDIR C:\MsLogs\FREBs
COPY C:\inetpub\logs\FailedReqLogFiles\W3SVC1\*.* C:\MsLogs\FREBs\

HttpErr logs

Events and errors logged by the HTTP.SYS driver, which IIS relies on. The default location:

C:\Windows\System32\LogFiles\HTTPERR\

Only include latest files, if too many logs are there; we only need the investigated timeframe. For instance, logs in the last 10 days.

Example:

MKDIR C:\MsLogs\HTTPERR
ROBOCOPY %systemroot%\System32\LogFiles\HTTPERR\ C:\MsLogs\HTTPERR\ /MAXAGE:10

The .NET configuration and root Web.config

Before executing an app, IIS is building its configuration by merging trees with settings — read «config files»: .NET FX config, then Asp.NET root Web.config, applicationHost.config of IIS, application’s local Web.config cascade (root site Web.config, then sub-application’s Web.config etc).

The .NET Framework (FX) configuration:

machine.config

The root ASP.NET configuration:

Web.config

Depending on application bitness, these can be found at:

C:\Windows\Microsoft.NET\Framework(64)\[version]\Config

Example:

C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config

It’s better if you can send the whole folder.

Example:

MKDIR C:\MsLogs\ASP.NET-Config
XCOPY %systemroot%\Microsoft.NET\Framework64\v4.0.30319\Config\*.* C:\MsLogs\ASP.NET-Config\ /E

Application’s local configuration

The Web.config file(s) of the root site — and sub-application(s), if that’s the case — being diagnosed.

Or at least send the application’s effective configuration, compiled by IIS from the above config files:

C:\inetpub\temp\appPools\[appPoolName]\[appPoolName].config

Example:

COPY C:\inetpub\temp\appPools\DefaultAppPool\DefaultAppPool.config C:\MsLogs\

Windows Events Viewer entries

We’ll take the following 3 logs from Windows Events Viewer

1. Application Event logs
2. System Event logs  
3. Security Event logs

Select each of them and then…

1. On the right side, pick Filter Current Log…, then only last 7 or 30 days
      (make sure the period covers moment when issue was manifesting).
2. Then, on the right side again, pick Save Filtered Log File As… EVTX,
      (make sure you include ‘Display information for these languages: English‘)

To collect entries in last 10 days, you can export them by running the following command lines:

WEVTUTIL export-log System C:\MsLogs\WinEvents-System.evtx /query:"*[System[TimeCreated[timediff(@SystemTime) <= 864000000]]]"
WEVTUTIL export-log Application C:\MsLogs\WinEvents-Application.evtx /query:"*[System[TimeCreated[timediff(@SystemTime) <= 864000000]]]"
WEVTUTIL export-log Security C:\MsLogs\WinEvents-Security.evtx /query:"*[System[TimeCreated[timediff(@SystemTime) <= 864000000]]]"

Note that timediff function returns milliseconds from now. One day means 24 hours x 60 minutes x 60 seconds x 1.000 milliseconds = 86.400.000.

IIS installation logs

Collect Component-Based Setup log file — the packages that Windows installed.
Then IIS setup logs — how the IIS instance installation went.

COPY C:\Windows\Logs\CBS\cbs.log C:\MsLogs\
COPY C:\Windows\iis.log C:\MsLogs\

System information

Export the output of MsInfo32, or simply run the command:

MsInfo32 /nfo C:\MsLogs\System-Info.nfo
MsInfo32 /report C:\MsLogs\System-Info.txt

Archive the resulting data collection folder C:\MsLogs\ in a ZIP file, then upload the archive in the file transfer space for the case.

You’ll need to send the collected data and/or share it with the engineer helping you do the troubleshooting.

Application issue details

You have to provide as much descriptive information as you can around what happens.

• Please provide the name of the site and/or application with the problem and also the Application Pool name;
• The affected URLs, possibly with screenshots or HTTP traces from client;
• Observed behavior, such as long response time, high-memory consumption or high-CPU;
• What the application is supposed to do, or what are the URLs exerting the problem are doing;
• Reproducing steps, if applicable;
• Any pattern of usage that triggers the behavior: certain time of day, certain users, certain system conditions etc.