Revelation: Why my diagnostic logging was not working using log4net TraceAppender

Question

I thought I better share with people some information that I found which has shed some light on why I've had such a hard time getting native Azure diagnostic logging working.  Hopefully, if I hit enough key words here, someone will find this discussion
in the future and may save you some time and heart ache.
Our application is a legacy ASP.Net application which I am porting to Azure.  The application makes extensive use of log4net and I decided that I would use the log4net TraceAppender to forward the logs to the Azure diagnostic listener.  Within
my worker role, I was configuring my azure logs like so:
private static void ConfigureLogging() {
CrashDumps.EnableCollection(true);
var diagConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();
var directories = diagConfig.Directories;
var infrastructureDiagnostics = diagConfig.DiagnosticInfrastructureLogs;
var applicationLogs = diagConfig.Logs;
var eventLogs = diagConfig.WindowsEventLog;
SetTransferPeriod(applicationLogs, 1);
SetTransferPeriod(directories, 1);
SetTransferPeriod(infrastructureDiagnostics, 1);
SetTransferPeriod(eventLogs, 1);
SetFilterLevel(applicationLogs, LogLevel.Information);
SetFilterLevel(eventLogs, LogLevel.Information);
SetFilterLevel(infrastructureDiagnostics, LogLevel.Warning);
DiagnosticMonitor.Start("DiagnosticsConnectionString", diagConfig);
Log4NetHelper.ConfigureLog4Net();
private static void SetFilterLevel(WindowsEventLogsBufferConfiguration eventLogs, LogLevel logLevel)
eventLogs.DataSources.Add( "Application!*" );
eventLogs.DataSources.Add( "System!*" );
eventLogs.ScheduledTransferLogLevelFilter = logLevel;
private static void SetFilterLevel(BasicLogsBufferConfiguration infrastructureDiagnostics, LogLevel logLevel)
infrastructureDiagnostics.ScheduledTransferLogLevelFilter = logLevel;
private static void SetTransferPeriod(DiagnosticDataBufferConfiguration directories, int minutes)
var period = TimeSpan.FromMinutes(minutes);
directories.ScheduledTransferPeriod = period;
Log4NetHelper.Configure uses the log4net programattic API to setup and configure a TraceAppender that captured all LogLevels (DEBUG or higher).  I also made sure that the follwing appeared in my Web.Config
<system.diagnostics>
<trace autoflush="false" indentsize="4">
<listeners>
<clear />
<add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="AzureDiagnostics">
<filter type="" />
</add>
</listeners>
</trace>
</system.diagnostics>
However, this never worked.  I saw a number of forum posts that suggested that the Azure filter level should be "Undefined" - namely that I should replace
SetFilterLevel(applicationLogs, LogLevel.Information);
with
SetFilterLevel(applicationLogs, LogLevel.Undefined);
I had set my filter level to Information, cos our app does so much Debug level logging, that I thought I'd save money in storage by really only capturing the INFO or higher messages.  But in the interest of getting the blasted thing to work, I set to
Undefined, and sure enough all the log message from all levels came through. Setting back to Information and NO messages come though.
I then noticed that when my logging was working (filter = Undefined) that ALL my log messages were in fact being logged at the Verbose level - even log4net calls to log.Error/log.Warn (exceptions messages being logged at Verbose is not a good sign!)
You have to realize, that my thought process was not structured at this point as I had no idea what was going on.  Until I found this article:
http://www.dotnetsolutions.co.uk/blog/archive/2010/02/22/windows-azure-diagnostics-%E2%80%93-why-the-trace-writeline-method-only-sends-verbose-messages/
While not related to log4net, it was an eye opener, and I used reflector to look at the TraceAppender, and I see they are using Trace.Write to write messages.  
In my mind, this explains why I was seeing the behaviour I was seeing:  AzureDiagnostic listener is converting all Trace.Write messags to Verbose (details in the above article) and log4net TraceAppender is converting all log messages to Trace.Write
messages.  Setting the filter to Information was thus stripping all the calls.
My solution is to inherit TraceAppender and change their impl of Append to use a switch on the LogLevel:
using System.Diagnostics;
using log4net.Appender;
using log4net.Core;
namespace XXX.Azure
public class AzureTraceAppender : TraceAppender
protected override void Append(LoggingEvent loggingEvent)
var level = loggingEvent.Level;
var message = RenderLoggingEvent(loggingEvent);
if (level >= Level.Error)
Trace.TraceError(message);
else if (level >= Level.Warn)
Trace.TraceWarning(message);
else if (level >= Level.Info)
Trace.TraceInformation(message);
else
Trace.Write(message);
if (ImmediateFlush)
Trace.Flush();
With log4net configured to use this appender, then log messages appearing in the WADLogsTable are all appearing at the correct (or nearest appropriate) level
YMMV
Pete

Answer 1

Thanks, that's extremely useful!
Using your appender, Verbose level messages still were not shown though. What did the trick was changing
Trace.Write(message);
to
Trace.WriteLine(message);