Middleware Mayhem: How FunHive Stopped a Logging Black Hole in Your .NET Pipeline

Every .NET API team eventually hits a moment where logging middleware stops being helpful and starts being a problem. Requests vanish without a trace, errors get logged twice (or not at all), and performance degrades under load. This guide is for developers and architects who need a practical, decision-oriented walkthrough of how to design logging middleware that works—without the black holes.

Who Must Choose and By When: The Logging Middleware Decision Frame

If you're building a new .NET API or maintaining an existing one, the logging middleware decision lands on your plate earlier than you expect. Typically, it's during the first sprint when a production bug surfaces and you realize you have no idea what happened. That's the moment you need a coherent strategy, not a quick fix.

The choice involves three layers: what to log, when to log it, and how to structure the pipeline so that logging doesn't interfere with request processing. Teams often start with a simple app.Use(async (context, next) => { ... }) and add more handlers as needs grow. Before long, the pipeline becomes a tangled mess where middleware order matters but nobody documented why.

You need to decide by the time you have more than one middleware component that touches logging. That could be as early as your first custom exception handler. Waiting until performance degrades or logs become unreadable means you'll be refactoring under pressure. The goal is to define a clear ordering and responsibility for each piece before the pipeline grows beyond three handlers.

This guide assumes you're using ASP.NET Core and are familiar with the IMiddleware interface or the convention-based RequestDelegate approach. We'll focus on the patterns that prevent the most common failures: silent swallowing of exceptions, logging sensitive data, and performance bottlenecks from synchronous I/O.

When the Decision Becomes Urgent

The urgency typically spikes after a production incident where logs were missing or misleading. For example, a team might discover that their global exception middleware logs the error but then rethrows, causing a second log from the framework. Or they find that request body logging is buffering large payloads in memory, causing out-of-memory errors under load. These are the signals that your current approach is broken.

If you haven't yet formalized your middleware logging strategy, the best time to act is before your next deployment. Even a simple documented convention—like "log at the start of the pipeline for request metadata, at the end for response status, and in exception middleware for errors only"—can prevent chaos.

Three Approaches to Logging Middleware: The Landscape

There are three common patterns teams use for logging in the ASP.NET Core pipeline. Each has strengths and weaknesses, and none is universally best. Understanding the trade-offs helps you choose the right mix for your API's complexity and traffic patterns.

Approach 1: Global Exception Middleware

This is the simplest pattern: a single middleware that wraps the entire pipeline in a try-catch, logs the exception, and returns a standardized error response. It's often the first logging middleware teams implement because it's easy and covers unhandled exceptions.

Pros: catches everything, simple to implement, works with any logging framework. Cons: logs only exceptions, not successful requests; can swallow exceptions if not rethrown correctly; often logs too much detail in development and not enough in production.

Many teams stop here, but this leaves a blind spot for 4xx errors and successful requests that are slow or return unexpected data. You need additional middleware to fill those gaps.

Approach 2: Request/Response Logging Middleware

This pattern logs every incoming request and outgoing response, typically including HTTP method, path, status code, and duration. Some implementations also log request and response bodies, which is where the black hole risk appears.

Pros: provides full visibility into API usage, helps debug client issues, enables performance monitoring. Cons: body logging can leak sensitive data (passwords, tokens) and consume significant memory for large payloads; synchronous stream reading can block the pipeline; logging every request at high volume can overwhelm your logging infrastructure.

The key is to be selective: log body only for errors or specific endpoints, and always use buffered, seekable streams to avoid corrupting the request pipeline.

Approach 3: Structured Telemetry Middleware

This is a more modern pattern that uses structured logging (e.g., Serilog, NLog, or OpenTelemetry) to emit events with properties rather than plain text messages. The middleware captures metrics like request duration, error count, and custom dimensions, and sends them to a centralized system.

Pros: enables querying and aggregation, supports distributed tracing, reduces log volume by sampling. Cons: requires a structured logging setup and a compatible backend (Elasticsearch, Application Insights); more complex to configure; may still need exception middleware for fallback.

Structured telemetry is ideal for microservices architectures where you need to correlate requests across services. But it's overkill for a simple CRUD API with low traffic.

Criteria for Choosing the Right Mix

To decide which combination of these approaches fits your project, evaluate the following criteria. Each factor should be weighted based on your team's priorities and constraints.

Traffic Volume and Logging Infrastructure

If your API handles thousands of requests per second, logging every request with full body content will overwhelm your log storage and increase latency. In that case, you need sampling, aggregation, or structured telemetry that reduces volume. For low-traffic internal APIs, request/response logging with body capture is often fine.

Consider the cost of your logging backend. Many teams are surprised by the bill after enabling verbose logging on a high-traffic endpoint. Always estimate log volume before deploying.

Sensitivity of Data

If your API processes personal data, financial information, or authentication tokens, body logging is risky. Even if you mask fields in the log output, the raw data may be captured in transit. In such cases, stick to logging metadata only (method, path, status, duration) and rely on exception middleware for error details.

Regulatory requirements like GDPR or PCI-DSS may prohibit logging certain fields altogether. Consult your compliance team before enabling body logging.

Debugging Needs

If your team frequently debugs client-reported issues where you need to see exactly what the client sent and what the server returned, request/response logging is invaluable. But you can limit it to specific endpoints or enable it conditionally based on a request header or environment variable.

For example, you might log full bodies only when a X-Debug: true header is present. This gives you on-demand visibility without the permanent overhead.

Operational Maturity

Teams with mature observability practices (dashboards, alerts, distributed tracing) will benefit most from structured telemetry middleware. Teams that rely on reading raw log files will prefer request/response logging with clear, readable messages.

Don't adopt structured telemetry just because it's trendy. It requires investment in tooling and training. If your team is small and your API is simple, global exception middleware plus basic request logging may be sufficient.

Trade-Offs Table: Choosing Your Logging Middleware Mix

No single pattern covers all needs. Most production APIs use a combination: global exception middleware for safety, request/response logging for debugging (with selective body capture), and structured telemetry for metrics and tracing. The exact blend depends on your criteria from the previous section.

Common Mistake: Mixing Patterns Without Clear Ordering

A frequent pitfall is adding multiple logging middleware without defining their order. For example, if exception middleware runs before request logging, an exception will be logged by the exception handler but the request logging middleware may never execute, leaving no record of the failed request. The fix is to place request logging early in the pipeline (so it captures every request) and exception middleware later (so it catches errors from downstream).

Another mistake: logging the same information in two places. If your structured telemetry middleware already logs request duration, don't also log it in a separate request logging middleware. Duplicate logs inflate storage and confuse analysis.

Implementation Path After the Choice

Once you've selected your middleware mix, follow these steps to implement it correctly. The order matters, and skipping steps often leads to the black hole behavior we're trying to avoid.

Step 1: Define Your Logging Contracts

Before writing any middleware, decide what information each piece will capture and in what format. For structured logging, define the property names and types. For plain text, agree on a consistent message template. This prevents each developer from adding their own style.

Example: a contract might say "Exception middleware logs only exception details and request path; request logging middleware logs method, path, status, duration, and client IP; body logging is disabled by default."

Step 2: Order Middleware Correctly

In Program.cs, the order of app.Use... calls determines the pipeline execution order. A typical safe order is:

1. Exception middleware (wraps everything, catches errors from all downstream)
2. Request logging middleware (logs incoming request metadata)
3. Other middleware (authentication, authorization, routing, etc.)
4. Response logging middleware (logs outgoing response status and duration)

If you use structured telemetry, place it after exception middleware but before other middleware to capture timing accurately.

Step 3: Implement Safe Body Logging

If you need to log request or response bodies, always enable buffering first. For requests, call context.Request.EnableBuffering() at the start of the middleware and reset the stream position after reading. For responses, use a MemoryStream to capture the output, then copy it to the original stream. Never read the body without buffering—it will be consumed and downstream middleware won't see it.

Also, limit body size. Log only the first N kilobytes to avoid memory issues. And never log binary content like images or file uploads.

Step 4: Test Under Load

Deploy your logging middleware to a staging environment and simulate realistic traffic. Monitor memory usage, CPU, and log volume. Look for unexpected increases. A common surprise is that logging middleware adds milliseconds per request, which at scale becomes seconds of added latency.

Use a tool like dotnet-counters or Application Insights to measure the impact. If you see high GC pressure, consider reducing the amount of data logged per request.

Risks If You Choose Wrong or Skip Steps

Choosing the wrong logging middleware pattern or implementing it poorly can lead to several serious problems. Understanding these risks helps justify the upfront investment in a deliberate design.

Silent Data Loss (The Black Hole)

The most feared risk is that errors occur but are never logged. This happens when exception middleware catches an exception but fails to log it before rethrowing, or when the logging middleware itself throws an exception that is caught by a higher-level handler that doesn't log. The result: you know something went wrong (because the client got a 500), but you have no clue what.

To prevent this, ensure your exception middleware logs the exception before rethrowing, and consider using a fallback logger (like ILogger to console) that works even if the main logging infrastructure is down.

Performance Degradation

Synchronous I/O in middleware blocks the thread pool. If your logging middleware reads the request body synchronously or writes logs synchronously, it can cause thread starvation under load. Always use asynchronous methods (ReadAsync, WriteAsync) and avoid blocking calls.

Another performance risk: logging too much data per request. Logging large payloads or many properties increases serialization overhead. Use sampling or truncation to keep each log entry small.

Security Breach via Logs

Logging sensitive data—passwords, credit card numbers, personal information—can turn your log files into a liability. If an attacker gains access to your logs, they have a treasure trove. Even if you trust your logging backend, consider that logs are often retained for long periods and may be accessible to more people than the production database.

Mitigation: never log sensitive fields. Use a whitelist approach: log only the fields you explicitly need. If you must log a sensitive value for debugging, mask it (e.g., replace all but last four characters with asterisks).

Increased Operational Cost

Every log entry costs storage and processing. If you log every request with full body content, your logging bill can skyrocket. Many teams have been surprised by a six-figure monthly bill from cloud logging services after enabling verbose logging across all environments.

Solution: use log levels wisely. Log verbose details only in development or debugging mode. In production, log at Information level for request metadata and Warning/Error for exceptions. Use sampling for high-traffic endpoints.

Mini-FAQ: Common Questions About Logging Middleware

Should I log at the beginning or end of the pipeline?

Both, but for different purposes. Log at the beginning to capture request metadata (method, path, headers) before any processing. Log at the end to capture response status and duration. Combining both gives you a complete picture of each request. However, avoid logging the same data twice. A common pattern is to log request start at the beginning, then log the result at the end using a stopwatch started at the beginning.

How do I avoid logging health check endpoints?

Health check endpoints are called frequently and logging them adds noise. In your request logging middleware, check if the request path matches your health check route (e.g., /health) and skip logging for those requests. Alternatively, log them at a lower level (Debug) so they don't appear in normal logs.

What if my logging middleware throws an exception?

This is a critical failure path. If the logging middleware itself throws, the exception should be caught by the global exception middleware (if placed before it) or by the framework. But the log entry is lost. To mitigate, wrap your logging logic in a try-catch that logs to a fallback (like Console.WriteLine) and does not rethrow. This ensures the request continues even if logging fails.

Should I use IMiddleware or the convention-based approach?

Both work. IMiddleware allows dependency injection per middleware instance, which is useful for services like ILogger. The convention-based approach (InvokeAsync) is simpler and doesn't require registration. Choose based on whether you need DI in the middleware. For logging, you typically need ILogger, so IMiddleware is often cleaner.

How do I log request and response bodies without breaking the pipeline?

Always enable buffering before reading the body. For requests: call context.Request.EnableBuffering(), read the body, then reset the position to 0. For responses: replace the response body stream with a MemoryStream, let the rest of the pipeline write to it, then copy the contents to the original stream after logging. This ensures downstream middleware and the client see the full body.

Recommendation Recap: Your Next Moves

Logging middleware doesn't have to be a black hole. The key is to make deliberate choices early, document your pipeline order, and test under realistic conditions. Here are three specific actions you can take today:

1. Audit your current pipeline. List every middleware component in Program.cs and note what each one logs. Identify any gaps or overlaps. If you have more than three logging-related middleware, consider consolidating.
2. Decide on a primary pattern. Based on your traffic, data sensitivity, and debugging needs, choose one or two patterns from the three we discussed. Implement them with clear ordering and contracts. Avoid mixing all three without a clear reason.
3. Add a fallback logger. Ensure that if your main logging infrastructure fails, the exception middleware can still write to console or a local file. This prevents silent failures when your logging backend is down.

Finally, treat your logging middleware as a living part of your API. As traffic grows or requirements change, revisit your choices. A pattern that works for a prototype may not scale to production. By staying intentional, you can keep your logs useful and your debugging effective.