๐ Part 4: Production Implementation Guide
๐งญ Series Navigation:
- ๐ Exception Handling Hub
- โถ๏ธ Part 1: Foundation
- โถ๏ธ Part 2: Architecture
- โถ๏ธ Part 3: Implementation
- โ Part 4: Production Guide (Current)
๐ Production Deployment
Deploy Exception Handling with Confidence Using Proven Strategies
๐ฏ Production Readiness Checklist
- Testing Strategies: Comprehensive exception scenario coverage
- Monitoring & Alerting: Real-time exception tracking and notifications
- Logging Standards: Structured logging for debugging and analysis
- Deployment Process: Step-by-step rollout with safety measures
๐งช Testing Exception Scenarios
Production-ready exception handling requires comprehensive testing that covers all failure scenarios. Your tests should validate both the happy path and exception flows.
Unit Testing Exception Flows
@ExtendWith(MockitoExtension.class)
class UserServiceTest {
@Mock
private UserRepository userRepository;
@Mock
private EmailService emailService;
@Mock
private DatabaseHealthCheck healthCheck;
@InjectMocks
private UserService userService;
@Test
@DisplayName("Should throw ServiceException when database is unavailable")
void createUser_DatabaseUnavailable_ThrowsServiceException() {
// Given
when(healthCheck.isDatabaseAvailable()).thenReturn(false);
CreateUserRequest request = new CreateUserRequest("test@example.com", "John Doe");
// When & Then
ServiceException exception = assertThrows(ServiceException.class,
() -> userService.createUser(request));
assertThat(exception.getErrorCode()).isEqualTo(EXTERNAL_SERVICE_ERROR);
assertThat(exception.getOrigin()).contains("UserService.createUser");
// Verify no database calls were made
verify(userRepository, never()).save(any());
}
@Test
@DisplayName("Should throw DuplicateException when email already exists")
void createUser_EmailExists_ThrowsDuplicateException() {
// Given
when(healthCheck.isDatabaseAvailable()).thenReturn(true);
when(emailService.isAvailable()).thenReturn(true);
when(userRepository.existsByEmail("test@example.com")).thenReturn(true);
CreateUserRequest request = new CreateUserRequest("test@example.com", "John Doe");
// When & Then
DuplicateException exception = assertThrows(DuplicateException.class,
() -> userService.createUser(request));
assertThat(exception.getErrorCode()).isEqualTo(USER_EMAIL_DUPLICATE);
assertThat(exception.getMessage()).contains("test@example.com");
}
@Test
@DisplayName("Should handle database errors during user creation")
void createUser_DatabaseError_ThrowsServiceException() {
// Given
when(healthCheck.isDatabaseAvailable()).thenReturn(true);
when(emailService.isAvailable()).thenReturn(true);
when(userRepository.existsByEmail(any())).thenReturn(false);
when(userRepository.save(any())).thenThrow(new DataAccessException("Connection failed") {});
CreateUserRequest request = new CreateUserRequest("test@example.com", "John Doe");
// When & Then
ServiceException exception = assertThrows(ServiceException.class,
() -> userService.createUser(request));
assertThat(exception.getErrorCode()).isEqualTo(MEMBER_REGISTRATION_FAILED);
assertThat(exception.getCause()).isInstanceOf(DataAccessException.class);
}
}
Integration Testing with TestContainers
@SpringBootTest
@Testcontainers
class UserIntegrationTest {
@Container
static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:13")
.withDatabaseName("testdb")
.withUsername("test")
.withPassword("test");
@Autowired
private TestRestTemplate restTemplate;
@Test
@DisplayName("Should return 409 when creating user with duplicate email")
void createUser_DuplicateEmail_Returns409() {
// Given - Create initial user
CreateUserRequest request = new CreateUserRequest("test@example.com", "John Doe");
restTemplate.postForEntity("/api/users", request, ServiceResponse.class);
// When - Try to create duplicate
ResponseEntity<ServiceResponse> response = restTemplate.postForEntity(
"/api/users", request, ServiceResponse.class);
// Then
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.CONFLICT);
assertThat(response.getBody().isError()).isTrue();
assertThat(response.getBody().getErrorCode()).isEqualTo("USER_EMAIL_DUPLICATE");
}
@Test
@DisplayName("Should return 503 when external service is unavailable")
void createUser_ServiceUnavailable_Returns503() {
// Given - Mock external service failure
mockEmailService.simulateFailure();
CreateUserRequest request = new CreateUserRequest("test@example.com", "John Doe");
// When
ResponseEntity<ServiceResponse> response = restTemplate.postForEntity(
"/api/users", request, ServiceResponse.class);
// Then
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.SERVICE_UNAVAILABLE);
assertThat(response.getBody().getErrorCode()).isEqualTo("EMAIL_SERVICE_UNAVAILABLE");
}
}
๐ Monitoring & Observability
Production exception handling requires comprehensive monitoring to detect issues early and provide actionable insights for debugging.
Exception Metrics with Micrometer
@Component
public class ExceptionMetrics {
private final MeterRegistry meterRegistry;
private final Counter.Builder exceptionCounter;
private final Timer.Builder exceptionTimer;
public ExceptionMetrics(MeterRegistry meterRegistry) {
this.meterRegistry = meterRegistry;
this.exceptionCounter = Counter.builder("business.exceptions")
.description("Count of business exceptions by type and service");
this.exceptionTimer = Timer.builder("business.exception.duration")
.description("Time spent handling exceptions");
}
public void recordException(BaseServiceException exception, String serviceName) {
// Count exceptions by type, service, and severity
exceptionCounter
.tag("exception.type", exception.getClass().getSimpleName())
.tag("error.code", exception.getErrorCode().name())
.tag("service", serviceName)
.tag("severity", exception.getErrorCode().getSeverity().name())
.tag("category", exception.getErrorCode().getCategory().name())
.register(meterRegistry)
.increment();
// Track HTTP status distribution
Counter.builder("http.exceptions")
.tag("status", String.valueOf(exception.getErrorCode().getHttpStatus().value()))
.tag("service", serviceName)
.register(meterRegistry)
.increment();
}
public Timer.Sample startExceptionTimer(String operation) {
return Timer.start(meterRegistry);
}
public void recordExceptionDuration(Timer.Sample sample, String operation, String outcome) {
sample.stop(exceptionTimer
.tag("operation", operation)
.tag("outcome", outcome)
.register(meterRegistry));
}
}
Structured Logging Configuration
# logback-spring.xml
<configuration>
<springProfile name="!local">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
<providers>
<timestamp/>
<logLevel/>
<loggerName/>
<message/>
<mdc/>
<arguments/>
<stackTrace/>
</providers>
</encoder>
</appender>
</springProfile>
<logger name="com.adventuretube.exceptions" level="INFO" additivity="false">
<appender-ref ref="STDOUT"/>
</logger>
<root level="INFO">
<appender-ref ref="STDOUT"/>
</root>
</configuration>
Exception Correlation and Tracing
@Component
public class ExceptionTracker {
private static final String CORRELATION_ID_HEADER = "X-Correlation-ID";
private static final String REQUEST_ID_HEADER = "X-Request-ID";
public void trackException(BaseServiceException exception, HttpServletRequest request) {
String correlationId = getOrCreateCorrelationId(request);
String requestId = getOrCreateRequestId(request);
// Add to MDC for logging
MDC.put("correlationId", correlationId);
MDC.put("requestId", requestId);
MDC.put("exceptionId", exception.getCorrelationId());
MDC.put("origin", exception.getOrigin());
MDC.put("errorCode", exception.getErrorCode().name());
MDC.put("userAgent", request.getHeader("User-Agent"));
MDC.put("clientIp", getClientIpAddress(request));
// Log structured exception data
log.error("Business exception occurred: {}",
Map.of(
"message", exception.getMessage(),
"errorCode", exception.getErrorCode().name(),
"httpStatus", exception.getErrorCode().getHttpStatus().value(),
"severity", exception.getErrorCode().getSeverity(),
"category", exception.getErrorCode().getCategory(),
"origin", exception.getOrigin(),
"timestamp", exception.getTimestamp(),
"correlationId", correlationId,
"requestId", requestId
), exception);
}
private String getOrCreateCorrelationId(HttpServletRequest request) {
String correlationId = request.getHeader(CORRELATION_ID_HEADER);
return correlationId != null ? correlationId : UUID.randomUUID().toString();
}
private String getOrCreateRequestId(HttpServletRequest request) {
String requestId = request.getHeader(REQUEST_ID_HEADER);
return requestId != null ? requestId : UUID.randomUUID().toString();
}
}
๐จ Alerting Strategy
Effective alerting helps teams respond quickly to production issues without creating alert fatigue.
๐จ Critical Alerts
- 5xx Error Rate > 5% in 5 minutes
- Database Connection Failures
- Authentication Service Down
- Circuit Breaker Open
Immediate PagerDuty notification
โ ๏ธ Warning Alerts
- 4xx Error Rate > 10% in 10 minutes
- Validation Failures Spike
- External Service Timeouts
- Memory Usage > 80%
Slack notification to team channel
โน๏ธ Informational
- New Exception Types detected
- Daily Exception Summary
- Performance Degradation
- Unusual Error Patterns
Daily digest email
๐ Deployment Process
A systematic deployment process ensures your exception handling changes are rolled out safely to production.
Pre-Deployment Checklist
โ Code Quality Gates
- โ All exception scenarios have unit tests (min 90% coverage)
- โ Integration tests pass for all error flows
- โ Performance tests show no regression
- โ Security scan passes (no exposed stack traces)
- โ Code review approved by 2+ team members
- โ Exception handling documentation updated
Staged Rollout Strategy
Stage 1: Canary (5% traffic)
- Deploy to 1-2 instances
- Monitor for 30 minutes
- Check error rate, response time, memory usage
- Validate exception logging and metrics
Stage 2: Beta (25% traffic)
- Expand to 25% of instances
- Monitor for 2 hours
- Run smoke tests on all endpoints
- Verify alerting triggers correctly
Stage 3: Full Rollout (100%)
- Deploy to all remaining instances
- Monitor for 24 hours
- Confirm all metrics are normal
- Update runbooks and documentation
Rollback Strategy
# Automated rollback triggers
- Error rate increase > 200% from baseline
- Response time increase > 300% from baseline
- Memory usage > 90% for > 5 minutes
- Critical alerts fired > 3 times in 10 minutes
# Rollback process
1. Immediate: Switch traffic to previous version (blue-green)
2. Investigate: Capture logs, metrics, and error samples
3. Analyze: Identify root cause and fix
4. Test: Verify fix in staging with failure scenarios
5. Redeploy: Follow staged rollout process again
๐ Production Runbook
๐ Exception Investigation Playbook
Step 1: Initial Assessment (0-5 minutes)
- Check error rate dashboard
- Identify affected services and endpoints
- Determine blast radius (% of users affected)
- Check if it’s a new issue or recurring pattern
Step 2: Immediate Actions (5-15 minutes)
- Enable additional logging if needed
- Check external service status pages
- Verify database connectivity and performance
- Consider circuit breaker activation if appropriate
Step 3: Deep Dive (15-60 minutes)
- Analyze exception logs with correlation IDs
- Check recent deployments and configuration changes
- Review application metrics and resource usage
- Examine user journey leading to exceptions
Step 4: Resolution & Communication
- Implement fix or apply temporary mitigation
- Update incident management system
- Communicate status to stakeholders
- Document lessons learned for future prevention
๐ Production Mastery Achieved!
โ Testing Excellence
- Comprehensive exception scenario coverage
- Unit and integration test strategies
- Production-like testing environments
โ Monitoring & Observability
- Exception metrics and dashboards
- Structured logging with correlation
- Real-time alerting strategies
โ Deployment Safety
- Staged rollout with safety gates
- Automated rollback triggers
- Quality gates and checklists
โ Operational Excellence
- Production runbooks and playbooks
- Incident response procedures
- Continuous improvement processes
๐ Series Complete! You’ve Mastered Exception Handling
You now have a complete, production-ready exception handling system with:
- Solid Foundation: HTTP mapping and priority order principles
- Robust Architecture: Custom exceptions and global handling
- Smart Implementation: Controller vs service layer patterns
- Production Confidence: Testing, monitoring, and deployment strategies
๐ Series Complete!
๐งญ Series Navigation:
- ๐ Exception Handling Hub
- โถ๏ธ Part 1: Foundation
- โถ๏ธ Part 2: Architecture
- โถ๏ธ Part 3: Implementation
- โ Part 4: Production Guide (Current)