Module 5
Building Scalable Distributed Systems with Java
This work is licensed under CC BY-NC-SA 4.0
© Way-Up 2025
Microservices is an architectural style that structures an application as a collection of small, autonomous services modeled around a business domain.
Organize services around what the business does
// E-commerce example
- Product Catalog Service
- Inventory Service
- Order Service
- Payment Service
- Customer Service
- Notification Service
Use Domain-Driven Design bounded contexts
// Banking example
- Account Management (Core Domain)
- Transaction Processing (Core Domain)
- Fraud Detection (Supporting Domain)
- Reporting (Generic Domain)
Principle: Each microservice has its own private database
Implementation: Use SAGA pattern or Event Sourcing for distributed transactions
Problem: Clients need to communicate with multiple microservices
@RestController
@RequestMapping("/api")
public class ApiGateway {
private final ProductClient productClient;
private final OrderClient orderClient;
private final UserClient userClient;
// Aggregates data from multiple services
@GetMapping("/user/{id}/dashboard")
public UserDashboard getUserDashboard(@PathVariable Long id) {
User user = userClient.getUser(id);
List<Order> orders = orderClient.getUserOrders(id);
List<Product> recommendations = productClient.getRecommendations(id);
return new UserDashboard(user, orders, recommendations);
}
}
// application.yml
spring:
cloud:
gateway:
routes:
- id: product-service
uri: lb://PRODUCT-SERVICE
predicates:
- Path=/api/products/**
filters:
- RewritePath=/api/products/(?<segment>.*), /$\{segment}
- id: order-service
uri: lb://ORDER-SERVICE
predicates:
- Path=/api/orders/**
filters:
- RewritePath=/api/orders/(?<segment>.*), /$\{segment}
Problem: How do services find each other in a dynamic environment?
// In each microservice
@SpringBootApplication
@EnableEurekaClient
public class ProductServiceApplication {
public static void main(String[] args) {
SpringApplication.run(ProductServiceApplication.class, args);
}
}
// application.yml
eureka:
client:
service-url:
defaultZone: http://localhost:8761/eureka/
instance:
prefer-ip-address: true
@FeignClient(name = "product-service")
public interface ProductClient {
@GetMapping("/products/{id}")
Product getProduct(@PathVariable Long id);
@GetMapping("/products/search")
List<Product> searchProducts(@RequestParam String query);
}
// Usage in another service
@Service
public class OrderService {
private final ProductClient productClient;
public Order createOrder(Long productId, int quantity) {
Product product = productClient.getProduct(productId);
// Create order logic
}
}
// Publisher (Order Service)
@Service
public class OrderService {
private final RabbitTemplate rabbitTemplate;
public void placeOrder(Order order) {
// Save order
orderRepository.save(order);
// Publish event
OrderPlacedEvent event = new OrderPlacedEvent(order.getId());
rabbitTemplate.convertAndSend("order-exchange",
"order.placed",
event);
}
}
// Consumer (Inventory Service)
@Component
public class InventoryEventListener {
@RabbitListener(queues = "inventory-queue")
public void handleOrderPlaced(OrderPlacedEvent event) {
// Reduce inventory
inventoryService.reduceStock(event.getOrderId());
}
}
Use when:
Challenges:
Use when:
Benefits:
Problem: Prevent cascading failures when a service is down
@Service
public class ProductService {
@CircuitBreaker(name = "productService", fallbackMethod = "getProductFallback")
@Retry(name = "productService")
@TimeLimiter(name = "productService")
public Product getProduct(Long id) {
// Call external service
return productClient.getProduct(id);
}
private Product getProductFallback(Long id, Exception e) {
log.warn("Circuit breaker activated for product: {}", id);
return Product.builder()
.id(id)
.name("Product Unavailable")
.available(false)
.build();
}
}
# application.yml
resilience4j:
circuitbreaker:
instances:
productService:
sliding-window-size: 10
failure-rate-threshold: 50
wait-duration-in-open-state: 10s
permitted-number-of-calls-in-half-open-state: 3
retry:
instances:
productService:
max-attempts: 3
wait-duration: 1s
timelimiter:
instances:
productService:
timeout-duration: 3s
Problem: How to maintain data consistency across multiple services without distributed transactions?
// Order Service
@Service
public class OrderService {
public void createOrder(OrderRequest request) {
Order order = orderRepository.save(new Order(request));
// Publish event
eventPublisher.publish(new OrderCreatedEvent(order));
}
@EventListener
public void handlePaymentFailed(PaymentFailedEvent event) {
// Compensating transaction
Order order = orderRepository.findById(event.getOrderId());
order.setStatus(OrderStatus.CANCELLED);
orderRepository.save(order);
}
}
// Payment Service
@EventListener
public void handleOrderCreated(OrderCreatedEvent event) {
try {
processPayment(event.getOrderId(), event.getAmount());
eventPublisher.publish(new PaymentSuccessEvent(event.getOrderId()));
} catch (Exception e) {
eventPublisher.publish(new PaymentFailedEvent(event.getOrderId()));
}
}
@Service
public class OrderSagaOrchestrator {
public void executeOrderSaga(OrderRequest request) {
Order order = null;
try {
// Step 1: Create Order
order = orderService.createOrder(request);
// Step 2: Reserve Inventory
inventoryService.reserveInventory(order.getProductId(),
order.getQuantity());
// Step 3: Process Payment
paymentService.processPayment(order.getId(), order.getAmount());
// Step 4: Confirm Order
orderService.confirmOrder(order.getId());
} catch (Exception e) {
// Compensating transactions in reverse order
if (order != null) {
paymentService.refund(order.getId());
inventoryService.releaseInventory(order.getProductId());
orderService.cancelOrder(order.getId());
}
throw new SagaFailedException("Order saga failed", e);
}
}
}
Pattern: Services communicate by publishing and consuming events
// Event Publisher
@Service
public class OrderEventPublisher {
private final StreamBridge streamBridge;
public void publishOrderPlaced(Order order) {
OrderPlacedEvent event = new OrderPlacedEvent(
order.getId(),
order.getCustomerId(),
order.getTotalAmount(),
Instant.now()
);
streamBridge.send("order-events", event);
}
}
// Event Consumer
@Component
public class InventoryEventConsumer {
@Bean
public Consumer<OrderPlacedEvent> handleOrderPlaced() {
return event -> {
log.info("Processing order: {}", event.getOrderId());
inventoryService.updateStock(event);
};
}
}
Command Query Responsibility Segregation: Separate read and write operations
// Write Model (Command)
@Service
public class ProductCommandService {
public void createProduct(CreateProductCommand command) {
Product product = new Product(command);
productRepository.save(product);
// Publish event for read model
eventPublisher.publish(new ProductCreatedEvent(product));
}
}
// Read Model (Query) - Optimized for queries
@Service
public class ProductQueryService {
private final MongoTemplate mongoTemplate; // NoSQL for fast reads
@EventListener
public void updateReadModel(ProductCreatedEvent event) {
ProductView view = new ProductView(event);
mongoTemplate.save(view);
}
public ProductView getProduct(Long id) {
return mongoTemplate.findById(id, ProductView.class);
}
public List<ProductView> searchProducts(String query) {
// Optimized search on read model
return mongoTemplate.find(
Query.query(Criteria.where("name").regex(query)),
ProductView.class
);
}
}
Pattern: Externalize configuration from services
// Config Server
@SpringBootApplication
@EnableConfigServer
public class ConfigServerApplication { }
// application.yml
spring:
cloud:
config:
server:
git:
uri: https://github.com/myorg/config-repo
default-label: main
# bootstrap.yml
spring:
application:
name: product-service
cloud:
config:
uri: http://localhost:8888
fail-fast: true
retry:
max-attempts: 5
Problem: How to trace requests across multiple services?
// Automatic tracing with Sleuth
@RestController
public class OrderController {
@GetMapping("/orders/{id}")
public Order getOrder(@PathVariable Long id) {
// Sleuth automatically adds trace/span IDs to logs
log.info("Fetching order: {}", id);
// Trace propagates to downstream calls
Product product = productClient.getProduct(id);
return orderService.getOrder(id);
}
}
// Log output shows trace and span IDs
// [order-service,a1b2c3d4,e5f6g7h8] Fetching order: 123
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-sleuth-zipkin</artifactId>
</dependency>
// Structured logging with Logback
@Service
public class OrderService {
private static final Logger log = LoggerFactory.getLogger(OrderService.class);
public Order createOrder(OrderRequest request) {
log.info("Creating order: userId={}, productId={}, quantity={}",
request.getUserId(),
request.getProductId(),
request.getQuantity());
try {
Order order = processOrder(request);
log.info("Order created successfully: orderId={}", order.getId());
return order;
} catch (Exception e) {
log.error("Failed to create order: userId={}",
request.getUserId(), e);
throw e;
}
}
}
<appender name="LOGSTASH"
class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<destination>logstash:5000</destination>
<encoder class="net.logstash.logback.encoder.LogstashEncoder" />
</appender>
@Component
public class DatabaseHealthIndicator implements HealthIndicator {
private final DataSource dataSource;
@Override
public Health health() {
try (Connection conn = dataSource.getConnection()) {
boolean valid = conn.isValid(1);
return valid ? Health.up().build() : Health.down().build();
} catch (Exception e) {
return Health.down(e).build();
}
}
}
// Custom metrics
@Service
public class OrderService {
private final MeterRegistry meterRegistry;
public void createOrder(Order order) {
// Increment counter
meterRegistry.counter("orders.created",
"status", "success").increment();
// Record gauge
meterRegistry.gauge("orders.total",
orderRepository.count());
}
}
# application.yml
management:
endpoints:
web:
exposure:
include: health,info,metrics,prometheus
metrics:
export:
prometheus:
enabled: true
tags:
application: ${spring.application.name}
# prometheus.yml
scrape_configs:
- job_name: 'spring-boot-services'
metrics_path: '/actuator/prometheus'
static_configs:
- targets:
- 'product-service:8080'
- 'order-service:8081'
- 'payment-service:8082'
@Component
public class JwtAuthenticationFilter extends OncePerRequestFilter {
@Override
protected void doFilterInternal(HttpServletRequest request,
HttpServletResponse response,
FilterChain filterChain) {
String token = extractToken(request);
if (token != null && jwtTokenProvider.validateToken(token)) {
Authentication auth = jwtTokenProvider.getAuthentication(token);
SecurityContextHolder.getContext().setAuthentication(auth);
}
filterChain.doFilter(request, response);
}
}
@Configuration
public class SecurityConfig {
@Bean
public SecurityFilterChain filterChain(HttpSecurity http) {
return http
.authorizeHttpRequests(auth -> auth
.requestMatchers("/api/public/**").permitAll()
.anyRequest().authenticated()
)
.addFilterBefore(jwtAuthFilter,
UsernamePasswordAuthenticationFilter.class)
.build();
}
}
@Configuration
public class OAuth2ClientConfig {
@Bean
public OAuth2AuthorizedClientManager authorizedClientManager(
ClientRegistrationRepository clientRegistrationRepository,
OAuth2AuthorizedClientRepository authorizedClientRepository) {
OAuth2AuthorizedClientProvider authorizedClientProvider =
OAuth2AuthorizedClientProviderBuilder.builder()
.clientCredentials()
.build();
DefaultOAuth2AuthorizedClientManager authorizedClientManager =
new DefaultOAuth2AuthorizedClientManager(
clientRegistrationRepository,
authorizedClientRepository
);
authorizedClientManager.setAuthorizedClientProvider(
authorizedClientProvider
);
return authorizedClientManager;
}
}
// Usage in Feign Client
@FeignClient(name = "product-service",
configuration = OAuth2FeignConfig.class)
public interface ProductClient {
@GetMapping("/products/{id}")
Product getProduct(@PathVariable Long id);
}
Run multiple service instances on a single server
Each service runs in its own Docker container
Each service runs in its own virtual machine
FROM eclipse-temurin:17-jre-alpine
WORKDIR /app
# Create non-root user
RUN addgroup -g 1000 appuser && \
adduser -D -u 1000 -G appuser appuser
# Copy JAR file
COPY target/product-service.jar app.jar
# Set ownership
RUN chown -R appuser:appuser /app
USER appuser
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s --start-period=40s \
CMD wget --no-verbose --tries=1 --spider http://localhost:8080/actuator/health || exit 1
ENTRYPOINT ["java", \
"-XX:MaxRAMPercentage=75.0", \
"-Djava.security.egd=file:/dev/./urandom", \
"-jar", "app.jar"]
version: '3.8'
services:
config-server:
build: ./config-server
ports:
- "8888:8888"
eureka-server:
build: ./eureka-server
ports:
- "8761:8761"
depends_on:
- config-server
product-service:
build: ./product-service
environment:
- SPRING_PROFILES_ACTIVE=docker
- CONFIG_SERVER_URL=http://config-server:8888
depends_on:
- config-server
- eureka-server
- postgres
postgres:
image: postgres:15-alpine
environment:
POSTGRES_DB: productdb
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
volumes:
- postgres-data:/var/lib/postgresql/data
volumes:
postgres-data:
apiVersion: apps/v1
kind: Deployment
metadata:
name: product-service
spec:
replicas: 3
selector:
matchLabels:
app: product-service
template:
metadata:
labels:
app: product-service
spec:
containers:
- name: product-service
image: myregistry/product-service:1.0.0
ports:
- containerPort: 8080
env:
- name: SPRING_PROFILES_ACTIVE
value: "kubernetes"
resources:
requests:
memory: "512Mi"
cpu: "500m"
limits:
memory: "1Gi"
cpu: "1000m"
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8080
initialDelaySeconds: 60
periodSeconds: 10
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8080
initialDelaySeconds: 30
periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: product-service
spec:
selector:
app: product-service
ports:
- protocol: TCP
port: 80
targetPort: 8080
type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: api-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: api.example.com
http:
paths:
- path: /products(/|$)(.*)
pathType: Prefix
backend:
service:
name: product-service
port:
number: 80
- path: /orders(/|$)(.*)
pathType: Prefix
backend:
service:
name: order-service
port:
number: 80
Services too tightly coupled, all must be deployed together
Too many synchronous calls between services
Multiple services accessing the same database
Manual deployment and testing processes
No visibility into distributed system behavior
Not handling timeouts, retries, circuit breaking
Task: Design a microservices architecture for an e-commerce platform
Deliverable: Architecture diagram with rationale for each decision
Task: Build a simple microservices setup with Spring Boot
Task: Implement order processing with SAGA pattern
Order placement requires coordination between Order, Inventory, and Payment services
In this module, you learned:
Next Module: Concurrency & Multi-Threading