Performance Tuning

Optimize platform performance for large-scale data transformation workloads.

Overview

Key areas for performance optimization:

  • JVM memory configuration

  • Database connection pooling

  • Parallel processing

  • Batch sizing

JVM Tuning

Memory Settings

JAVA_OPTS="-Xms4g -Xmx8g -XX:+UseG1GC"

Recommendations: * Initial heap (-Xms): 50% of max heap * Max heap (-Xmx): 50-75% of container memory * Use G1GC for large heaps (> 4GB)

Database Connection Tuning

spring.datasource.hikari.maximum-pool-size=20
spring.datasource.hikari.minimum-idle=5
spring.datasource.hikari.connection-timeout=30000

Parallel Processing

Scale horizontally with multiple agents:

# Docker Compose
services:
  agent:
    replicas: 4  # 4 parallel workers

Batch Sizing

Tune batch sizes for your workload:

default_config:
  batch_size: 10000  # Rows per batch

  • Larger batches: Better throughput, more memory

  • Smaller batches: Lower memory, more overhead

See Also