# Transformation Issues

> Source: https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/common_issues/transformation
> For the complete documentation index, see [llms.txt](https://docs.synthesized.io/llms.txt).

Troubleshoot problems during data masking, generation, and subsetting.

## Foreign Key Constraint Violations

### Symptom

Error: "Foreign key constraint violation", "Referential integrity error".

### Common Causes

1. **Parent table processed after child**

The platform should handle this automatically, but if you see this error:

```yaml
# Ensure parent tables are listed first
table_schema:
  - table_name_pattern: "customers"  # Parent
  - table_name_pattern: "orders"     # Child
```

1. **Missing foreign key metadata**

Solution: Define virtual foreign keys:

```yaml
foreign_keys:
  - columns: ["customer_id"]
    reference_table: "customers"
    reference_columns: ["id"]
```

See: [Virtual Foreign Keys](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/managing_data_properties/virtual_foreign_keys)

## Type Conversion Errors

### Symptom

"Cannot convert value", "Type mismatch", "Invalid data type".

### Solutions

1. **Check transformer compatibility**:

```yaml
# ❌ Wrong: Email transformer on numeric column
- columns: ["customer_id"]
  params:
    type: person_generator
    column_templates: ["${email}"]

# ✓ Correct: Use appropriate transformer
- columns: ["customer_id"]
  params:
    type: int_sequence_generator
```

1. **Verify column data types**:

```sql
-- Check actual data type
SELECT data_type
FROM information_schema.columns
WHERE table_name = 'customers'
AND column_name = 'email';
```

1. **Use custom transformations**:

See: [Scripting Transformer](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/advanced_workflows/scripting_transformer)

## NULL Value Handling

### Symptom

Unexpected NULL values in output, "NOT NULL constraint violation".

### Solutions

1. **Preserve NULLs from source**:

The platform preserves NULL values by default.

1. **Generate non-NULL values**:

```yaml
transformations:
  - columns: ["email"]
    params:
      type: person_generator
      column_templates: ["${email}"]
    # Platform preserves NULLs by default; configure as needed
```

1. **Check NOT NULL constraints**:

```sql
-- Verify constraint
SELECT column_name, is_nullable
FROM information_schema.columns
WHERE table_name = 'customers';
```

## Memory Errors

### Symptom

"OutOfMemoryError", "GC overhead limit exceeded", workflow hangs.

### Solutions

1. **Increase JVM heap**:

```shell
export JAVA_OPTS="-Xmx8g"
```

1. **Reduce batch size**:

```yaml
default_config:
  batch_size: 5000  # Smaller batches
```

1. **Process tables separately**:

Split large workflows into multiple smaller workflows.

## Slow Transformation Performance

### Symptom

Workflow takes much longer than expected.

### Solutions

1. **Check database indexes**:

```sql
-- Ensure foreign key columns are indexed
CREATE INDEX idx_customer_id ON orders(customer_id);
```

1. **Optimize transformations**:
  
  - Avoid complex scripting transformers when simple ones suffice
  - Use deterministic transformations when possible
2. **Scale horizontally**:

Add more workers: See [Scaling Guide](https://docs.synthesized.io/tdk/latest/user_guide/050_deployment_operations/scaling/)

## Character Encoding Issues

### Symptom

Special characters appear garbled or corrupted.

### Solutions

1. **Verify database encoding**:

```sql
-- PostgreSQL
SHOW server_encoding;

-- MySQL
SHOW VARIABLES LIKE 'character_set%';
```

1. **Set JDBC encoding**:

```yaml
url: jdbc:postgresql://host:5432/db?characterEncoding=UTF-8
```

## See Also

- [Reading Logs](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/debugging/logs)
- [Performance Troubleshooting](https://docs.synthesized.io/tdk/latest/user_guide/080_troubleshooting/performance/)
- [Transformations Reference](https://docs.synthesized.io/tdk/latest/user_guide/040_reference/transformers/transformations)