# Process Unstructured Files

> Source: https://docs.synthesized.io/tdk/latest/user_guide/020_guides/unstructured_data/processing_files
> For the complete documentation index, see [llms.txt](https://docs.synthesized.io/llms.txt).

Mask the sensitive data in a folder of documents with `docanon`, Synthesized’s command-line tool for unstructured data. Masked files are delivered in their original formats.

> **NOTE**
> **Available on special request.** Document masking isn’t part of the standard TDK distribution. [Contact Synthesized](https://www.synthesized.io/contact-sales) to get it.

For how unstructured data is processed, see [Unstructured Data](https://docs.synthesized.io/tdk/latest/user_guide/030_core_concepts/unstructured_data).

## Before you start

- The `docanon` tool from Synthesized, and Java 17 on the machine that runs it. It doesn’t need a license key, and it doesn’t connect to Governor.
- A folder with the documents to mask. It can mix formats; see [Supported formats](#_supported_formats).
- Optional: a local language model with an OpenAI-compatible endpoint. Without one, names, addresses and organizations written in free text aren’t detected.

## Mask a folder of documents

1. Create a policy file:
   
   ```shell
   docanon policy init > policy.yaml
   ```
2. In `policy.yaml`, choose what happens to each type of sensitive data. See [What it finds](#_what_it_finds) and [Policy modes](#_policy_modes).
3. Run it on the folder:
   
   ```shell
   docanon run ./documents --policy policy.yaml --out /path/to/new-run-folder --no-model
   ```
   
   `--out` must be a folder that doesn’t exist yet. To use a local language model, replace `--no-model` with `--model <url>`.

Start with a small, representative set of files. Review the results, adjust the policy, and only then run the whole collection.

## Check the result

 
| In the output folder | What it holds |
| --- | --- |
| 
`<out>/delivery/`

 | 

The masked files, in their original formats

 |
| 

`<out>/package/delivery.zip`

 | 

The same files, packaged as one archive

 |
| 

`<out>/results/`

 | 

The results of the run

 |

The exit code tells you whether the files were delivered:

 
| Code | Meaning |
| --- | --- |
| 
`0`

 | 

The files were delivered.

 |
| 

`1`

 | 

An error in the launch, the policy or the model.

 |
| 

`2`

 | 

The tool’s own check blocked the delivery.

 |

Open the delivered files in the applications that will use them, and check that the sensitive values you expect are replaced. A delivery is only as complete as detection: values that weren’t detected stay in the files.

## What it finds

`docanon` detects about 50 types of sensitive data, in seven groups: personal, organization, health, finance, secrets, business and system data.

## Policy modes

In the policy, each type’s `mode:` sets what happens to it. Modes are case-sensitive: write them in capitals.

 
| Mode | What it does |
| --- | --- |
| 
`KEEP`

 | 

Leaves the value as it is

 |
| 

`TOKENIZE`

 | 

Replaces the value with a token. Tokens stay the same across runs.

 |
| 

`REDACT`

 | 

Replaces the value with a placeholder, set by `tokens.redact`. The default is `"[{TYPE}]"`.

 |
| 

`TRANSFORM`

 | 

Replaces the value with one made by a TDK generator, configured in `params`

 |

## Supported formats

 
| Kind | Formats |
| --- | --- |
| 
Office documents

 | 

Word, Excel and PowerPoint, including macro-enabled files and templates

 |
| 

PDF

 | 

Including scanned PDFs, which are read with OCR

 |
| 

Images

 | 

PNG, JPG, BMP, GIF and TIFF

 |
| 

Email and calendars

 | 

EML, MBOX and ICS

 |
| 

Web and exports

 | 

HTML, SVG, JSON and Slack exports

 |
| 

Text

 | 

CSV, TSV, plain text, Markdown and log files

 |
| 

Archives

 | 

ZIP

 |

## Related pages

- [Unstructured Data](https://docs.synthesized.io/tdk/latest/user_guide/030_core_concepts/unstructured_data)
- [PII Detection & Protection](https://docs.synthesized.io/tdk/latest/user_guide/030_core_concepts/pii)
- [Mask a Database](https://docs.synthesized.io/tdk/latest/user_guide/020_guides/masking_data/masking)
