# How to process (extremely) large EDI files with Stedi

> You can use Stedi to process EDI files of any size and split large transactions into smaller payloads for easier downstream ingestion.

Published Nov 14, 2023

Category: EDI

Large EDI files are common across many industries, including healthcare (such as 834 Benefit Enrollments), logistics (210 Motor Carrier Freight Details and Invoices), and retail (846 Inventory Advice). Unlike most other EDI solutions, Stedi has virtually no file size limitations and can process EDI files that are gigabytes in size.

However, large files that have been translated into JSON still pose significant challenges for downstream applications. The translation ratio of an EDI file to JSON is typically 1:10, which means that a large EDI file can easily produce a multi-gigabyte payload that is difficult for downstream applications to receive and ingest.

To solve this problem, we’re excited to introduce Fragments, a new feature in our [Large File Processing module](https://www.stedi.com/pricing). Fragments allow you to automatically split processed transactions into smaller chunks for easier downstream ingestion.

## Use fragments to split large transactions

Large files are often the result of transactions containing many repeated loops or segments. A healthcare provider may send an [834 Healthcare Benefit Enrollment](https://portal.stedi.com/app/guides/view/hipaa/benefit-enrollment-and-maintenance-x220a1/01GRYB6D6RAWSG8ATBD6GXM13C) file containing tens of thousands of individual members – each one in the `INS` segment – or a brand may send an [846 Inventory Inquiry/Advice](https://www.stedi.com/edi/x12/transaction-set/846) file containing millions of SKUs – each one in the `LIN` segment.

With Fragments, you can use repeated segments like these to split the transaction. For example, when you enable fragments on the `INS` loop in an 834, Stedi emits batches of `INS` loops instead of a single giant JSON payload.

For each fragment batch, Stedi emits a `fragment.processed.v2` event, which you can use to automatically send Destination webhooks to your API. Fragment processed events contain details about the original transaction as well as the actual fragment payload itself.

An 834 Benefit Enrollment split into fragments

## Configure fragments

You can configure fragments for any transaction in minutes without writing any code. Before you begin, create a partnership in Stedi between you and your trading partner.

### Set up the Stedi guide

First, you need to enable fragments in the Stedi guide for the transaction.

Stedi guides are a machine-readable format for the trading partner EDI specifications you typically receive as PDF or CSV files. The [Stedi Network](https://www.stedi.com/edi/network) has pre-built guides for hundreds of popular partners that you can import into your account and use in your integrations for free.

You can enable fragments on one repeated segment within each guide. To enable fragments, open the guide in your Stedi account, click the segment, toggle **Set as fragment** to `ON`, and click **Publish changes**.

Enable fragments in a Stedi guide

### Create a transaction setting

Next, you need to create an inbound transaction setting. Transaction settings define the EDI transactions you plan to exchange with your trading partner and the guide Stedi should use to validate data.

When creating the transaction setting, choose the guide you previously configured with fragments and then toggle **Enable fragments** to `ON`.

Create an inbound transaction setting with fragment-enabled guide

### Send fragments to your API

Finally, you can automatically deliver fragments to your API. To do this, add an event binding for `fragment.processed.v2` events to a Destination webhook.

Set up a destination webhook for fragment events

You can also use the Get Fragment API to manually retrieve fragments for processed transactions as needed.

```shell
  curl --request GET \
    --url https://core.us.stedi.com/2023-08-01/transactions/{transactionId}/fragments/{fragmentIndex} \
    --header 'Authorization: Key {STEDI_API_KEY}'
```

## Get started on Stedi

Stedi allows you to seamlessly process large EDI files of virtually any size. You can also use fragments to efficiently manage large transactions and reduce the development work required to integrate with new trading partners. Check out the Fragments documentation for complete details.

To get started building EDI integrations on Stedi, [book a demo](https://www.stedi.com/contact) with our team.