The large file processing module allows you to translate single EDI files that are more than 50MB in size.
Since the translation ratio of an EDI file to JSON typically approaches 1:10, a translated JSON artifact will be multiple times the size of the original EDI file. To avoid overwhelming downstream systems, you can use fragments to split translated transactions from Stedi into smaller, more manageable chunks.
How fragments work
Large files are often the result of transaction sets containing many repeated loops or segments. For example, a healthcare provider may send an 837 Health Care Claim containing many individual claims, or a brand may send an 846 Inventory Inquiry/Advice file containing millions of SKUs.
You can enable fragments for one repeated EDI segment in each transaction set. Once enabled, Stedi splits the translated transaction payload into chunks based on the selected segment.
For example, if you enable fragments on the
LIN loop in an 846 Inventory Inquiry/Advice, Stedi will fragment a translated 846 file and emit the following events:
transaction.processed.v2event for each unique transaction set that was included in the original file.
- One or more
fragment.processed.v2events containing several iterations of the
LINloop (inventory items)
When you receive a
transaction.processed.v2 event through a Destination webhook, the transaction payload contains an empty array in the position of the fragmented segment. Adjacent to that segment, the payload also includes an
x-stedi-fragment-detail section with general information about the fragments that Stedi created.
When you receive
fragment.processed.v2 event through a Destination webhook, the payload contains the fragment as well as general information about the transaction.
You must enable fragments within the transaction’s guide and the corresponding inbound transaction setting.
To enable fragments within a Stedi guide:
- Go to the Guides page and edit the guide you want to update.
- Click the node (segment) in the guide that you want to use to split the file. The node must be a looping data structure, such as an HL loop containing inventory items.
- Toggle Set as fragment to
- Publish the changes to your guide.
Once you have enabled fragments on the Stedi guide, create an inbound transaction setting with the guide and toggle Enable fragments to
ON. You can use the default batch size (800 segments) or set a custom batch size (up to 50,000 segments).
Within this partnership, Stedi will now split inbound transactions of this type into fragments.
To view fragments, go to the Transactions page and click a transaction. On the transaction’s details page, you can review each fragment’s metadata and output JSON.
You have two options for ingesting fragments into your downstream system:
- Push-based: Configure a destination webhook for
fragment.processed.v2events and ingest fragments as Stedi emits these events.
- Pull-based: Configure a destination webhook for
transaction.processed.v2events only. After you receive a
transaction.processed.v2event, you can use the API to retrieve associated fragments in batches according to your system’s requirements.