Introducing large file support: Process EDI files up to 500MB (and growing)

EDI formats are designed to use as little space as possible, but most companies still eventually need to process files that are tens or hundreds of megabytes. Unfortunately, large EDI files are a notorious pain point for EDI systems, as most cannot handle payloads over a few dozen megabytes and sometimes take hours or days to work through larger payloads.

Large file support in Stedi Core

Stedi Core is an EDI integration hub that translates inbound EDI to JSON and generates outbound EDI from JSON—for any of the 300+ transaction sets across every X12 version release.

If your integration requires parsing EDI files or transaction sets beyond Core’s current upper limits, Stedi can automatically split files and process them in manageable pieces. With this approach, you can use Core to process transaction sets and files of virtually any size.

Is there a size limit for EDI files?

No. Neither the X12 nor EDIFACT standards explicitly specify an upper limit for file size.

Even though there are no explicit limits, your EDI system should list upper bounds for the file size and individual transaction set size it can handle successfully. However, it can be hard to predict whether files approaching the upper limit will produce failures.

This is because the upper limit of what an EDI system can successfully process often depends not on the file size but on the size and complexity of the largest EDI transaction set within the file. Sometimes files are 500MB, but no single transaction is larger than 1MB. In other cases, a 60MB file contains a single transaction set that's 50MB. Depending on how the system is designed, it may have no problem parsing the former but a lot of trouble parsing the latter.

Large EDI file examples

There are many scenarios in which it’s important to have an EDI solution that can handle large files.

• Many transaction sets within a single file: Your trading partner may send batches of business data instead of generating a new EDI file for every transaction set. For example, a carrier may batch multiple 210 Motor Carrier Freight Details and Invoice transactions into a single file at regular intervals.
• Very large transaction sets: Some transaction sets can contain many repeated segments or loops. For example, a healthcare provider could send an 837 HealthCare Claim that contains multiple, separate insurance claims, or a brand may send an 846 Inventory Inquiry/Advice file containing millions of SKUs.
• A lot of data in one or more elements: For example, a single 275 Patient Information transaction containing x-ray images could produce a very large file.

Try Stedi Core

Stedi Core seamlessly processes large EDI files and allows you to search, inspect, and debug all real-time transaction data.