What is a bucket?
Updated October 13, 2022
A bucket is a place where you can store large data objects. The object has a key, which you can use to refer to the object. In practice, most people think about objects and keys as files and file paths.
All data you store in a bucket is encrypted at rest. The data will persist in your bucket, unless you actively delete it.
You don’t have to worry about storage space when working with buckets; you’ll never run out. The same goes for bandwith: you can upload and download data as much as you need. You do pay for storage and bandwith, but you only pay for what you actually use.
- See also: Pricing
A bucket doesn’t have the concept of directories. All objects are stored in a flat collection.
The key of an object can contain any character, including the
/. This is often used to simulate a directory structure. For example, if the key is
/documents/instructions, you can pretend the object is a file with the name
instructions inside the directory
documents. Stedi SFTP uses this trick to simulate a directory structure.
This does lead to subtle differences between directory structures and buckets. For example, you can’t have an empty directory in a bucket. You can solve this by creating an object with the directory name as the key, but then you need to make sure you don’t treat the directory as if it is a file.
A bucket often serves as a source of input for another Stedi product, or as a place to store its output. For example, an EDI file stored in a bucket is passed to EDI Core and the resulting JSON gets stored in a bucket.
This requires a bit of coordination. You can write a function that triggers whenever a new EDI file comes in. It passes the file on to EDI Core and waits for output. It then writes the output to a bucket.
To assist with building event-driven workflows, Buckets supports sending notifications when objects in them change. You can connect a function to receive these events and it will be invoked whenever something happens to an object in the bucket. You should take extra care when creating new objects in response to a bucket event. If your event handler function writes to the same bucket as it is listening to, this can cause a loop where it triggers more executions of itself. In order to avoid this, use a separate bucket for storing the output data. If you need to store the output in the same bucket, you can use different object prefix paths and explicitly check that the path of an incoming event requires a response in your event handler code.
Buckets belong to a Stedi-account. Every member of the account has access to all buckets in the account and to all objects inside those buckets. When you write code that accesses Stedi Buckets, you provide an API key to prove that your code is allowed to access all buckets and objects in your account.
If you want to give people outside your account access to some objects but not all of them, then you can use Stedi SFTP.