Python Tutorial: Uploading and retrieving files

in the last lesson we learned how to list create and delete buckets now it's time to put stuff in them let's take a look at how objects work the files in s3 buckets are called objects an object can be anything an image a video file CSV or a log file managing objects is a key component of many data pipelines objects and buckets in s3 work somewhat like files and folders on our desktop each bucket has a name objects names are called keys a bucket name is just a name an object's key is the full path of the object from the buckets route a buckets name is unique in all of us 3 an object's key is unique in the bucket a bucket contains many objects but an object can only belong to one bucket first we create the client and assign it to the s3 variable now we can perform operations on our objects and buckets let's upload an object into a bucket we upload the file using the clients upload file method the file name is the local file path bucket parameter takes the name of the bucket we are uploading to he is what we want to name the object in s3 we are not capturing the return from this method in a variable the method doesn't return anything if there's an error it will throw an exception woohoo our file is now in s3 I've uploaded a few more objects for us to play with let's list them with Bato tree call the clients list objects method passing GID requests for bucket name optionally we can limit the response to two objects with the max keys argument if we omit it s3 will return up to a thousand objects in our bucket if they exist another way to limit the response is to use the optional prefix argument passing it will limit the response to objects that start with the string we provide the response dictionary contains the contents key this key contains a list of objects and their info each object dictionary is returned with a key a modified date and the object size and bytes if we want to know these things about a single object we can use the clients head object method passing the bucket name and object key notice that because we are only working with one object there is no contents dictionary the metadata is directly in the response dictionary to download a file we use the clients download file method we pass the file name or the local path we want the file to download - then we specify the bucket and key of the object we want to download sometimes an object is outlived its usefulness and needs to be deleted use the clients delete object method passing the bucket name and object key to delete the object in this lesson we learn that buckets are like folders and objects are like files within them we learn to create the client before we can do anything else we learn how to upload files to a bucket how to list objects in a bucket how to head object or get object metadata how to download a file from a bucket and finally how to delete an object let's help Sam continue working on a her pipeline