AWS s3 python sdk code examples
Yet another easy-to-understand, easy-to-use aws s3 python sdk code examples.
github地址:https://github.com/garyelephant/aws-s3-python-sdk-examples.
"""
Yet another s3 python sdk example.
based on boto 2.27.0
""" import time
import os
import urllib import boto.s3.connection
import boto.s3.key def test():
print '--- running AWS s3 examples ---'
c = boto.s3.connection.S3Connection('<YOUR_AWS_ACCESS_KEY>', '<YOUR_AWS_SECRET_KEY>') print 'original bucket number:', len(c.get_all_buckets()) bucket_name = 'yet.another.s3.example.code'
print 'creating a bucket:', bucket_name
try:
bucket = c.create_bucket(bucket_name)
except boto.exception.S3CreateError as e:
print ' ' * 4, 'error occured:'
print ' ' * 8, 'http status code:', e.status
print ' ' * 8, 'reason:', e.reason
print ' ' * 8, 'body:', e.body
return test_bucket_name = 'no.existence.yet.another.s3.example.code'
print 'if you just want to know whether the bucket(\'%s\') exists or not' % (test_bucket_name,), \
'and don\'t want to get this bucket'
try:
test_bucket = c.head_bucket(test_bucket_name)
except boto.exception.S3ResponseError as e:
if e.status == 403 and e.reason == 'Forbidden':
print ' ' * 4, 'the bucket(\'%s\') exists but you don\'t have the permission.' % (test_bucket_name,)
elif e.status == 404 and e.reason == 'Not Found':
print ' ' * 4, 'the bucket(\'%s\') doesn\'t exist.' % (test_bucket_name,) print 'or use lookup() instead of head_bucket() to do the same thing.', \
'it will return None if the bucket does not exist instead of throwing an exception.'
test_bucket = c.lookup(test_bucket_name)
if test_bucket is None:
print ' ' * 4, 'the bucket(\'%s\') doesn\'t exist.' % (test_bucket_name,) print 'now you can get the bucket(\'%s\')' % (bucket_name,)
bucket = c.get_bucket(bucket_name) print 'add some objects to bucket ', bucket_name
keys = ['sample.txt', 'notes/2006/January/sample.txt', 'notes/2006/February/sample2.txt',\
'notes/2006/February/sample3.txt', 'notes/2006/February/sample4.txt', 'notes/2006/sample5.txt']
print ' ' * 4, 'these key names are:'
for name in keys:
print ' ' * 8, name filename = './_test_dir/sample.txt'
print ' ' * 4, 'you can contents of object(\'%s\') from filename(\'%s\')' % (keys[0], filename,)
key = boto.s3.key.Key(bucket, keys[0])
bytes_written = key.set_contents_from_filename(filename)
assert bytes_written == os.path.getsize(filename), ' error occured:broken file' print ' ' * 4, 'or set contents of object(\'%s\') by opened file object' % (keys[1],)
fp = open(filename, 'r')
key = boto.s3.key.Key(bucket, keys[1])
bytes_written = key.set_contents_from_file(fp)
assert bytes_written == os.path.getsize(filename), ' error occured:broken file' print ' ' * 4, 'you can also set contents the remaining key objects from string'
for name in keys[2:]:
print ' ' * 8, 'key:', name
key = boto.s3.key.Key(bucket, name)
s = 'This is the content of %s ' % (name,)
key.set_contents_from_string(s)
print ' ' * 8, '..contents:', key.get_contents_as_string()
# use get_contents_to_filename() to save contents to a specific file in the filesystem. #print 'You have %d objects in bucket %s' % () print 'list all objects added into \'%s\' bucket' % (bucket_name,)
objs = bucket.list()
for key in objs:
print ' ' * 4, key.name p = 'notes/2006/'
print 'list objects start with \'%s\'' % (p,)
objs = bucket.list(prefix = p)
for key in objs:
print ' ' * 4, key.name print 'list objects or key prefixs like \'%s/*\', something like what\'s in the top of \'%s\' folder ?' % (p, p,)
objs = bucket.list(prefix = p, delimiter = '/')
for key in objs:
print ' ' * 4, key.name keys_per_page = 4
print 'manually handle the results paging from s3,', ' number of keys per page:', keys_per_page
print ' ' * 4, 'get page 1'
objs = bucket.get_all_keys(max_keys = keys_per_page)
for key in objs:
print ' ' * 8, key.name print ' ' * 4, 'get page 2'
last_key_name = objs[-1].name #last key of last page is the marker to retrive next page.
objs = bucket.get_all_keys(max_keys = keys_per_page, marker = last_key_name)
for key in objs:
print ' ' * 8, key.name
"""
get_all_keys() a lower-level method for listing contents of a bucket.
This closely models the actual S3 API and requires you to manually handle the paging of results.
For a higher-level method that handles the details of paging for you, you can use the list() method.
""" print 'you must delete all objects in the bucket \'%s\' before delete this bucket' % (bucket_name, )
print ' ' * 4, 'you can delete objects one by one'
bucket.delete_key(keys[0])
print ' ' * 4, 'or you can delete multiple objects using a single HTTP request with delete_keys().'
bucket.delete_keys(keys[1:]) print 'now you can delete the bucket \'%s\'' % (bucket_name,)
c.delete_bucket(bucket) #references:
# [1] http://docs.pythonboto.org/
# [2] amazon s3 api references if __name__ == '__main__':
test()
转载本文请注明作者和出处[Gary的影响力]http://garyelephant.me,请勿用于不论什么商业用途!
Author: Gary Gao( garygaowork[at]gmail.com) 关注互联网、分布式、高性能、NoSQL、自己主动化、软件团队
支持我的工作: https://me.alipay.com/garygao
AWS s3 python sdk code examples的更多相关文章
- aws s3 python sdk
http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.get_object abort_multipar ...
- Edit Static Web File Http Header Metadata of AWS S3 from SDK | SDK编程方式编辑储存在AWS S3中Web类文件的Http Header元数据
1.Motivation | 起因 A requirement from the product department requires download image from AWS S3 buck ...
- Python使用boto3操作AWS S3中踩过的坑
最近在AWS上开发部署应用. 看了这篇关于AWS中国区填坑的文章,结合自己使用AWS的经历,补充两个我自己填的坑. http://www.jianshu.com/p/0d0fd39a40c9?utm_ ...
- ceph rgw s3 java sdk 上传大文件分批的方法
Using the AWS Java SDK for Multipart Upload (High-Level API) Topics Upload a File Abort Multipart Up ...
- Ceph RGW服务 使用s3 java sdk 分片文件上传API 报‘SignatureDoesNotMatch’ 异常的定位及规避方案
import java.io.File; import com.amazonaws.AmazonClientException; import com.amazonaws.auth.profile ...
- aws s3文件上传设置accesskey、secretkey、sessiontoken
背景: 最近跟进的项目会封装aws S3资源管理细节,对外提供获取文件上传凭证的API,业务方使用获取到的凭证信息直接请求aws进行文件上传.因此,测试过程需要验证S3文件上传的有效性.aws官网有提 ...
- Amazon AWS S3 操作手册
Install the SDK The recommended way to use the AWS SDK for Java in your project is to consume it fro ...
- AWS S3 对象存储服务
虽然亚马逊云非常牛逼,虽然亚马逊云财大气粗,虽然亚马逊用的人也非常多,可是这个文档我简直无法接受,特别是客服,令人发指的回复速度,瞬间让人无语,可是毕竟牛逼.忍了,躺一次坑而已 1.图片上传 1.1 ...
- AWS S3使用小结
使用场景一:储存网站的图片,并能被任何人访问 1. 创建一个bucket,名字与需要绑定的域名一致. 例如,根域名是mysite.com,希望把所有图片放在pic.mysite.com下面,访问的时候 ...
随机推荐
- spring mvc 使用及json 日期转换解决方案
http://blog.csdn.net/z69183787/article/details/40375479
- java super 隐式参数
第41集 所有构造器里,第一句话就是super() (隐式的,系统自动执行) 鸟构造器调用动物构造器,动物构造器调用object构造器. (系统默认的) tostring() 方法 ...
- M-JPEG和MPEG-4的区别 M-JPEG VS MPEG
http://blog.sina.com.cn/s/blog_4b357b300100gre9.html M-JPEG VS MPEG http://blog.csdn.net/bluesky_sun ...
- javaweb学习总结(四十)——编写自己的JDBC框架
一.元数据介绍 元数据指的是"数据库"."表"."列"的定义信息. 1.1.DataBaseMetaData元数据 Connection.g ...
- Form - CHECKBOX全选功能
FORM BUILDER开发,遇到这样一个需求: 添加一个CHECKBOX完成全选功能,红框为新添加的CHECKBOX(如图示) Try to use APP_RECORD.FOR_ALL_RECOR ...
- uc浏览器qq浏览器广告过滤
- 按CTRL,SHIFT,ALT等键扩展easyui的datagrid多选实现
//------------------------------------------------------------------------------- // 当然页面文件中还需要引入的文件 ...
- [WebKit]浏览器的加载与页面性能优化
非常棒.非常系统的一份资料,值得阅读! 原文来自百度泛用户体验. 作者:nwind 本文将探讨浏览器渲染的loading过程,主要有2个目的: 了解浏览器在loading过程中的实现细节,具体都做了什 ...
- Android批量插入数据到SQLite数据库
Android中在sqlite插入数据的时候默认一条语句就是一个事务,因此如果存在上万条数据插入的话,那就需要执行上万次插入操作,操作速度可想而知.因此在Android中插入数据时,使用批量插入的方式 ...
- Java线程之Phaser
Phaser是一个灵活的线程同步工具,他包含了CyclicBarrier和CountDownLatch的相关功能 首先,来看一下如何用Phaser替代CountDownLatch.对于CountDow ...