Java通过aws java sdk在AWS S3上进行操作客户端
有两种方式可以安装aws java sdk:
方式一:可以在这个网站上下载适用于 Java 的 AWS 开发工具包
https://aws.amazon.com/cn/sdk-for-java/
点击网页右侧的AWS Toolkit for eclipse进行安装,之后打开eclipse会提示下载aws java sdk,下载完之后就可以新建AWS Project了。
这样下载的好处是安装完后新建AWS Project的时候会给出一些例程,我下面的代码AWSClient类中有很多方法都是改自这些例程的。
方式二:通过maven dependency下载。
可以在下面这个网站获得最新的maven dependency:
http://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk
我这里使用的maven dependency是:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.10.26</version>
</dependency>
采用default方法连接aws通过这种方法一开始是行不通的,原因是aws java sdk会在用户目录下查找一个文件,这个文件对应的位置是:
~/.aws/credentials
credentials文件里面记录的登陆的 key id 和 secret access key:
[default]
aws_access_key_id=YourAccessKeyId
aws_secret_access_key=YourSecretAccessKey
使用maven的方法不会帮你创建credentials文件,所以你需要手动创建文件并把上述内容添加进去,YourAccessKeyId 和 YourSecretAccessKey 改成你自己的就行。
另一种采用显式的输入 access key id 和 secret access key 的方法可以不用配置~/.aws/credentials这个文件。
AWSClient类中的方法简述:
init():通过default方法连接aws;
init_with_key():通过在代码中包含 access key id 和 secret access key 连接 aws;
createBucket(String bucketName):用于创建一个名为bucketName的s3 bucket;
private static void listObjects(String bucketName):列出名为bucketName的bucket里面的所有object(object相当于所有文件和文件夹);
isObjectExit(String bucketName, String key):判断名为bucketName的bucket里面是否有一个名为key的object;
createSampleFile(String bucketName, String filename):在本地创建一个名为filename的文件,输入一些信息后,将其传到名为bucketName的s3 bucket里面去;
showContentOfAnObject(String bucketName, String key):输出"s3://"+bucketname+"/"key对应的object对应的信息;
showContentOfAnGzipObject(String bucketName, String key):输出"s3://"+bucketname+"/"key对应的object对应的信息(这个object对应的文件时gzip格式的);
listBuckets():列出我的s3上所有的bucket的名字;
deleteBucket(String bucketName):删除一个名为bucketName的bucket;
deleteObjectsWithPrefix(String bucketName, String prefix):删除名为bucketName的bucket里面所有key的前缀和prefix匹配的object;
uploadFileToBucket(String path, String bucketName):上传一个本地文件(对应位置为path)上传到名为bucketName的bucket;
createFolder(String bucketName, String folderName):在aws上创建一个目录(这个函数上网查了两天才找到方法,很重要!)。
代码:
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.util.zip.GZIPInputStream; import com.amazonaws.AmazonClientException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.Bucket;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.ObjectListing;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.model.S3ObjectSummary;
import com.amazonaws.services.s3.transfer.TransferManager;
import com.amazonaws.services.s3.transfer.Upload; public class AWSClient {
static AmazonS3 s3;
static TransferManager tx; private static String AWS_ACCESS_KEY = "YOUR_AWS_ACCESS_KEY";
private static String AWS_SECRET_KEY = "YOUR_AWS_SECRET_KEY"; static final String bucketName = "YOUR_BUCKET_NAME"; private static void init() throws Exception {
AWSCredentials credentials = null;
try {
credentials = new ProfileCredentialsProvider("default").getCredentials();
} catch (Exception e) {
throw new AmazonClientException("Exception on credentials.", e);
}
s3 = new AmazonS3Client(credentials);
//Region usWest2 = Region.getRegion(Regions.US_WEST_2);
//s3.setRegion(usWest2);
tx = new TransferManager(s3);
} private static void init_with_key() throws Exception {
AWSCredentials credentials = null;
credentials = new BasicAWSCredentials(AWS_ACCESS_KEY, AWS_SECRET_KEY);
s3 = new AmazonS3Client(credentials);
//Region usWest2 = Region.getRegion(Regions.US_WEST_2);
//s3.setRegion(usWest2);
tx = new TransferManager(s3);
} private static void createBucket(String bucketName) {
if(s3.doesBucketExist(bucketName) == true) {
System.out.println(bucketName + " already exist!");
return;
}
System.out.println("creating " + bucketName + " ...");
s3.createBucket(bucketName);
System.out.println(bucketName + " has been created!");
} private static void listObjects(String bucketName) {
System.out.println("Listing objects of " + bucketName);
ObjectListing objectListing = s3.listObjects(bucketName);
int objectNum = 0;
for(S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
System.out.println(" - " + objectSummary.getKey());
objectNum ++;
}
System.out.println("total " + objectNum + " object(s).");
} private static boolean isObjectExit(String bucketName, String key) {
int len = key.length();
ObjectListing objectListing = s3.listObjects(bucketName);
String s = new String();
for(S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
s = objectSummary.getKey();
int slen = s.length();
if(len == slen) {
int i;
for(i=0;i<len;i++) if(s.charAt(i) != key.charAt(i)) break;
if(i == len) return true;
}
}
return false;
} private static void createSampleFile(String bucketName, String filename) throws IOException {
if(isObjectExit(bucketName, filename) == true) {
System.out.println(filename +" already exists in " + bucketName + "!");
return;
}
System.out.println("creating file " + filename);
File file = new File(filename);
file.deleteOnExit(); Writer writer = new OutputStreamWriter(new FileOutputStream(file));
writer.write("abcdefghijklmnopqrstuvwxyz\n");
writer.write("01234567890112345678901234\n");
writer.write("!@#$%^&*()-=[]{};':',.<>/?\n");
writer.write("01234567890112345678901234\n");
writer.write("abcdefghijklmnopqrstuvwxyz\n");
writer.close(); s3.putObject(bucketName, filename, file);
System.out.println("create sample file " + filename + " succeed!");
} private static void showContentOfAnObject(String bucketName, String key) {
S3Object object = s3.getObject(new GetObjectRequest(bucketName, key));
InputStream input = object.getObjectContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
try {
while (true) {
String line = reader.readLine();
if (line == null) break;
System.out.println(" " + line);
}
System.out.println();
} catch (IOException e) {
e.printStackTrace();
}
} private static void showContentOfAnGzipObject(String bucketName, String key) {
try {
S3Object object = s3.getObject(new GetObjectRequest(bucketName, key));
InputStream input = new GZIPInputStream(object.getObjectContent());
BufferedReader reader = new BufferedReader(new InputStreamReader(input)); while (true) {
String line = reader.readLine();
if (line == null) break;
System.out.println(" " + line);
}
System.out.println();
} catch (IOException e) {
e.printStackTrace();
}
} private static void listBuckets() {
System.out.println("Listing buckets");
int bucketNum = 0;
for(Bucket bucket : s3.listBuckets()) {
System.out.println(" - " + bucket.getName());
bucketNum ++;
}
System.out.println("total " + bucketNum + " bucket(s).");
} private static void deleteBucket(String bucketName) {
if(s3.doesBucketExist(bucketName) == false) {
System.out.println(bucketName + " does not exists!");
return;
}
System.out.println("deleting " + bucketName + " ...");
ObjectListing objectListing = s3.listObjects(bucketName);
for(S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
String key = objectSummary.getKey();
s3.deleteObject(bucketName, key);
}
s3.deleteBucket(bucketName);
System.out.println(bucketName + " has been deleted!");
} private static void deleteObjectsWithPrefix(String bucketName, String prefix) {
if(s3.doesBucketExist(bucketName) == false) {
System.out.println(bucketName + " does not exists!");
return;
}
System.out.println("deleting " + prefix +"* in " + bucketName + " ...");
int pre_len = prefix.length();
ObjectListing objectListing = s3.listObjects(bucketName);
for(S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
String key = objectSummary.getKey();
int len = key.length();
if(len < pre_len) continue;
int i;
for(i=0;i<pre_len;i++)
if(key.charAt(i) != prefix.charAt(i))
break;
if(i < pre_len) continue;
s3.deleteObject(bucketName, key);
}
System.out.println("All " + prefix + "* deleted!");
} private static void uploadFileToBucket(String path, String bucketName) {
File fileToUpload = new File(path);
if(fileToUpload.exists() == false) {
System.out.println(path + " not exists!");
return;
}
PutObjectRequest request = new PutObjectRequest(bucketName, fileToUpload.getName(), fileToUpload);
Upload upload = tx.upload(request);
while((int)upload.getProgress().getPercentTransferred() < 100) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
System.out.println(path + " upload succeed!");
} private static void createFolder(String bucketName, String folderName) {
// Create metadata for my folder & set content-length to 0
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(0);
// Create empty content
InputStream emptyContent = new ByteArrayInputStream(new byte[0]);
// Create a PutObjectRequest passing the foldername suffixed by /
PutObjectRequest putObjectRequest =
new PutObjectRequest(bucketName, folderName+"/",
emptyContent, metadata);
// Send request to S3 to create folder
s3.putObject(putObjectRequest);
} public static void main(String[] args) throws Exception { init(); showContentOfAnGzipObject(bucketName, "testfile.gz"); System.exit(0);
}
}
Java通过aws java sdk在AWS S3上进行操作客户端的更多相关文章
- boto3--通过Python的SDK连接aws
通过Python的SDK连接aws 参考: https://aws.amazon.com/cn/developers/getting-started/python/ aws上生成访问密钥 ID 和私有 ...
- 往aws中的s3上传数据
在官网下载对应的sdk http://aws.amazon.com/cn/sdk-for-net/ 然后再aws上为s3上的bucket生成密钥对 access_key和secret_key 调用对应 ...
- Java版 人脸识别SDK demo
虹软人脸识别SDK之Java版,支持SDK 1.1+,以及当前最新版本2.0,滴滴,抓紧上车! 前言 由于业务需求,最近跟人脸识别杠上了,本以为虹软提供的SDK是那种面向开发语言的,结果是一堆dll· ...
- Java版 人脸识别SDK dem
虹软人脸识别SDK之Java版,支持SDK 1.1+,以及2.0版本,滴滴,抓紧上车! 前言由于业务需求,最近跟人脸识别杠上了,本以为虹软提供的SDK是那种面向开发语言的,结果是一堆dll······ ...
- Java离线人脸识别SDK 支持arcface 2.0 最新版
虹软人脸识别SDK之Java版,支持SDK 1.1+,以及当前最新版本2.0,滴滴,抓紧上车! JDK SDK Win release license status 前言 由于业务需求,最近跟人脸识别 ...
- AWS中国区使用https访问部署在S3上的网站
问题描述 最近一个项目需要通过https的方式访问部署在S3上的网站,通过搜索引擎找到一篇文章,可以在AWS Global实现整个过程.但是目前AWS中国区有限制,CloudFront不能使用AWS ...
- AWS Python SDK boto3中的基本概念与使用方法
最近在用boto3编写AWS的lamda函数,学习到了boto3中的一些基本概念与使用方法.在此进行总结. 1. boto3提供了两个级别的接口来访问AWS服务:High Level的Resource ...
- [Java面试二]Java基础知识精华部分.
一:java概述(快速浏览): 1991 年Sun公司的James Gosling等人开始开发名称为 Oak 的语言,希望用于控制嵌入在有线电视交换盒.PDA等的微处理器: 1994年将Oak语言更名 ...
- java的几个版本以及jre,jdk等概念——【转载】JDK、Java SE、Java EE、Java ME我该选
我们平时使用的一些软件,有一部分需要Java环境的支持,但是SUN那么多的产品,让人眼花缭乱的版本号,前看后看都差不多的缩写,让我们选择起来的时候常常望而却步,只好跟着感觉走.所以下面我要介绍的就是那 ...
随机推荐
- IOS-UISearchBar
UISearchBar控件 最近用到搜索功能.于是,经过不断的研究,终于,有点懂了. 那就来总结一下吧,好记性不如烂笔头! 搜索,无疑可以使用UISearchBar控件! 那就先了解一下UISea ...
- 【Python】装饰器理解
以下文章转载自:点这里 关于装饰器相关的帖子记录在这里: 廖雪峰, thy专栏, stackflow Python的函数是对象 简单的例子: def shout(word="yes" ...
- 我要复习python啦(一)
一.变量 那些曾经怎么也看不懂的东西,突然有一天就懂了.这就是复习的力量吗? 1 变量的赋值 a = 10 做了上面的图所描述的事情 1)开辟一块内存,创建一个值为10的整数 2)创建一个a的标记 3 ...
- vue请求本地json数据
1.下载vue-resource插件 cnpm install vue-resource 1.2全局引入vue-resource: 在main.js import VueResource from ' ...
- PHP工作笔记:离线执行php任务
直接上代码,主要函数 ignore_user_abort(true);这个函数忽略了终端被关闭(打开的网页被关闭),后面 getfiles()这函数是执行采集任务的自定义函数,后面又配置了下路径打开写 ...
- SQL 动态PIVOT查询
DECLARE @sql_str VARCHAR(8000)DECLARE @sql_col VARCHAR(8000) SELECT @sql_col = ISNULL(@sql_col + ',' ...
- jQuery选项卡wdScrollTab
实例Demo 运行一下 参数说明 Config active Number Active tab index. Base on 0. autoResizable Boolean Whether ...
- 怎么解决安装SqlServer2008总是提示Restart computer as failed
安装SqlServer2008总是提示Restart computer as failed,重启电脑依然报这个錯.解决办法是在cmd窗口中使用命令setup.exe /SkipRules=Reboot ...
- 【剑指offer】包含min函数的栈,C++实现
博客文章索引地址 博客文章中代码的github地址 1.题目 定义栈的数据结构,请在该类型中实现一个能够得到栈最小元素的min函数.在该栈中调用push.pop.top.min的时间复杂度都是o(1) ...
- nginx在使用proxy_pass的情况下开启error_page
error_page用于指定特定错误发生时要显示的url,但是如果请求经proxy_pass处理后,如何使error_page对upstream产生的错误进行处理呢? 方法很简单. 保持之前的erro ...