kubernetes生产实践之redis-cluster
方案一 自定义yaml文件安装redis cluster
背景
在Kubernetes中部署Redis集群面临挑战，因为每个Redis实例都依赖于一个配置文件，该文件可以跟踪其他集群实例及其角色。为此，需要结合使用Kubernetes StatefulSets和PersistentVolumes实现。
redis cluster架构示意图：
创建StatefulSet yaml文件
---
apiVersion: v1
kind: ConfigMap
metadata:
name: redis-cluster
data:
update.sh: |
#!/bin/sh
REDIS_NODES="/data/nodes.conf"
sed -i -e "/myself/ s/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/${POD_IP}/" ${REDIS_NODES}
exec "$@"
redis.conf: |+
bind 0.0.0.0
cluster-enabled yes
cluster-require-full-coverage no
cluster-node-timeout 30000
cluster-config-file /data/nodes.conf
cluster-migration-barrier 1
appendonly yes
protected-mode no
---
apiVersion: apps.kruise.io/v1beta1
# apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis-cluster
spec:
serviceName: redis-cluster
replicas: 6
selector:
matchLabels:
app: redis-cluster
template:
metadata:
labels:
app: redis-cluster
spec:
containers:
- name: redis
image: redis:6.2.1-alpine
ports:
- containerPort: 6379
name: client
- containerPort: 16379
name: gossip
command: ["/conf/update.sh", "redis-server", "/conf/redis.conf"]
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
volumeMounts:
- name: conf
mountPath: /conf
readOnly: false
- name: data
mountPath: /data
readOnly: false
volumes:
- name: conf
configMap:
name: redis-cluster
defaultMode: 0755
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 100Gi
storageClassName: rbd
注释：
cluster-migration-barrier 那些分配后仍然剩余migration barrier个从节点的主节点才会触发节点分配，而不是分配前有migration barrier个从节点的主节点就会触发节点分配，默认是1，生产环境建议维持默认值
protected-mode no 参数是为了禁止外网访问redis，如果启用了，则只能够通过lookback ip（127.0.0.1）访问Redis，如果外网访问redis，会报出异常。
apiVersion: apps.kruise.io/v1beta1 控制器这里使用kruise提供的Advanced StatefulSet，如果集群没有安装kruise，可以使用 apps/v1
安装redis-cluster
[root@qd01-stop-k8s-master001 redis]# kubectl apply -f install-redis.yaml
configmap/redis-cluster created
statefulset.apps.kruise.io/redis-cluster created
[root@qd01-stop-k8s-master001 redis]# kubectl get po -n op
NAME READY STATUS RESTARTS AGE
redis-cluster-0 1/1 Running 0 3m26s
redis-cluster-1 1/1 Running 0 3m14s
redis-cluster-2 1/1 Running 0 2m54s
redis-cluster-3 1/1 Running 0 2m23s
redis-cluster-4 1/1 Running 0 2m14s
redis-cluster-5 1/1 Running 0 114s
创建redis-cluster service
---
apiVersion: v1
kind: Service
metadata:
name: redis-cluster
namespace: op
spec:
type: ClusterIP
ports:
- port: 6379
targetPort: 6379
name: client
- port: 16379
targetPort: 16379
name: gossip
selector:
app: redis-cluster
[root@qd01-stop-k8s-master001 redis]# kubectl apply -f redis-svc.yml
service/redis-cluster created
[root@qd01-stop-k8s-master001 redis]# kubectl get svc -n op
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis-cluster ClusterIP 10.97.197.224 <none> 6379/TCP,16379/TCP 9s
测试能后连通
[root@qd01-stop-k8s-master001 redis]# telnet 10.97.197.224 6379
Trying 10.97.197.224...
Connected to 10.97.197.224.
Escape character is '^]'.
初始化redis-cluster
执行如下命令，获取到pod IP，然后使用redis-cli --cluster创建集群
[root@qd01-stop-k8s-master001 redis]# kubectl -n op exec -it redis-cluster-0 -- redis-cli --cluster create --cluster-replicas 1 $(kubectl -n op get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 {end}')
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 100.88.43.67:6379 to 100.64.147.152:6379
Adding replica 100.113.170.5:6379 to 100.98.174.217:6379
Adding replica 100.64.147.153:6379 to 100.80.158.227:6379
M: b47b27a3dbddf3fc1370cbe14ae753f4fce20b04 100.64.147.152:6379
slots:[0-5460] (5461 slots) master
M: 09543217c903350e963fc4fdf4acb73f8a1b7f8b 100.98.174.217:6379
slots:[5461-10922] (5462 slots) master
M: 5389ace495b68eeac85370d6783648dff68f2fb6 100.80.158.227:6379
slots:[10923-16383] (5461 slots) master
S: b1f39714c006ae55b12b18e6537303d7a00e1704 100.64.147.153:6379
replicates 5389ace495b68eeac85370d6783648dff68f2fb6
S: 0113f4668ec2f3ca2e9470c44bd5faab532b0936 100.88.43.67:6379
replicates b47b27a3dbddf3fc1370cbe14ae753f4fce20b04
S: e1e2f18ae66c79f1943390beabb59613abbad38a 100.113.170.5:6379
replicates 09543217c903350e963fc4fdf4acb73f8a1b7f8b
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
..
>>> Performing Cluster Check (using node 100.64.147.152:6379)
M: b47b27a3dbddf3fc1370cbe14ae753f4fce20b04 100.64.147.152:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 0113f4668ec2f3ca2e9470c44bd5faab532b0936 100.88.43.67:6379
slots: (0 slots) slave
replicates b47b27a3dbddf3fc1370cbe14ae753f4fce20b04
M: 09543217c903350e963fc4fdf4acb73f8a1b7f8b 100.98.174.217:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: 5389ace495b68eeac85370d6783648dff68f2fb6 100.80.158.227:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: e1e2f18ae66c79f1943390beabb59613abbad38a 100.113.170.5:6379
slots: (0 slots) slave
replicates 09543217c903350e963fc4fdf4acb73f8a1b7f8b
S: b1f39714c006ae55b12b18e6537303d7a00e1704 100.64.147.153:6379
slots: (0 slots) slave
replicates 5389ace495b68eeac85370d6783648dff68f2fb6
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
验证集群信息
[root@qd01-stop-k8s-master001 redis]# kubectl -n op exec -it redis-cluster-0 -- redis-cli cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:178
cluster_stats_messages_pong_sent:181
cluster_stats_messages_sent:359
cluster_stats_messages_ping_received:176
cluster_stats_messages_pong_received:178
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:359
[root@qd01-stop-k8s-master001 redis]# kubectl -n op exec -it redis-cluster-0 -- redis-cli cluster nodes
0113f4668ec2f3ca2e9470c44bd5faab532b0936 100.88.43.67:6379@16379 slave b47b27a3dbddf3fc1370cbe14ae753f4fce20b04 0 1615348311156 1 connected
09543217c903350e963fc4fdf4acb73f8a1b7f8b 100.98.174.217:6379@16379 master - 0 1615348314162 2 connected 5461-10922
b47b27a3dbddf3fc1370cbe14ae753f4fce20b04 100.64.147.152:6379@16379 myself,master - 0 1615348312000 1 connected 0-5460
5389ace495b68eeac85370d6783648dff68f2fb6 100.80.158.227:6379@16379 master - 0 1615348312000 3 connected 10923-16383
e1e2f18ae66c79f1943390beabb59613abbad38a 100.113.170.5:6379@16379 slave 09543217c903350e963fc4fdf4acb73f8a1b7f8b 0 1615348313160 2 connected
b1f39714c006ae55b12b18e6537303d7a00e1704 100.64.147.153:6379@16379 slave 5389ace495b68eeac85370d6783648dff68f2fb6 0 1615348312158 3 connected
从输出可以看到，集群总共6个节点，三主三从
方案二 使用kubeDB安装redis
安装kubeDB
1、安装KubeDB
获取AppsCode License https://license-issuer.appscode.com/
下载KubeDB charts https://github.com/appscode/charts/tree/master/stable/kubedb-community
[root@qd01-stop-k8s-master001 kubedb-community]# unzip kubedb-community-v0.16.2.tgz
[root@qd01-stop-k8s-master001 kubedb-community]# cd kubedb-community
[root@qd01-stop-k8s-master001 kubedb-community]# ls -al
total 96
drwxr-xr-x 4 root root 158 Mar 10 15:26 .
drwxr-xr-x 3 root root 66 Mar 10 15:24 ..
-rw-r--r-- 1 root root 351 Feb 16 09:55 Chart.yaml
drwxr-xr-x 2 root root 28 Mar 10 15:24 ci
-rw-r--r-- 1 root root 493 Feb 16 09:55 doc.yaml
-rw-r--r-- 1 root root 353 Feb 16 09:55 .helmignore
-rw-r--r-- 1 root root 24422 Feb 16 09:55 README.md
drwxr-xr-x 2 root root 4096 Mar 10 15:24 templates
-rw-r--r-- 1 root root 47437 Feb 16 09:55 values.openapiv3_schema.yaml
-rw-r--r-- 1 root root 5230 Feb 16 09:55 values.yaml
修改values.yaml，把License文件放到kubedb-community目录下
2、使用helm安装
[root@qd01-stop-k8s-master001 kubedb-community]# helm install kubedb-community --namespace kube-system --set-file license=./kubedb-community-license.txt -f values.yaml .
NAME: kubedb-community
LAST DEPLOYED: Wed Mar 10 15:38:59 2021
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
To verify that KubeDB has started, run:
kubectl get deployment --namespace kube-system -l "app.kubernetes.io/name=kubedb-community,app.kubernetes.io/instance=kubedb-community"
Now install/upgrade appscode/kubedb-catalog chart.
To install, run:
helm install kubedb-catalog appscode/kubedb-catalog --version v0.16.2 --namespace kube-system
To upgrade, run:
helm upgrade kubedb-catalog appscode/kubedb-catalog --version v0.16.2 --namespace kube-system
执行如下命令查看是否安装完成
[root@qd01-stop-k8s-master001 kubedb-community]# kubectl get deployment --namespace kube-system -l "app.kubernetes.io/name=kubedb-community,app.kubernetes.io/instance=kubedb-community"
NAME READY UP-TO-DATE AVAILABLE AGE
kubedb-community 1/1 1 1 38s
等待crds注册成功
[root@qd01-stop-k8s-master001 kubedb-community]# kubectl get crds -l app.kubernetes.io/name=kubedb -w
NAME CREATED AT
elasticsearches.kubedb.com 2021-03-10T07:39:42Z
elasticsearchversions.catalog.kubedb.com 2021-03-10T07:39:45Z
etcds.kubedb.com 2021-03-10T07:39:42Z
etcdversions.catalog.kubedb.com 2021-03-10T07:39:45Z
memcacheds.kubedb.com 2021-03-10T07:39:43Z
memcachedversions.catalog.kubedb.com 2021-03-10T07:39:45Z
mongodbs.kubedb.com 2021-03-10T07:39:43Z
mongodbversions.catalog.kubedb.com 2021-03-10T07:39:45Z
mysqls.kubedb.com 2021-03-10T07:39:43Z
mysqlversions.catalog.kubedb.com 2021-03-10T07:39:46Z
perconaxtradbs.kubedb.com 2021-03-10T07:39:43Z
perconaxtradbversions.catalog.kubedb.com 2021-03-10T07:39:46Z
pgbouncers.kubedb.com 2021-03-10T07:39:44Z
pgbouncerversions.catalog.kubedb.com 2021-03-10T07:39:46Z
postgreses.kubedb.com 2021-03-10T07:39:44Z
postgresversions.catalog.kubedb.com 2021-03-10T07:39:46Z
proxysqls.kubedb.com 2021-03-10T07:39:44Z
proxysqlversions.catalog.kubedb.com 2021-03-10T07:39:46Z
redises.kubedb.com 2021-03-10T07:39:45Z
redisversions.catalog.kubedb.com 2021-03-10T07:39:46Z
3、安装KubeDB Catalog
同样，先下载https://github.com/appscode/charts/tree/master/stable/kubedb-catalog
[root@qd01-stop-k8s-master001 kubedb-catalog]# tar -zxf kubedb-catalog-v0.16.2.tgz
[root@qd01-stop-k8s-master001 kubedb-catalog]# cd kubedb-catalog
[root@qd01-stop-k8s-master001 kubedb-catalog]# ls -al
total 24
drwxr-xr-x 3 root root 148 Mar 10 15:48 .
drwxr-xr-x 3 root root 28 Mar 10 15:48 ..
-rw-r--r-- 1 root root 321 Jan 26 20:08 Chart.yaml
-rw-r--r-- 1 root root 467 Jan 26 20:08 doc.yaml
-rw-r--r-- 1 root root 353 Jan 26 20:08 .helmignore
-rw-r--r-- 1 root root 3195 Jan 26 20:08 README.md
drwxr-xr-x 12 root root 188 Mar 10 15:48 templates
-rw-r--r-- 1 root root 744 Jan 26 20:08 values.openapiv3_schema.yaml
-rw-r--r-- 1 root root 1070 Jan 26 20:08 values.yaml
[root@qd01-stop-k8s-master001 kubedb-catalog]# helm install kubedb-catalog --namespace kube-system -f values.yaml .
NAME: kubedb-catalog
LAST DEPLOYED: Wed Mar 10 15:50:50 2021
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
使用kubedb安装redis
1、先看下官方给的redis生命周期示意图
kubedb安装redis支持如下特性
Features Availability
Clustering ✓
Instant Backup ✗
Scheduled Backup ✗
Persistent Volume ✓
Initialize using Snapshot ✗
Initialize using Script ✗
Custom Configuration ✓
Using Custom docker image ✓
Builtin Prometheus Discovery ✓
Using Prometheus operator ✓
2、查看支持的版本
[root@qd01-stop-k8s-master001 kubedb-catalog]# kubectl get redisversions
NAME VERSION DB_IMAGE DEPRECATED AGE
4.0.11 4.0.11 kubedb/redis:4.0.11 15m
4.0.6-v2 4.0.6 kubedb/redis:4.0.6-v2 15m
5.0.3-v1 5.0.3 kubedb/redis:5.0.3-v1 15m
6.0.6 6.0.6 kubedb/redis:6.0.6 15m
3、编辑yaml安装文件
可以参照https://github.com/kubedb/docs/blob/v2021.01.26/docs/examples/redis/clustering/demo-1.yaml
这里选择安装6.0.6这个版本，我的集群storageClassName: "rbd"，请根据实际修改
如果想自定义redis.conf,请参考https://github.com/kubedb/docs/blob/v2021.01.26/docs/examples/redis/custom-config/redis-custom.yaml
apiVersion: kubedb.com/v1alpha2
kind: Redis
metadata:
name: redis-cluster
namespace: op
spec:
version: 6.0.6
mode: Cluster
cluster:
master: 3
replicas: 1
storageType: Durable
storage:
resources:
requests:
storage: 1Gi
storageClassName: "rbd"
accessModes:
- ReadWriteOnce
执行安装
[root@qd01-stop-k8s-master001 kubedb-community]# kubectl apply -f redis-cluster.yaml
redis.kubedb.com/redis-cluster created
安装完成，可以如下查看
[root@qd01-stop-k8s-master001 kubedb-community]# kubectl get rd,po -n op
NAME VERSION STATUS AGE
redis.kubedb.com/redis-cluster 6.0.6 Provisioning 6m55s
NAME READY STATUS RESTARTS AGE
pod/redis-cluster-shard0-0 1/1 Running 0 6m54s
pod/redis-cluster-shard0-1 1/1 Running 0 6m18s
pod/redis-cluster-shard1-0 1/1 Running 0 5m38s
pod/redis-cluster-shard1-1 1/1 Running 0 5m1s
pod/redis-cluster-shard2-0 1/1 Running 0 4m30s
pod/redis-cluster-shard2-1 1/1 Running 0 4m8s
[root@qd01-stop-k8s-master001 redis]# kubectl get svc -n op
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis-cluster ClusterIP 10.97.197.224 <none> 6379/TCP 5h16m
redis-cluster-pods ClusterIP None <none> 6379/TCP 17m
4、验证集群
[root@qd01-stop-k8s-master001 kubedb-community]# kubectl get pods -n op -o jsonpath='{range.items[*]}{.metadata.name} ---------- {.status.podIP}:6379{"\t\n"}{end}' | grep redis
redis-cluster-shard0-0 ---------- 100.64.147.156:6379
redis-cluster-shard0-1 ---------- 100.98.174.218:6379
redis-cluster-shard1-0 ---------- 100.126.252.204:6379
redis-cluster-shard1-1 ---------- 100.113.170.6:6379
redis-cluster-shard2-0 ---------- 100.107.55.69:6379
redis-cluster-shard2-1 ---------- 100.78.230.4:6379
[root@qd01-stop-k8s-master001 redis]# kubectl -n op exec -it redis-cluster-shard0-0 -- redis-cli cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:3
cluster_my_epoch:1
cluster_stats_messages_ping_sent:864
cluster_stats_messages_pong_sent:882
cluster_stats_messages_sent:1746
cluster_stats_messages_ping_received:879
cluster_stats_messages_pong_received:864
cluster_stats_messages_meet_received:3
cluster_stats_messages_received:1746
[root@qd01-stop-k8s-master001 redis]# kubectl -n op exec -it redis-cluster-shard0-0 -- redis-cli cluster nodes
1895cb4b9c31b848666c61000e502f55a29a8255 100.64.147.155:6379@16379 master - 0 1615365162008 2 connected 5461-10922
30bdbf2ca37001774498a9b935afbc1cd2ce389c 100.126.252.203:6379@16379 slave 2c06092fafa99e0158e39e6237a04fed25be3550 0 1615365163000 1 connected
9b2cfbd5c1b417121d410141b6da9512ad29ce3c 100.78.230.3:6379@16379 slave e83446c368839c5fdccf5f70e3b1004eb67cb651 0 1615365163512 3 connected
2c06092fafa99e0158e39e6237a04fed25be3550 100.82.197.130:6379@16379 myself,master - 0 1615365162000 1 connected 0-5460
1379d2b20f26ab13d53068d276ec5d988b7a0273 100.64.122.197:6379@16379 slave 1895cb4b9c31b848666c61000e502f55a29a8255 0 1615365163000 2 connected
e83446c368839c5fdccf5f70e3b1004eb67cb651 100.107.55.68:6379@16379 master - 0 1615365164014 3 connected 10923-16383
kubernetes生产实践之redis-cluster的更多相关文章
- K8S部署Redis Cluster集群
kubernetes部署单节点redis: https://www.cnblogs.com/zisefeizhu/p/14282299.html Redis 介绍 • Redis代表REmote DI ...
- K8S部署Redis Cluster集群（三主三从模式） - 部署笔记
一.Redis 介绍 Redis代表REmote DIctionary Server是一种开源的内存中数据存储,通常用作数据库,缓存或消息代理.它可以存储和操作高级数据类型,例如列表,地图,集合和排序 ...
- Redis中国用户组|唯品会Redis cluster大规模生产实践
嘉宾:陈群 很高兴有机会在Redis中国用户组给大家分享redis cluster的生产实践.目前在唯品会主要负责redis/hbase的运维和开发支持工作,也参与工具开发工作 Outline 一.生 ...
- 后端程序员之路 25、Redis Cluster
官方教程和功能介绍: REDIS cluster-tutorial -- Redis中文资料站 -- Redis中国用户组(CRUG)http://www.redis.cn/topics/cluste ...
- JuiceFS v1.0 beta3 发布，支持 etcd、Amazon MemoryDB、Redis Cluster
JuiceFS v1.0 beta3 在元数据引擎方面继续增强,新增 etcd 支持小于 200 万文件的使用场景,相比 Redis 可以提供更好的可用性和安全性.同时支持了 Amazon Memor ...
- 在 Kubernetes 中部署 Redis 集群
在 Kubernetes 中部署 Redis 集群 在Kubernetes中部署Redis集群面临挑战,因为每个 Redis 实例都依赖于一个配置文件,该文件可以跟踪其他集群实例及其角色.为此,我们需 ...
- window下使用Redis Cluster部署Redis集群
日常的项目很多时候都需要用到缓存.redis算是一个比较好的选择.一般情况下做一个主从就可以满足一些比较小的项目需要.在一些并发量比较大的项目可能就需要用到集群了,redis在Windows下做集群可 ...
- Redis Cluster 分区实现原理
Redis Cluster本身提供了自动将数据分散到Redis Cluster不同节点的能力,分区实现的关键点问题包括:如何将数据自动地打散到不同的节点,使得不同节点的存储数据相对均匀:如何保证客户端 ...
- Redis Cluster
使用 Redis Cluster Redis 3.0 在2015年出了Stable版本,3.0版本相对于2.8版本带来的主要新特性包括: 实现了Redis Cluster,从而做到了对集群的支持: 引 ...
随机推荐
- SQL Server 新安装启用sa用户/sa用户登录提示管道另一端无进程
安装时只用windows验证 安装完成后: 首先选中服务器(右键)->属性->安全性->服务器身份验证修改为"SQL SERVER和WINDOWS身份验证模式"其 ...
- WSL1中安装Docker
# step 0: clear# sudo apt-get remove docker docker-engine docker-ce docker.io# step 1: 安装必要的一些系统工具su ...
- 6.PowerShell DSC核心概念之LCM
什么是LCM? 本地配置管理器 (LCM) 是DSC的引擎. LCM 在每个目标节点上运行,负责分析和执行发送到节点的配置. 它还负责 DSC 的许多方面,包括以下各方面. 确定刷新模式(推送或请求) ...
- 大数据去重（data deduplication）方案
数据去重(data deduplication)是大数据领域司空见惯的问题了.除了统计UV等传统用法之外,去重的意义更在于消除不可靠数据源产生的脏数据--即重复上报数据或重复投递数据的影响,使计算产生 ...
- 流媒体传输协议之 RTP（下篇）
本系列文章将整理各个流媒体传输协议,包括 RTP/RTCP,RTMP,希望通过深入梳理协议的设计细节,能够给流媒体领域的开发者带来一定的启发. 作者:逸殊 审核:泰一 接上篇:< 流媒体传输协议 ...
- Ubuntu 下更改pip源使用清华源
一.新建目录 sudo -s mkdisk ~./pip vim ~./pip/pip.conf 二.复制下面代码,并保存 [global] index-url = https://pypi.tuna ...
- Leetcode（102）-二叉树的层次遍历
给定一个二叉树,返回其按层次遍历的节点值. (即逐层地,从左到右访问所有节点). 例如:给定二叉树: [3,9,20,null,null,15,7], 3 / \ 9 20 / \ 15 7 返回其层 ...
- full page screen capture in js
full page screen capture in js html2canvas https://html2canvas.hertzen.com/ https://github.com/nikla ...
- 下载com.springsource.org.aspectj.weaver-1.6.8.RELEASE.jar
看别人都说在repo.maven.com下载,没想到竟然要登录 索性我直接在国内阿里云的镜像仓库下载好了,速度又快又方便 搜索aspectj 下载地址:https://maven.aliyun.com ...
- Vue学习笔记-API调试工具--->国产apipost按装(比postman好按装好用)
一 使用环境: windows 7 64位操作系统 二 Vue学习笔记-API调试工具--->apipost按装 1.下载: https://www.apipost.cn/ (比postm ...