OpenMMLab AI实战营 第三课笔记
OpenMMLab AI实战营 第三课笔记
目录
花朵五分类数据集：https://www.kaggle.com/datasets/alxmamaev/flowers-recognition
进入 mmclassification 目录
In [1]:
import os
os.chdir('mmclassification')
导入工具包
In [2]:
import torch
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print('device', device)
device cuda:0
下载数据集
In [3]:
!wget https://zihao-openmmlab.obs.cn-east-3.myhuaweicloud.com/20220716-mmclassification/dataset/flower.zip -O data/flower.zip
--2022-07-16 22:34:18-- https://zihao-openmmlab.obs.cn-east-3.myhuaweicloud.com/20220716-mmclassification/dataset/flower.zip
Connecting to 172.16.0.13:5848... connected.
Proxy request sent, awaiting response... 200 OK
Length: 230662310 (220M) [application/zip]
Saving to: ‘data/flower.zip’
data/flower.zip 100%[===================>] 219.98M 27.9MB/s in 7.8s
2022-07-16 22:34:28 (28.3 MB/s) - ‘data/flower.zip’ saved [230662310/230662310]
In [4]:
# 解压
!unzip data/flower.zip -d data >> /dev/null
In [13]:
from PIL import Image
Image.open('data/flower/test/daisy/11023214096_b5b39fab08.jpg')
数据集目录结构
In [21]:
'''
flower
├── classes.txt
├── train.txt
├── val.txt
├── test.txt
├── train
│ ├── daisy
│ ├── dandelion
│ ├── rose
│ ├── sunflower
│ └── tulip
├── test
│ ├── daisy
│ ├── dandelion
│ ├── rose
│ ├── sunflower
│ └── tulip
└── val
├── daisy
├── dandelion
├── rose
├── sunflower
└── tulip
'''
下载 config 配置文件
In [30]:
'''
Model config, which specify the basic structure of the model, e.g. number of the input channels.
Dataset config, which contains details about the dataset, e.g. type of the dataset.
Schedule config, which specify the training schedules, e.g. learning rate.
Runtime config, which contains the rest of details, e.g. log config.
'''
In [11]:
!wget https://zihao-openmmlab.obs.cn-east-3.myhuaweicloud.com/20220716-mmclassification/configs/mobilenet_v2_1x_flower.py -O configs/mobilenet_v2/mobilenet_v2_1x_flower.py
--2022-07-16 22:51:45-- https://zihao-openmmlab.obs.cn-east-3.myhuaweicloud.com/20220716-mmclassification/configs/mobilenet_v2_1x_flower.py
Connecting to 172.16.0.13:5848... connected.
Proxy request sent, awaiting response... 200 OK
Length: 1975 (1.9K) [binary/octet-stream]
Saving to: ‘configs/mobilenet_v2/mobilenet_v2_1x_flower.py’
configs/mobilenet_v 100%[===================>] 1.93K --.-KB/s in 0s
2022-07-16 22:51:45 (8.72 MB/s) - ‘configs/mobilenet_v2/mobilenet_v2_1x_flower.py’ saved [1975/1975]
命令行-训练
In [12]:
!python tools/train.py \
configs/mobilenet_v2/mobilenet_v2_1x_flower.py \
--work-dir work_dirs/mobilenet_v2_1x_flower
/home/featurize/work/MMClassification教程/mmclassification/mmcls/utils/setup_env.py:33: UserWarning: Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
f'Setting OMP_NUM_THREADS environment variable for each process '
/home/featurize/work/MMClassification教程/mmclassification/mmcls/utils/setup_env.py:43: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
f'Setting MKL_NUM_THREADS environment variable for each process '
2022-07-16 22:51:55,465 - mmcls - INFO - Environment info:
------------------------------------------------------------
sys.platform: linux
Python: 3.7.10 (default, Jun 4 2021, 14:48:32) [GCC 7.5.0]
CUDA available: True
GPU 0: NVIDIA RTX A4000
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 11.2, V11.2.152
GCC: gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
PyTorch: 1.10.0+cu113
PyTorch compiling details: PyTorch built with:
- GCC 7.3
- C++ Version: 201402
- Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX512
- CUDA Runtime 11.3
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
- CuDNN 8.2
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,
TorchVision: 0.11.1+cu113
OpenCV: 4.5.4
MMCV: 1.6.0
MMCV Compiler: GCC 9.3
MMCV CUDA Compiler: 11.3
MMClassification: 0.23.1+d2e5054
------------------------------------------------------------
2022-07-16 22:51:55,465 - mmcls - INFO - Distributed training: False
2022-07-16 22:51:55,601 - mmcls - INFO - Config:
model = dict(
type='ImageClassifier',
backbone=dict(type='MobileNetV2', widen_factor=1.0),
neck=dict(type='GlobalAveragePooling'),
head=dict(
type='LinearClsHead',
num_classes=5,
in_channels=1280,
loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
topk=(1, 3)))
dataset_type = 'ImageNet'
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='RandomResizedCrop', size=224, backend='pillow'),
dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='ToTensor', keys=['gt_label']),
dict(type='Collect', keys=['img', 'gt_label'])
]
test_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='Resize', size=(256, -1), backend='pillow'),
dict(type='CenterCrop', crop_size=224),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
]
data = dict(
samples_per_gpu=32,
workers_per_gpu=2,
train=dict(
type='ImageNet',
data_prefix='data/flower/train',
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='RandomResizedCrop', size=224, backend='pillow'),
dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='ToTensor', keys=['gt_label']),
dict(type='Collect', keys=['img', 'gt_label'])
],
classes='data/flower/classes.txt'),
val=dict(
type='ImageNet',
data_prefix='data/flower/val',
ann_file='data/flower/val.txt',
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='Resize', size=(256, -1), backend='pillow'),
dict(type='CenterCrop', crop_size=224),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
],
classes='data/flower/classes.txt'),
test=dict(
type='ImageNet',
data_prefix='data/flower/test',
ann_file='data/flower/test.txt',
pipeline=[
dict(type='LoadImageFromFile'),
dict(type='Resize', size=(256, -1), backend='pillow'),
dict(type='CenterCrop', crop_size=224),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
],
classes='data/flower/classes.txt'))
evaluation = dict(
interval=1,
metric=['accuracy', 'precision', 'f1_score'],
metric_options=dict(topk=(1, )))
optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=None)
lr_config = dict(policy='step', gamma=0.98, step=[1])
runner = dict(type='EpochBasedRunner', max_epochs=2)
checkpoint_config = dict(interval=1)
log_config = dict(interval=100, hooks=[dict(type='TextLoggerHook')])
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = 'https://download.openmmlab.com/mmclassification/v0/mobilenet_v2/mobilenet_v2_batch256_imagenet_20200708-3b2dc3af.pth'
resume_from = None
workflow = [('train', 1)]
work_dir = 'work_dirs/mobilenet_v2_1x_flower'
gpu_ids = [0]
2022-07-16 22:51:55,601 - mmcls - INFO - Set random seed to 943425345, deterministic: False
2022-07-16 22:51:55,802 - mmcls - INFO - initialize MobileNetV2 with init_cfg [{'type': 'Kaiming', 'layer': ['Conv2d']}, {'type': 'Constant', 'val': 1, 'layer': ['_BatchNorm', 'GroupNorm']}]
2022-07-16 22:51:55,832 - mmcls - INFO - initialize LinearClsHead with init_cfg {'type': 'Normal', 'layer': 'Linear', 'std': 0.01}
2022-07-16 22:52:02,074 - mmcls - INFO - load checkpoint from http path: https://download.openmmlab.com/mmclassification/v0/mobilenet_v2/mobilenet_v2_batch256_imagenet_20200708-3b2dc3af.pth
2022-07-16 22:52:02,104 - mmcls - WARNING - The model and loaded state dict do not match exactly
size mismatch for head.fc.weight: copying a param with shape torch.Size([1000, 1280]) from checkpoint, the shape in current model is torch.Size([5, 1280]).
size mismatch for head.fc.bias: copying a param with shape torch.Size([1000]) from checkpoint, the shape in current model is torch.Size([5]).
2022-07-16 22:52:02,105 - mmcls - INFO - Start running, host: featurize@featurize, work_dir: /home/featurize/work/MMClassification教程/mmclassification/work_dirs/mobilenet_v2_1x_flower
2022-07-16 22:52:02,105 - mmcls - INFO - Hooks will be executed in the following order:
before_run:
(VERY_HIGH ) StepLrUpdaterHook
(NORMAL ) CheckpointHook
(LOW ) EvalHook
(VERY_LOW ) TextLoggerHook
--------------------
before_train_epoch:
(VERY_HIGH ) StepLrUpdaterHook
(LOW ) IterTimerHook
(LOW ) EvalHook
(VERY_LOW ) TextLoggerHook
--------------------
before_train_iter:
(VERY_HIGH ) StepLrUpdaterHook
(LOW ) IterTimerHook
(LOW ) EvalHook
--------------------
after_train_iter:
(ABOVE_NORMAL) OptimizerHook
(NORMAL ) CheckpointHook
(LOW ) IterTimerHook
(LOW ) EvalHook
(VERY_LOW ) TextLoggerHook
--------------------
after_train_epoch:
(NORMAL ) CheckpointHook
(LOW ) EvalHook
(VERY_LOW ) TextLoggerHook
--------------------
before_val_epoch:
(LOW ) IterTimerHook
(VERY_LOW ) TextLoggerHook
--------------------
before_val_iter:
(LOW ) IterTimerHook
--------------------
after_val_iter:
(LOW ) IterTimerHook
--------------------
after_val_epoch:
(VERY_LOW ) TextLoggerHook
--------------------
after_run:
(VERY_LOW ) TextLoggerHook
--------------------
2022-07-16 22:52:02,105 - mmcls - INFO - workflow: [('train', 1)], max: 2 epochs
2022-07-16 22:52:02,105 - mmcls - INFO - Checkpoints will be saved to /home/featurize/work/MMClassification教程/mmclassification/work_dirs/mobilenet_v2_1x_flower by HardDiskBackend.
2022-07-16 22:52:11,810 - mmcls - INFO - Saving checkpoint at 1 epochs
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 715/715, 354.3 task/s, elapsed: 2s, ETA: 0s2022-07-16 22:52:13,944 - mmcls - INFO - Epoch(val) [1][23] accuracy_top-1: 66.1538, precision: 73.5692, f1_score: 65.5141
2022-07-16 22:52:23,245 - mmcls - INFO - Saving checkpoint at 2 epochs
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 715/715, 360.9 task/s, elapsed: 2s, ETA: 0s2022-07-16 22:52:25,354 - mmcls - INFO - Epoch(val) [2][23] accuracy_top-1: 88.6713, precision: 89.7683, f1_score: 88.7995
用训练得到的图像分类模型，对新图像预测
In [16]:
import matplotlib.pyplot as plt
import mmcv
from mmcls.apis import inference_model, init_model, show_result_pyplot
img = mmcv.imread('data/flower/test/daisy/11023214096_b5b39fab08.jpg')
# img = mmcv.imread('data/cat2.jpg')
# 图像分类模型 config 配置文件
config_file = 'configs/mobilenet_v2/mobilenet_v2_1x_flower.py'
# 图像分类模型 checkpoint 权重文件
checkpoint_file = 'work_dirs/mobilenet_v2_1x_flower/latest.pth'
# 通过 config 配置文件 和 checkpoint 权重文件 构建模型
model = init_model(config_file, checkpoint_file, device=device)
result = inference_model(model, img)
print('类别', result['pred_class'], '置信度', result['pred_score'])
show_result_pyplot(model, img, result)
load checkpoint from local path: work_dirs/mobilenet_v2_1x_flower/latest.pth
类别 daisy 置信度 0.9996930360794067
将训练得到的模型在测试集上预测，获得所有测试集数据的预测结果
In [17]:
!python tools/test.py \
configs/mobilenet_v2/mobilenet_v2_1x_flower.py \
work_dirs/mobilenet_v2_1x_flower/latest.pth \
--out testset_result.json
/home/featurize/work/MMClassification教程/mmclassification/mmcls/utils/setup_env.py:33: UserWarning: Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
f'Setting OMP_NUM_THREADS environment variable for each process '
/home/featurize/work/MMClassification教程/mmclassification/mmcls/utils/setup_env.py:43: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
f'Setting MKL_NUM_THREADS environment variable for each process '
load checkpoint from local path: work_dirs/mobilenet_v2_1x_flower/latest.pth
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 715/715, 358.6 task/s, elapsed: 2s, ETA: 0s
dumping results to results_flower.json
将训练得到的模型在测试集上预测，获得图像分类评估结果
In [18]:
!python tools/test.py \
configs/mobilenet_v2/mobilenet_v2_1x_flower.py \
work_dirs/mobilenet_v2_1x_flower/latest.pth \
--metrics accuracy precision recall f1_score support \
--metric-options topk=1
/home/featurize/work/MMClassification教程/mmclassification/mmcls/utils/setup_env.py:33: UserWarning: Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
f'Setting OMP_NUM_THREADS environment variable for each process '
/home/featurize/work/MMClassification教程/mmclassification/mmcls/utils/setup_env.py:43: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
f'Setting MKL_NUM_THREADS environment variable for each process '
load checkpoint from local path: work_dirs/mobilenet_v2_1x_flower/latest.pth
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 715/715, 352.3 task/s, elapsed: 2s, ETA: 0s
accuracy : 88.67
support : 715.0
precision : 89.77
recall : 88.83
f1_score : 88.8
OpenMMLab AI实战营 第三课笔记的更多相关文章
- Elasticsearch7.X 入门学习第三课笔记----search api学习（URI Search）
原文:Elasticsearch7.X 入门学习第三课笔记----search api学习(URI Search) 版权声明:本文为博主原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出 ...
- 华为云 AI 实战营计划，带你迈上 AI 之路
当今,AI的开发人才需求呈现极大的供需不平衡.所有开发者都关心,要如何从一名开发者晋升为AI开发者?AI开发能力,是主要的进入障碍.不用慌,华为云推出了 <华为云ModelArts-Lab AI ...
- Nodejs课堂笔记-第三课 构建一个nodejs的Docker镜像
本文由Vikings(http://www.cnblogs.com/vikings-blog/) 原创,转载请标明.谢谢! 因为一直做Linux有关的开发工作,所以不习惯在Windows平台编译和测试 ...
- 红帽学习笔记[RHCSA] 第三课[输出重定向、Vi编辑器]
第三课 关于Linux的输入输出 输入输出 0 stdin 标准输入 仅读取 1 stdout 标准输出 仅写入 2 stderr 标准错误 仅写入 3 filename 其他文件 读取和/或写入 输 ...
- 小马哥的 Java 项目实战营学习笔记（1）
小马哥的 Java 项目实战营 小马哥的 Java 项目实战营 第二节:数据存储之 JDBC JDBC 核心 API 数据源 接口 - javax.sql.DataSource获取方式 1.普通对象初 ...
- 清华大学ucore操作系统课笔记
操作系统 清华大学ucore操作系统课笔记 全文思维导图 1. 操作系统概述 1.1 什么是操作系统? 操作系统的定义 没有公认的精确定义 一个控制程序 一个系统软件 控制程序执行过程,防止错误和计算 ...
- CodeIgniter框架入门教程——第三课 URL及ajax
本文转载自:http://www.softeng.cn/?p=74 这节课讲一下CI框架的路由规则,以及如何在CI框架下实现ajax功能. 首先,先介绍CI框架的路由规则,因为CI框架是在PHP的基础 ...
- Udacity调试课笔记之断言异常
Udacity调试课笔记之断言异常 这一单元的内容不是很多,如Zeller教授所说,就是如何写.检查断言,并如何使用工具实现自动推导出断言的条件. 现在,多数的编程语言,尤其是高级编程语言都会有内置的 ...
- 菜农群课笔记之ICP与ISP----20110412(整理版)
耗时一上午时间对HOT大叔昨晚的群课内容进行温故并整理,现将其上传,若想看直播可到下面链接处下载:http://bbs.21ic.com/icview-229746-1-1.html 成 ...
- Octave Tutorial（《Machine Learning》）之第三课《数据计算》
第三课 Culculating Data 数据计算 矩阵计算 1.简单的四则运算 2.相乘除,乘方运算(元素位运算) ".*"为对应元素的相乘计算 "./"为对 ...
随机推荐
- 关于git的安装
window平台下面: 步骤一: 首先去官网下载安装包: 官方链接:https://git-scm.com/download/win 至于选择32位还是64为的,各位就见仁见智了(根据自己的系统). ...
- 干活总结 | SQL 语法速成手册
本文针对关系型数据库的一般语法.限于篇幅,本文侧重说明用法,不会展开讲解特性.原理.本文内容包括基本概念.增删改查.子查询.连接和组合.函数.排序和分组.数据定义.事务处理.权限管理等等. 一.基本概 ...
- 学习JavaScript第五天
文章目录 1.HTML DOM 1.1 表单相关元素 ① form 元素 ② 文本输入框类和文本域(input 和 textarea) ③ select 元素 1.2 表格相关元素 ① table 元 ...
- Selenium测试form表单之checkbox和radio
一.定义form表单 用到的元素:checkbox和radiobutton 下图定义了一个选择爱好和选择性别的form表单,区域1用到的表单元素是checkbox(复选框),区域2用到的表单元素是ra ...
- Paimon lookup store 实现
Lookup Store 主要用于 Paimon 中的 Lookup Compaction 以及 Lookup join 的场景. 会将远程的列存文件在本地转化为 KV 查找的格式. Hash htt ...
- 说说Java的类加载机制？究竟什么是双亲委派模型？
首先引入一个概念,什么是Java类加载器? 一句话总结:类加载器(class loader)用来加载 Java 类到 Java 虚拟机中. 官方总结:Java类加载器(英语:Java Classloa ...
- Visual Studio 快速分析 .NET Dump 文件
前言 在开发和维护 .NET 应用程序的过程中,有时会遇到难以捉摸的性能瓶颈或内存泄漏等问题.这些问题往往发生在生产环境中,难以复现.为了更准确地诊断这些运行时问题,通常会收集应用程序在生产环境中的内 ...
- html face属性
无意间发现邮件里面的字体非常像手写的,然后点击HTML源码发现,使用了这个face属性. 代码如下: <font face="comic sans ms">PS: 你看 ...
- 关于开启了auth的站点如何使用curl访问
有的站点开了 auth 如何访问呢 可以直接这样 将 用户名密码放入URL中 http://username:password@host:8080/index.html 即可.
- Gitlab之CICD环境变量
CI/CD variables are a type of environment variable. You can use them to: Control the behavior of job ...