AI识别检验报告 -PaddleNLP UIE-X 在医疗领域的实战
UIE-X在医疗领域的实战
PaddleNLP全新发布UIE-X ，除已有纯文本抽取的全部功能外，新增文档抽取能力。
UIE-X延续UIE的思路，基于跨模态布局增强预训练模型文心ERNIE-Layout重训模型，融合文本、图像、布局等信息进行联合建模，能够深度理解多模态文档。基于Prompt思想，实现开放域信息抽取，支持零样本抽取，小样本能力领先。
项目链接：https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/information_extraction
本案例为UIE-X在医疗领域的实战，通过少量标注+模型微调即可具备定制场景的端到端文档信息提取能力！
1.项目背景
目前医疗领域有大量的医学检查报告单，病历，发票，CT影像，眼科等等的医疗图片数据。现阶段，针对这些图片都是靠人工分类，结构化录入系统中，做患者的全生命周期的管理。
耗时耗力，人工成本极大。如果能靠人工智能的技术做到图片的自动分类和结构化，将大大的降低成本，提高系统录入的整体效率。
2.案例简介
本案例基于PaddleNLP最新开源的UIE-X，以医学检查单这种医疗领域常见的图片类型为例，展示从数据标注、模型训练到Taskflow一键部署的全流程解决方案
数据集来源：https://tianchi.aliyun.com/dataset/126039
数据集样例展示：
医疗场景常见图片展示：
3.环境准备
!pip install --upgrade --user paddleocr
!pip install --upgrade --user paddlenlp
我们推荐使用数据标注平台Label-Studio进行数据标注，本案例也打通了从标注到训练的通道，即Label-Studio导出数据后可通过label_studio.py脚本轻松将数据转换为输入模型时需要的形式，实现无缝衔接。为了达到这个目的，您可以参考信息抽取任务Label-Studio标注指南在Label-Studio平台上标注数据：
# 下载标注数据：
!wget https://paddlenlp.bj.bcebos.com/datasets/medical_checklist.zip
!unzip medical_checklist.zip
数据转换
!python label_studio.py \
--label_studio_file ./medical_checklist/label_studio.json \
--save_dir ./medical_checklist \
--splits 0.8 0.2 0\
--task_type ext \
5.模型微调
!python finetune.py \
--device gpu \
--logging_steps 5 \
--save_steps 25 \
--eval_steps 25 \
--seed 42 \
--model_name_or_path uie-x-base \
--output_dir ./checkpoint/model_best \
--train_path medical_checklist/train.txt \
--dev_path medical_checklist/dev.txt \
--per_device_train_batch_size 16 \
--per_device_eval_batch_size 16 \
--num_train_epochs 5 \
--learning_rate 1e-5 \
--label_names 'start_positions' 'end_positions' \
--do_train \
--do_eval \
--do_export \
--export_model_dir ./checkpoint/model_best \
--overwrite_output_dir \
--disable_tqdm True \
--metric_for_best_model eval_f1 \
--load_best_model_at_end True \
--save_total_limit 1
[2023-07-21 15:36:09,684] [ WARNING] - evaluation_strategy reset to IntervalStrategy.STEPS for do_eval is True. you can also set evaluation_strategy='epoch'.
[2023-07-21 15:36:09,684] [ INFO] - The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
[2023-07-21 15:36:09,684] [ INFO] - ============================================================
[2023-07-21 15:36:09,685] [ INFO] - Model Configuration Arguments
[2023-07-21 15:36:09,685] [ INFO] - paddle commit id :3fa7a736e32508e797616b6344d97814c37d3ff8
[2023-07-21 15:36:09,685] [ INFO] - export_model_dir :./checkpoint/model_best
[2023-07-21 15:36:09,685] [ INFO] - model_name_or_path :uie-x-base
[2023-07-21 15:36:09,685] [ INFO] -
[2023-07-21 15:36:09,685] [ INFO] - ============================================================
[2023-07-21 15:36:09,685] [ INFO] - Data Configuration Arguments
[2023-07-21 15:36:09,685] [ INFO] - paddle commit id :3fa7a736e32508e797616b6344d97814c37d3ff8
[2023-07-21 15:36:09,685] [ INFO] - dev_path :medical_checklist/dev.txt
[2023-07-21 15:36:09,685] [ INFO] - max_seq_len :512
[2023-07-21 15:36:09,685] [ INFO] - train_path :medical_checklist/train.txt
[2023-07-21 15:36:09,685] [ INFO] -
[2023-07-21 15:36:09,685] [ WARNING] - Process rank: -1, device: gpu, world_size: 1, distributed training: False, 16-bits training: False
[2023-07-21 15:36:09,686] [ INFO] - Model config ErnieLayoutConfig {
"attention_probs_dropout_prob": 0.1,
"bos_token_id": 0,
"coordinate_size": 128,
"enable_recompute": false,
"eos_token_id": 2,
"fuse": false,
"gradient_checkpointing": false,
"has_relative_attention_bias": true,
"has_spatial_attention_bias": true,
"has_visual_segment_embedding": false,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 768,
"image_feature_pool_shape": [
7,
7,
256
],
"initializer_range": 0.02,
"intermediate_size": 3072,
"layer_norm_eps": 1e-12,
"max_2d_position_embeddings": 1024,
"max_position_embeddings": 514,
"max_rel_2d_pos": 256,
"max_rel_pos": 128,
"model_type": "ernie_layout",
"num_attention_heads": 12,
"num_hidden_layers": 12,
"output_past": true,
"pad_token_id": 1,
"paddlenlp_version": null,
"pool_act": "tanh",
"rel_2d_pos_bins": 64,
"rel_pos_bins": 32,
"shape_size": 128,
"task_id": 0,
"task_type_vocab_size": 3,
"type_vocab_size": 100,
"use_task_id": true,
"vocab_size": 250002
}
[2023-07-21 15:36:09,687] [ INFO] - Configuration saved in /home/aistudio/.paddlenlp/models/uie-x-base/config.json
[2023-07-21 15:36:09,687] [ INFO] - Downloading uie_x_base.pdparams from https://bj.bcebos.com/paddlenlp/models/transformers/uie_x/uie_x_base.pdparams
100%|██████████████████████████████████████| 1.05G/1.05G [00:15<00:00, 73.4MB/s]
W0721 15:36:28.591925 856 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0721 15:36:28.595674 856 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
[2023-07-21 15:36:30,069] [ INFO] - All model checkpoint weights were used when initializing UIEX.
[2023-07-21 15:36:30,069] [ INFO] - All the weights of UIEX were initialized from the model checkpoint at uie-x-base.
If your task is similar to the task the model of the checkpoint was trained on, you can already use UIEX for predictions without further training.
[2023-07-21 15:36:30,070] [ INFO] - We are using <class 'paddlenlp.transformers.ernie_layout.tokenizer.ErnieLayoutTokenizer'> to load 'uie-x-base'.
[2023-07-21 15:36:30,071] [ INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/transformers/ernie_layout/vocab.txt and saved to /home/aistudio/.paddlenlp/models/uie-x-base
[2023-07-21 15:36:30,132] [ INFO] - Downloading vocab.txt from https://bj.bcebos.com/paddlenlp/models/transformers/ernie_layout/vocab.txt
100%|██████████████████████████████████████| 2.70M/2.70M [00:00<00:00, 48.4MB/s]
[2023-07-21 15:36:30,263] [ INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/transformers/ernie_layout/sentencepiece.bpe.model and saved to /home/aistudio/.paddlenlp/models/uie-x-base
[2023-07-21 15:36:30,325] [ INFO] - Downloading sentencepiece.bpe.model from https://bj.bcebos.com/paddlenlp/models/transformers/ernie_layout/sentencepiece.bpe.model
100%|██████████████████████████████████████| 4.83M/4.83M [00:00<00:00, 63.2MB/s]
[2023-07-21 15:36:31,214] [ INFO] - tokenizer config file saved in /home/aistudio/.paddlenlp/models/uie-x-base/tokenizer_config.json
[2023-07-21 15:36:31,214] [ INFO] - Special tokens file saved in /home/aistudio/.paddlenlp/models/uie-x-base/special_tokens_map.json
[2023-07-21 15:36:33,843] [ INFO] - ============================================================
[2023-07-21 15:36:33,844] [ INFO] - Training Configuration Arguments
[2023-07-21 15:36:33,844] [ INFO] - paddle commit id :3fa7a736e32508e797616b6344d97814c37d3ff8
[2023-07-21 15:36:33,844] [ INFO] - _no_sync_in_gradient_accumulation:True
[2023-07-21 15:36:33,844] [ INFO] - activation_quantize_type :None
[2023-07-21 15:36:33,844] [ INFO] - adam_beta1 :0.9
[2023-07-21 15:36:33,844] [ INFO] - adam_beta2 :0.999
[2023-07-21 15:36:33,844] [ INFO] - adam_epsilon :1e-08
[2023-07-21 15:36:33,844] [ INFO] - algo_list :None
[2023-07-21 15:36:33,844] [ INFO] - batch_num_list :None
[2023-07-21 15:36:33,844] [ INFO] - batch_size_list :None
[2023-07-21 15:36:33,844] [ INFO] - bf16 :False
[2023-07-21 15:36:33,844] [ INFO] - bf16_full_eval :False
[2023-07-21 15:36:33,844] [ INFO] - bias_correction :False
[2023-07-21 15:36:33,844] [ INFO] - current_device :gpu:0
[2023-07-21 15:36:33,844] [ INFO] - dataloader_drop_last :False
[2023-07-21 15:36:33,844] [ INFO] - dataloader_num_workers :0
[2023-07-21 15:36:33,845] [ INFO] - device :gpu
[2023-07-21 15:36:33,845] [ INFO] - disable_tqdm :True
[2023-07-21 15:36:33,845] [ INFO] - do_compress :False
[2023-07-21 15:36:33,845] [ INFO] - do_eval :True
[2023-07-21 15:36:33,845] [ INFO] - do_export :True
[2023-07-21 15:36:33,845] [ INFO] - do_predict :False
[2023-07-21 15:36:33,845] [ INFO] - do_train :True
[2023-07-21 15:36:33,845] [ INFO] - eval_batch_size :16
[2023-07-21 15:36:33,845] [ INFO] - eval_steps :25
[2023-07-21 15:36:33,845] [ INFO] - evaluation_strategy :IntervalStrategy.STEPS
[2023-07-21 15:36:33,845] [ INFO] - flatten_param_grads :False
[2023-07-21 15:36:33,845] [ INFO] - fp16 :False
[2023-07-21 15:36:33,845] [ INFO] - fp16_full_eval :False
[2023-07-21 15:36:33,845] [ INFO] - fp16_opt_level :O1
[2023-07-21 15:36:33,845] [ INFO] - gradient_accumulation_steps :1
[2023-07-21 15:36:33,845] [ INFO] - greater_is_better :True
[2023-07-21 15:36:33,845] [ INFO] - ignore_data_skip :False
[2023-07-21 15:36:33,845] [ INFO] - input_dtype :int64
[2023-07-21 15:36:33,845] [ INFO] - input_infer_model_path :None
[2023-07-21 15:36:33,845] [ INFO] - label_names :['start_positions', 'end_positions']
[2023-07-21 15:36:33,845] [ INFO] - lazy_data_processing :True
[2023-07-21 15:36:33,845] [ INFO] - learning_rate :1e-05
[2023-07-21 15:36:33,845] [ INFO] - load_best_model_at_end :True
[2023-07-21 15:36:33,845] [ INFO] - local_process_index :0
[2023-07-21 15:36:33,845] [ INFO] - local_rank :-1
[2023-07-21 15:36:33,845] [ INFO] - log_level :-1
[2023-07-21 15:36:33,845] [ INFO] - log_level_replica :-1
[2023-07-21 15:36:33,846] [ INFO] - log_on_each_node :True
[2023-07-21 15:36:33,846] [ INFO] - logging_dir :./checkpoint/model_best/runs/Jul21_15-36-09_jupyter-2631487-6518069
[2023-07-21 15:36:33,846] [ INFO] - logging_first_step :False
[2023-07-21 15:36:33,846] [ INFO] - logging_steps :5
[2023-07-21 15:36:33,846] [ INFO] - logging_strategy :IntervalStrategy.STEPS
[2023-07-21 15:36:33,846] [ INFO] - lr_scheduler_type :SchedulerType.LINEAR
[2023-07-21 15:36:33,846] [ INFO] - max_grad_norm :1.0
[2023-07-21 15:36:33,846] [ INFO] - max_steps :-1
[2023-07-21 15:36:33,846] [ INFO] - metric_for_best_model :eval_f1
[2023-07-21 15:36:33,846] [ INFO] - minimum_eval_times :None
[2023-07-21 15:36:33,846] [ INFO] - moving_rate :0.9
[2023-07-21 15:36:33,846] [ INFO] - no_cuda :False
[2023-07-21 15:36:33,846] [ INFO] - num_train_epochs :5.0
[2023-07-21 15:36:33,846] [ INFO] - onnx_format :True
[2023-07-21 15:36:33,846] [ INFO] - optim :OptimizerNames.ADAMW
[2023-07-21 15:36:33,846] [ INFO] - output_dir :./checkpoint/model_best
[2023-07-21 15:36:33,846] [ INFO] - overwrite_output_dir :True
[2023-07-21 15:36:33,846] [ INFO] - past_index :-1
[2023-07-21 15:36:33,846] [ INFO] - per_device_eval_batch_size :16
[2023-07-21 15:36:33,846] [ INFO] - per_device_train_batch_size :16
[2023-07-21 15:36:33,846] [ INFO] - prediction_loss_only :False
[2023-07-21 15:36:33,846] [ INFO] - process_index :0
[2023-07-21 15:36:33,846] [ INFO] - prune_embeddings :False
[2023-07-21 15:36:33,846] [ INFO] - recompute :False
[2023-07-21 15:36:33,846] [ INFO] - remove_unused_columns :True
[2023-07-21 15:36:33,846] [ INFO] - report_to :['visualdl']
[2023-07-21 15:36:33,846] [ INFO] - resume_from_checkpoint :None
[2023-07-21 15:36:33,846] [ INFO] - round_type :round
[2023-07-21 15:36:33,847] [ INFO] - run_name :./checkpoint/model_best
[2023-07-21 15:36:33,847] [ INFO] - save_on_each_node :False
[2023-07-21 15:36:33,847] [ INFO] - save_steps :25
[2023-07-21 15:36:33,847] [ INFO] - save_strategy :IntervalStrategy.STEPS
[2023-07-21 15:36:33,847] [ INFO] - save_total_limit :1
[2023-07-21 15:36:33,847] [ INFO] - scale_loss :32768
[2023-07-21 15:36:33,847] [ INFO] - seed :42
[2023-07-21 15:36:33,847] [ INFO] - sharding :[]
[2023-07-21 15:36:33,847] [ INFO] - sharding_degree :-1
[2023-07-21 15:36:33,847] [ INFO] - should_log :True
[2023-07-21 15:36:33,847] [ INFO] - should_save :True
[2023-07-21 15:36:33,847] [ INFO] - skip_memory_metrics :True
[2023-07-21 15:36:33,847] [ INFO] - strategy :dynabert+ptq
[2023-07-21 15:36:33,847] [ INFO] - train_batch_size :16
[2023-07-21 15:36:33,847] [ INFO] - use_pact :True
[2023-07-21 15:36:33,847] [ INFO] - warmup_ratio :0.1
[2023-07-21 15:36:33,847] [ INFO] - warmup_steps :0
[2023-07-21 15:36:33,847] [ INFO] - weight_decay :0.0
[2023-07-21 15:36:33,847] [ INFO] - weight_quantize_type :channel_wise_abs_max
[2023-07-21 15:36:33,847] [ INFO] - width_mult_list :None
[2023-07-21 15:36:33,847] [ INFO] - world_size :1
[2023-07-21 15:36:33,847] [ INFO] -
[2023-07-21 15:36:33,849] [ INFO] - ***** Running training *****
[2023-07-21 15:36:33,849] [ INFO] - Num examples = 686
[2023-07-21 15:36:33,849] [ INFO] - Num Epochs = 5
[2023-07-21 15:36:33,849] [ INFO] - Instantaneous batch size per device = 16
[2023-07-21 15:36:33,849] [ INFO] - Total train batch size (w. parallel, distributed & accumulation) = 16
[2023-07-21 15:36:33,849] [ INFO] - Gradient Accumulation steps = 1
[2023-07-21 15:36:33,849] [ INFO] - Total optimization steps = 215.0
[2023-07-21 15:36:33,849] [ INFO] - Total num train samples = 3430.0
[2023-07-21 15:36:33,856] [ INFO] - Number of trainable parameters = 281693122
[2023-07-21 15:36:55,804] [ INFO] - loss: 0.00139983, learning_rate: 1e-05, global_step: 5, interval_runtime: 21.9466, interval_samples_per_second: 3.645, interval_steps_per_second: 0.228, epoch: 0.1163
[2023-07-21 15:37:17,246] [ INFO] - loss: 0.00095238, learning_rate: 1e-05, global_step: 10, interval_runtime: 21.4431, interval_samples_per_second: 3.731, interval_steps_per_second: 0.233, epoch: 0.2326
[2023-07-21 15:37:38,397] [ INFO] - loss: 0.00227169, learning_rate: 1e-05, global_step: 15, interval_runtime: 21.1288, interval_samples_per_second: 3.786, interval_steps_per_second: 0.237, epoch: 0.3488
[2023-07-21 15:37:59,719] [ INFO] - loss: 0.00058537, learning_rate: 1e-05, global_step: 20, interval_runtime: 21.3431, interval_samples_per_second: 3.748, interval_steps_per_second: 0.234, epoch: 0.4651
[2023-07-21 15:38:20,879] [ INFO] - loss: 0.00099298, learning_rate: 1e-05, global_step: 25, interval_runtime: 21.1605, interval_samples_per_second: 3.781, interval_steps_per_second: 0.236, epoch: 0.5814
[2023-07-21 15:38:20,879] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:38:20,880] [ INFO] - Num examples = 35
[2023-07-21 15:38:20,880] [ INFO] - Total prediction steps = 3
[2023-07-21 15:38:20,880] [ INFO] - Pre device batch size = 16
[2023-07-21 15:38:20,880] [ INFO] - Total Batch size = 16
[2023-07-21 15:38:31,387] [ INFO] - eval_loss: 0.0014212249079719186, eval_precision: 0.9344262295081968, eval_recall: 0.9047619047619048, eval_f1: 0.9193548387096775, eval_runtime: 10.5013, eval_samples_per_second: 3.333, eval_steps_per_second: 0.286, epoch: 0.5814
[2023-07-21 15:38:31,387] [ INFO] - Saving model checkpoint to ./checkpoint/model_best/checkpoint-25
[2023-07-21 15:38:31,390] [ INFO] - Configuration saved in ./checkpoint/model_best/checkpoint-25/config.json
[2023-07-21 15:38:33,536] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/checkpoint-25/tokenizer_config.json
[2023-07-21 15:38:33,537] [ INFO] - Special tokens file saved in ./checkpoint/model_best/checkpoint-25/special_tokens_map.json
[2023-07-21 15:38:46,593] [ INFO] - loss: 0.00054665, learning_rate: 1e-05, global_step: 30, interval_runtime: 25.7138, interval_samples_per_second: 3.111, interval_steps_per_second: 0.194, epoch: 0.6977
[2023-07-21 15:39:07,860] [ INFO] - loss: 0.00042223, learning_rate: 1e-05, global_step: 35, interval_runtime: 21.2605, interval_samples_per_second: 3.763, interval_steps_per_second: 0.235, epoch: 0.814
[2023-07-21 15:39:29,450] [ INFO] - loss: 0.00070746, learning_rate: 1e-05, global_step: 40, interval_runtime: 21.5964, interval_samples_per_second: 3.704, interval_steps_per_second: 0.232, epoch: 0.9302
[2023-07-21 15:39:50,745] [ INFO] - loss: 0.00027768, learning_rate: 1e-05, global_step: 45, interval_runtime: 21.2946, interval_samples_per_second: 3.757, interval_steps_per_second: 0.235, epoch: 1.0465
[2023-07-21 15:40:12,219] [ INFO] - loss: 0.00037302, learning_rate: 1e-05, global_step: 50, interval_runtime: 21.4753, interval_samples_per_second: 3.725, interval_steps_per_second: 0.233, epoch: 1.1628
[2023-07-21 15:40:12,220] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:40:12,220] [ INFO] - Num examples = 35
[2023-07-21 15:40:12,220] [ INFO] - Total prediction steps = 3
[2023-07-21 15:40:12,220] [ INFO] - Pre device batch size = 16
[2023-07-21 15:40:12,221] [ INFO] - Total Batch size = 16
[2023-07-21 15:40:22,304] [ INFO] - eval_loss: 0.0014475114876404405, eval_precision: 0.9482758620689655, eval_recall: 0.873015873015873, eval_f1: 0.9090909090909091, eval_runtime: 10.0828, eval_samples_per_second: 3.471, eval_steps_per_second: 0.298, epoch: 1.1628
[2023-07-21 15:40:22,305] [ INFO] - Saving model checkpoint to ./checkpoint/model_best/checkpoint-50
[2023-07-21 15:40:22,308] [ INFO] - Configuration saved in ./checkpoint/model_best/checkpoint-50/config.json
[2023-07-21 15:40:24,464] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/checkpoint-50/tokenizer_config.json
[2023-07-21 15:40:24,465] [ INFO] - Special tokens file saved in ./checkpoint/model_best/checkpoint-50/special_tokens_map.json
[2023-07-21 15:40:37,740] [ INFO] - loss: 0.00019248, learning_rate: 1e-05, global_step: 55, interval_runtime: 25.5206, interval_samples_per_second: 3.135, interval_steps_per_second: 0.196, epoch: 1.2791
[2023-07-21 15:40:58,905] [ INFO] - loss: 0.00021258, learning_rate: 1e-05, global_step: 60, interval_runtime: 21.1645, interval_samples_per_second: 3.78, interval_steps_per_second: 0.236, epoch: 1.3953
[2023-07-21 15:41:20,213] [ INFO] - loss: 0.00024681, learning_rate: 1e-05, global_step: 65, interval_runtime: 21.3084, interval_samples_per_second: 3.754, interval_steps_per_second: 0.235, epoch: 1.5116
[2023-07-21 15:41:41,237] [ INFO] - loss: 0.000169, learning_rate: 1e-05, global_step: 70, interval_runtime: 21.024, interval_samples_per_second: 3.805, interval_steps_per_second: 0.238, epoch: 1.6279
[2023-07-21 15:42:02,163] [ INFO] - loss: 0.00036645, learning_rate: 1e-05, global_step: 75, interval_runtime: 20.9256, interval_samples_per_second: 3.823, interval_steps_per_second: 0.239, epoch: 1.7442
[2023-07-21 15:42:02,163] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:42:02,163] [ INFO] - Num examples = 35
[2023-07-21 15:42:02,164] [ INFO] - Total prediction steps = 3
[2023-07-21 15:42:02,164] [ INFO] - Pre device batch size = 16
[2023-07-21 15:42:02,164] [ INFO] - Total Batch size = 16
[2023-07-21 15:42:12,158] [ INFO] - eval_loss: 0.001322056632488966, eval_precision: 0.9508196721311475, eval_recall: 0.9206349206349206, eval_f1: 0.9354838709677418, eval_runtime: 9.9708, eval_samples_per_second: 3.51, eval_steps_per_second: 0.301, epoch: 1.7442
[2023-07-21 15:42:12,159] [ INFO] - Saving model checkpoint to ./checkpoint/model_best/checkpoint-75
[2023-07-21 15:42:12,161] [ INFO] - Configuration saved in ./checkpoint/model_best/checkpoint-75/config.json
[2023-07-21 15:42:14,264] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/checkpoint-75/tokenizer_config.json
[2023-07-21 15:42:14,264] [ INFO] - Special tokens file saved in ./checkpoint/model_best/checkpoint-75/special_tokens_map.json
[2023-07-21 15:42:18,485] [ INFO] - Deleting older checkpoint [checkpoint/model_best/checkpoint-25] due to args.save_total_limit
[2023-07-21 15:42:27,793] [ INFO] - loss: 0.00060927, learning_rate: 1e-05, global_step: 80, interval_runtime: 25.6304, interval_samples_per_second: 3.121, interval_steps_per_second: 0.195, epoch: 1.8605
[2023-07-21 15:42:48,729] [ INFO] - loss: 0.00068383, learning_rate: 1e-05, global_step: 85, interval_runtime: 20.9361, interval_samples_per_second: 3.821, interval_steps_per_second: 0.239, epoch: 1.9767
[2023-07-21 15:43:09,835] [ INFO] - loss: 0.00042777, learning_rate: 1e-05, global_step: 90, interval_runtime: 21.1056, interval_samples_per_second: 3.79, interval_steps_per_second: 0.237, epoch: 2.093
[2023-07-21 15:43:30,942] [ INFO] - loss: 0.00013877, learning_rate: 1e-05, global_step: 95, interval_runtime: 21.1075, interval_samples_per_second: 3.79, interval_steps_per_second: 0.237, epoch: 2.2093
[2023-07-21 15:43:52,187] [ INFO] - loss: 0.00042886, learning_rate: 1e-05, global_step: 100, interval_runtime: 21.2446, interval_samples_per_second: 3.766, interval_steps_per_second: 0.235, epoch: 2.3256
[2023-07-21 15:43:52,188] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:43:52,188] [ INFO] - Num examples = 35
[2023-07-21 15:43:52,188] [ INFO] - Total prediction steps = 3
[2023-07-21 15:43:52,188] [ INFO] - Pre device batch size = 16
[2023-07-21 15:43:52,188] [ INFO] - Total Batch size = 16
[2023-07-21 15:44:02,369] [ INFO] - eval_loss: 0.001290834159590304, eval_precision: 0.9508196721311475, eval_recall: 0.9206349206349206, eval_f1: 0.9354838709677418, eval_runtime: 10.1799, eval_samples_per_second: 3.438, eval_steps_per_second: 0.295, epoch: 2.3256
[2023-07-21 15:44:02,369] [ INFO] - Saving model checkpoint to ./checkpoint/model_best/checkpoint-100
[2023-07-21 15:44:02,371] [ INFO] - Configuration saved in ./checkpoint/model_best/checkpoint-100/config.json
[2023-07-21 15:44:04,511] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/checkpoint-100/tokenizer_config.json
[2023-07-21 15:44:04,511] [ INFO] - Special tokens file saved in ./checkpoint/model_best/checkpoint-100/special_tokens_map.json
[2023-07-21 15:44:08,763] [ INFO] - Deleting older checkpoint [checkpoint/model_best/checkpoint-50] due to args.save_total_limit
[2023-07-21 15:44:17,868] [ INFO] - loss: 0.00011366, learning_rate: 1e-05, global_step: 105, interval_runtime: 25.6806, interval_samples_per_second: 3.115, interval_steps_per_second: 0.195, epoch: 2.4419
[2023-07-21 15:44:39,049] [ INFO] - loss: 4.777e-05, learning_rate: 1e-05, global_step: 110, interval_runtime: 21.1812, interval_samples_per_second: 3.777, interval_steps_per_second: 0.236, epoch: 2.5581
[2023-07-21 15:45:00,245] [ INFO] - loss: 0.00013845, learning_rate: 1e-05, global_step: 115, interval_runtime: 21.1969, interval_samples_per_second: 3.774, interval_steps_per_second: 0.236, epoch: 2.6744
[2023-07-21 15:45:21,118] [ INFO] - loss: 0.00040561, learning_rate: 1e-05, global_step: 120, interval_runtime: 20.8727, interval_samples_per_second: 3.833, interval_steps_per_second: 0.24, epoch: 2.7907
[2023-07-21 15:45:41,985] [ INFO] - loss: 0.00054928, learning_rate: 1e-05, global_step: 125, interval_runtime: 20.8671, interval_samples_per_second: 3.834, interval_steps_per_second: 0.24, epoch: 2.907
[2023-07-21 15:45:41,986] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:45:41,986] [ INFO] - Num examples = 35
[2023-07-21 15:45:41,986] [ INFO] - Total prediction steps = 3
[2023-07-21 15:45:41,986] [ INFO] - Pre device batch size = 16
[2023-07-21 15:45:41,986] [ INFO] - Total Batch size = 16
[2023-07-21 15:45:52,179] [ INFO] - eval_loss: 0.0013684021541848779, eval_precision: 0.9508196721311475, eval_recall: 0.9206349206349206, eval_f1: 0.9354838709677418, eval_runtime: 10.1923, eval_samples_per_second: 3.434, eval_steps_per_second: 0.294, epoch: 2.907
[2023-07-21 15:45:52,180] [ INFO] - Saving model checkpoint to ./checkpoint/model_best/checkpoint-125
[2023-07-21 15:45:52,182] [ INFO] - Configuration saved in ./checkpoint/model_best/checkpoint-125/config.json
[2023-07-21 15:45:54,324] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/checkpoint-125/tokenizer_config.json
[2023-07-21 15:45:54,324] [ INFO] - Special tokens file saved in ./checkpoint/model_best/checkpoint-125/special_tokens_map.json
[2023-07-21 15:45:58,570] [ INFO] - Deleting older checkpoint [checkpoint/model_best/checkpoint-100] due to args.save_total_limit
[2023-07-21 15:46:07,445] [ INFO] - loss: 5.219e-05, learning_rate: 1e-05, global_step: 130, interval_runtime: 25.4597, interval_samples_per_second: 3.142, interval_steps_per_second: 0.196, epoch: 3.0233
[2023-07-21 15:46:28,712] [ INFO] - loss: 0.00026077, learning_rate: 1e-05, global_step: 135, interval_runtime: 21.2671, interval_samples_per_second: 3.762, interval_steps_per_second: 0.235, epoch: 3.1395
[2023-07-21 15:46:49,731] [ INFO] - loss: 6.99e-05, learning_rate: 1e-05, global_step: 140, interval_runtime: 21.0185, interval_samples_per_second: 3.806, interval_steps_per_second: 0.238, epoch: 3.2558
[2023-07-21 15:47:10,751] [ INFO] - loss: 0.00023049, learning_rate: 1e-05, global_step: 145, interval_runtime: 21.0205, interval_samples_per_second: 3.806, interval_steps_per_second: 0.238, epoch: 3.3721
[2023-07-21 15:47:31,889] [ INFO] - loss: 0.00015275, learning_rate: 1e-05, global_step: 150, interval_runtime: 21.1372, interval_samples_per_second: 3.785, interval_steps_per_second: 0.237, epoch: 3.4884
[2023-07-21 15:47:31,889] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:47:31,889] [ INFO] - Num examples = 35
[2023-07-21 15:47:31,889] [ INFO] - Total prediction steps = 3
[2023-07-21 15:47:31,890] [ INFO] - Pre device batch size = 16
[2023-07-21 15:47:31,890] [ INFO] - Total Batch size = 16
[2023-07-21 15:47:42,271] [ INFO] - eval_loss: 0.0013476903550326824, eval_precision: 0.9508196721311475, eval_recall: 0.9206349206349206, eval_f1: 0.9354838709677418, eval_runtime: 10.3813, eval_samples_per_second: 3.371, eval_steps_per_second: 0.289, epoch: 3.4884
[2023-07-21 15:47:42,272] [ INFO] - Saving model checkpoint to ./checkpoint/model_best/checkpoint-150
[2023-07-21 15:47:42,274] [ INFO] - Configuration saved in ./checkpoint/model_best/checkpoint-150/config.json
[2023-07-21 15:47:44,424] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/checkpoint-150/tokenizer_config.json
[2023-07-21 15:47:44,424] [ INFO] - Special tokens file saved in ./checkpoint/model_best/checkpoint-150/special_tokens_map.json
[2023-07-21 15:47:48,728] [ INFO] - Deleting older checkpoint [checkpoint/model_best/checkpoint-125] due to args.save_total_limit
[2023-07-21 15:47:57,472] [ INFO] - loss: 0.00024907, learning_rate: 1e-05, global_step: 155, interval_runtime: 25.5832, interval_samples_per_second: 3.127, interval_steps_per_second: 0.195, epoch: 3.6047
[2023-07-21 15:48:18,254] [ INFO] - loss: 0.00027028, learning_rate: 1e-05, global_step: 160, interval_runtime: 20.7824, interval_samples_per_second: 3.849, interval_steps_per_second: 0.241, epoch: 3.7209
[2023-07-21 15:48:39,309] [ INFO] - loss: 0.0001771, learning_rate: 1e-05, global_step: 165, interval_runtime: 21.0551, interval_samples_per_second: 3.8, interval_steps_per_second: 0.237, epoch: 3.8372
[2023-07-21 15:49:00,354] [ INFO] - loss: 0.00024041, learning_rate: 1e-05, global_step: 170, interval_runtime: 21.0449, interval_samples_per_second: 3.801, interval_steps_per_second: 0.238, epoch: 3.9535
[2023-07-21 15:49:21,382] [ INFO] - loss: 4.51e-05, learning_rate: 1e-05, global_step: 175, interval_runtime: 21.0273, interval_samples_per_second: 3.805, interval_steps_per_second: 0.238, epoch: 4.0698
[2023-07-21 15:49:21,382] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:49:21,382] [ INFO] - Num examples = 35
[2023-07-21 15:49:21,382] [ INFO] - Total prediction steps = 3
[2023-07-21 15:49:21,382] [ INFO] - Pre device batch size = 16
[2023-07-21 15:49:21,382] [ INFO] - Total Batch size = 16
[2023-07-21 15:49:31,953] [ INFO] - eval_loss: 0.0013263615546748042, eval_precision: 0.9508196721311475, eval_recall: 0.9206349206349206, eval_f1: 0.9354838709677418, eval_runtime: 10.57, eval_samples_per_second: 3.311, eval_steps_per_second: 0.284, epoch: 4.0698
[2023-07-21 15:49:31,954] [ INFO] - Saving model checkpoint to ./checkpoint/model_best/checkpoint-175
[2023-07-21 15:49:31,956] [ INFO] - Configuration saved in ./checkpoint/model_best/checkpoint-175/config.json
[2023-07-21 15:49:34,699] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/checkpoint-175/tokenizer_config.json
[2023-07-21 15:49:34,700] [ INFO] - Special tokens file saved in ./checkpoint/model_best/checkpoint-175/special_tokens_map.json
[2023-07-21 15:49:40,286] [ INFO] - Deleting older checkpoint [checkpoint/model_best/checkpoint-150] due to args.save_total_limit
[2023-07-21 15:49:48,671] [ INFO] - loss: 0.0003263, learning_rate: 1e-05, global_step: 180, interval_runtime: 27.2898, interval_samples_per_second: 2.931, interval_steps_per_second: 0.183, epoch: 4.186
[2023-07-21 15:50:09,486] [ INFO] - loss: 0.00014406, learning_rate: 1e-05, global_step: 185, interval_runtime: 20.8144, interval_samples_per_second: 3.843, interval_steps_per_second: 0.24, epoch: 4.3023
[2023-07-21 15:50:31,097] [ INFO] - loss: 0.00010923, learning_rate: 1e-05, global_step: 190, interval_runtime: 21.6107, interval_samples_per_second: 3.702, interval_steps_per_second: 0.231, epoch: 4.4186
[2023-07-21 15:50:52,282] [ INFO] - loss: 8.216e-05, learning_rate: 1e-05, global_step: 195, interval_runtime: 21.1856, interval_samples_per_second: 3.776, interval_steps_per_second: 0.236, epoch: 4.5349
[2023-07-21 15:51:14,299] [ INFO] - loss: 9.251e-05, learning_rate: 1e-05, global_step: 200, interval_runtime: 22.0164, interval_samples_per_second: 3.634, interval_steps_per_second: 0.227, epoch: 4.6512
[2023-07-21 15:51:14,299] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:51:14,299] [ INFO] - Num examples = 35
[2023-07-21 15:51:14,299] [ INFO] - Total prediction steps = 3
[2023-07-21 15:51:14,299] [ INFO] - Pre device batch size = 16
[2023-07-21 15:51:14,300] [ INFO] - Total Batch size = 16
[2023-07-21 15:51:24,773] [ INFO] - eval_loss: 0.0014609990175813437, eval_precision: 0.9508196721311475, eval_recall: 0.9206349206349206, eval_f1: 0.9354838709677418, eval_runtime: 10.4732, eval_samples_per_second: 3.342, eval_steps_per_second: 0.286, epoch: 4.6512
[2023-07-21 15:51:24,774] [ INFO] - Saving model checkpoint to ./checkpoint/model_best/checkpoint-200
[2023-07-21 15:51:24,776] [ INFO] - Configuration saved in ./checkpoint/model_best/checkpoint-200/config.json
[2023-07-21 15:51:27,228] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/checkpoint-200/tokenizer_config.json
[2023-07-21 15:51:27,228] [ INFO] - Special tokens file saved in ./checkpoint/model_best/checkpoint-200/special_tokens_map.json
[2023-07-21 15:51:32,347] [ INFO] - Deleting older checkpoint [checkpoint/model_best/checkpoint-175] due to args.save_total_limit
[2023-07-21 15:51:41,379] [ INFO] - loss: 0.00016781, learning_rate: 1e-05, global_step: 205, interval_runtime: 27.0808, interval_samples_per_second: 2.954, interval_steps_per_second: 0.185, epoch: 4.7674
[2023-07-21 15:52:03,510] [ INFO] - loss: 0.00013611, learning_rate: 1e-05, global_step: 210, interval_runtime: 22.1302, interval_samples_per_second: 3.615, interval_steps_per_second: 0.226, epoch: 4.8837
[2023-07-21 15:52:23,996] [ INFO] - loss: 0.0001641, learning_rate: 1e-05, global_step: 215, interval_runtime: 20.4867, interval_samples_per_second: 3.905, interval_steps_per_second: 0.244, epoch: 5.0
[2023-07-21 15:52:23,997] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:52:23,997] [ INFO] - Num examples = 35
[2023-07-21 15:52:23,997] [ INFO] - Total prediction steps = 3
[2023-07-21 15:52:23,997] [ INFO] - Pre device batch size = 16
[2023-07-21 15:52:23,997] [ INFO] - Total Batch size = 16
[2023-07-21 15:52:33,805] [ INFO] - eval_loss: 0.0011874400079250336, eval_precision: 0.9508196721311475, eval_recall: 0.9206349206349206, eval_f1: 0.9354838709677418, eval_runtime: 9.8078, eval_samples_per_second: 3.569, eval_steps_per_second: 0.306, epoch: 5.0
[2023-07-21 15:52:33,806] [ INFO] - Saving model checkpoint to ./checkpoint/model_best/checkpoint-215
[2023-07-21 15:52:33,808] [ INFO] - Configuration saved in ./checkpoint/model_best/checkpoint-215/config.json
[2023-07-21 15:52:36,141] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/checkpoint-215/tokenizer_config.json
[2023-07-21 15:52:36,141] [ INFO] - Special tokens file saved in ./checkpoint/model_best/checkpoint-215/special_tokens_map.json
[2023-07-21 15:52:41,717] [ INFO] - Deleting older checkpoint [checkpoint/model_best/checkpoint-200] due to args.save_total_limit
[2023-07-21 15:52:42,252] [ INFO] -
Training completed.
[2023-07-21 15:52:42,252] [ INFO] - Loading best model from ./checkpoint/model_best/checkpoint-75 (score: 0.9354838709677418).
[2023-07-21 15:52:43,847] [ INFO] - train_runtime: 969.9908, train_samples_per_second: 3.536, train_steps_per_second: 0.222, train_loss: 0.0003774468271267535, epoch: 5.0
[2023-07-21 15:52:43,915] [ INFO] - Saving model checkpoint to ./checkpoint/model_best
[2023-07-21 15:52:43,917] [ INFO] - Configuration saved in ./checkpoint/model_best/config.json
[2023-07-21 15:52:46,306] [ INFO] - tokenizer config file saved in ./checkpoint/model_best/tokenizer_config.json
[2023-07-21 15:52:46,306] [ INFO] - Special tokens file saved in ./checkpoint/model_best/special_tokens_map.json
[2023-07-21 15:52:46,314] [ INFO] - ***** train metrics *****
[2023-07-21 15:52:46,315] [ INFO] - epoch = 5.0
[2023-07-21 15:52:46,315] [ INFO] - train_loss = 0.0004
[2023-07-21 15:52:46,315] [ INFO] - train_runtime = 0:16:09.99
[2023-07-21 15:52:46,315] [ INFO] - train_samples_per_second = 3.536
[2023-07-21 15:52:46,315] [ INFO] - train_steps_per_second = 0.222
[2023-07-21 15:52:46,318] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:52:46,318] [ INFO] - Num examples = 35
[2023-07-21 15:52:46,318] [ INFO] - Total prediction steps = 3
[2023-07-21 15:52:46,318] [ INFO] - Pre device batch size = 16
[2023-07-21 15:52:46,318] [ INFO] - Total Batch size = 16
[2023-07-21 15:52:55,755] [ INFO] - eval_loss: 0.001322056632488966, eval_precision: 0.9508196721311475, eval_recall: 0.9206349206349206, eval_f1: 0.9354838709677418, eval_runtime: 9.4374, eval_samples_per_second: 3.709, eval_steps_per_second: 0.318, epoch: 5.0
[2023-07-21 15:52:55,756] [ INFO] - ***** eval metrics *****
[2023-07-21 15:52:55,756] [ INFO] - epoch = 5.0
[2023-07-21 15:52:55,756] [ INFO] - eval_f1 = 0.9355
[2023-07-21 15:52:55,756] [ INFO] - eval_loss = 0.0013
[2023-07-21 15:52:55,756] [ INFO] - eval_precision = 0.9508
[2023-07-21 15:52:55,756] [ INFO] - eval_recall = 0.9206
[2023-07-21 15:52:55,756] [ INFO] - eval_runtime = 0:00:09.43
[2023-07-21 15:52:55,756] [ INFO] - eval_samples_per_second = 3.709
[2023-07-21 15:52:55,756] [ INFO] - eval_steps_per_second = 0.318
[2023-07-21 15:52:55,759] [ INFO] - Exporting inference model to ./checkpoint/model_best/model
[2023-07-21 15:53:55,567] [ INFO] - Inference model exported.
6.模型评估
!python evaluate.py \
--device "gpu" \
--model_path ./checkpoint/model_best \
--test_path ./medical_checklist/dev.txt \
--output_dir ./checkpoint/model_best \
--label_names 'start_positions' 'end_positions'\
--max_seq_len 512 \
--per_device_eval_batch_size 16
[2023-07-21 15:55:25,012] [ INFO] - The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
[2023-07-21 15:55:25,012] [ INFO] - ============================================================
[2023-07-21 15:55:25,013] [ INFO] - Model Configuration Arguments
[2023-07-21 15:55:25,013] [ INFO] - paddle commit id :3fa7a736e32508e797616b6344d97814c37d3ff8
[2023-07-21 15:55:25,013] [ INFO] - model_path :./checkpoint/model_best
[2023-07-21 15:55:25,013] [ INFO] -
[2023-07-21 15:55:25,013] [ INFO] - ============================================================
[2023-07-21 15:55:25,013] [ INFO] - Data Configuration Arguments
[2023-07-21 15:55:25,013] [ INFO] - paddle commit id :3fa7a736e32508e797616b6344d97814c37d3ff8
[2023-07-21 15:55:25,013] [ INFO] - debug :False
[2023-07-21 15:55:25,013] [ INFO] - max_seq_len :512
[2023-07-21 15:55:25,013] [ INFO] - schema_lang :ch
[2023-07-21 15:55:25,013] [ INFO] - test_path :./medical_checklist/dev.txt
[2023-07-21 15:55:25,013] [ INFO] -
[2023-07-21 15:55:25,014] [ INFO] - We are using <class 'paddlenlp.transformers.ernie_layout.tokenizer.ErnieLayoutTokenizer'> to load './checkpoint/model_best'.
[2023-07-21 15:55:25,693] [ INFO] - loading configuration file ./checkpoint/model_best/config.json
[2023-07-21 15:55:25,694] [ INFO] - Model config ErnieLayoutConfig {
"architectures": [
"UIEX"
],
"attention_probs_dropout_prob": 0.1,
"bos_token_id": 0,
"coordinate_size": 128,
"dtype": "float32",
"enable_recompute": false,
"eos_token_id": 2,
"fuse": false,
"gradient_checkpointing": false,
"has_relative_attention_bias": true,
"has_spatial_attention_bias": true,
"has_visual_segment_embedding": false,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 768,
"image_feature_pool_shape": [
7,
7,
256
],
"initializer_range": 0.02,
"intermediate_size": 3072,
"layer_norm_eps": 1e-12,
"max_2d_position_embeddings": 1024,
"max_position_embeddings": 514,
"max_rel_2d_pos": 256,
"max_rel_pos": 128,
"model_type": "ernie_layout",
"num_attention_heads": 12,
"num_hidden_layers": 12,
"output_past": true,
"pad_token_id": 1,
"paddlenlp_version": null,
"pool_act": "tanh",
"rel_2d_pos_bins": 64,
"rel_pos_bins": 32,
"shape_size": 128,
"task_id": 0,
"task_type_vocab_size": 3,
"type_vocab_size": 100,
"use_task_id": true,
"vocab_size": 250002
}
W0721 15:55:29.126700 3399 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0721 15:55:29.130168 3399 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
[2023-07-21 15:55:31,058] [ INFO] - All model checkpoint weights were used when initializing UIEX.
[2023-07-21 15:55:31,058] [ INFO] - All the weights of UIEX were initialized from the model checkpoint at ./checkpoint/model_best.
If your task is similar to the task the model of the checkpoint was trained on, you can already use UIEX for predictions without further training.
[2023-07-21 15:55:31,259] [ INFO] - ============================================================
[2023-07-21 15:55:31,259] [ INFO] - Training Configuration Arguments
[2023-07-21 15:55:31,259] [ INFO] - paddle commit id :3fa7a736e32508e797616b6344d97814c37d3ff8
[2023-07-21 15:55:31,260] [ INFO] - _no_sync_in_gradient_accumulation:True
[2023-07-21 15:55:31,260] [ INFO] - adam_beta1 :0.9
[2023-07-21 15:55:31,260] [ INFO] - adam_beta2 :0.999
[2023-07-21 15:55:31,260] [ INFO] - adam_epsilon :1e-08
[2023-07-21 15:55:31,260] [ INFO] - bf16 :False
[2023-07-21 15:55:31,260] [ INFO] - bf16_full_eval :False
[2023-07-21 15:55:31,260] [ INFO] - current_device :gpu:0
[2023-07-21 15:55:31,260] [ INFO] - dataloader_drop_last :False
[2023-07-21 15:55:31,260] [ INFO] - dataloader_num_workers :0
[2023-07-21 15:55:31,260] [ INFO] - device :gpu
[2023-07-21 15:55:31,260] [ INFO] - disable_tqdm :False
[2023-07-21 15:55:31,260] [ INFO] - do_eval :False
[2023-07-21 15:55:31,260] [ INFO] - do_export :False
[2023-07-21 15:55:31,260] [ INFO] - do_predict :False
[2023-07-21 15:55:31,260] [ INFO] - do_train :False
[2023-07-21 15:55:31,260] [ INFO] - eval_batch_size :16
[2023-07-21 15:55:31,261] [ INFO] - eval_steps :None
[2023-07-21 15:55:31,261] [ INFO] - evaluation_strategy :IntervalStrategy.NO
[2023-07-21 15:55:31,261] [ INFO] - flatten_param_grads :False
[2023-07-21 15:55:31,261] [ INFO] - fp16 :False
[2023-07-21 15:55:31,261] [ INFO] - fp16_full_eval :False
[2023-07-21 15:55:31,261] [ INFO] - fp16_opt_level :O1
[2023-07-21 15:55:31,261] [ INFO] - gradient_accumulation_steps :1
[2023-07-21 15:55:31,261] [ INFO] - greater_is_better :None
[2023-07-21 15:55:31,261] [ INFO] - ignore_data_skip :False
[2023-07-21 15:55:31,261] [ INFO] - label_names :['start_positions', 'end_positions']
[2023-07-21 15:55:31,261] [ INFO] - lazy_data_processing :True
[2023-07-21 15:55:31,261] [ INFO] - learning_rate :5e-05
[2023-07-21 15:55:31,261] [ INFO] - load_best_model_at_end :False
[2023-07-21 15:55:31,261] [ INFO] - local_process_index :0
[2023-07-21 15:55:31,261] [ INFO] - local_rank :-1
[2023-07-21 15:55:31,261] [ INFO] - log_level :-1
[2023-07-21 15:55:31,261] [ INFO] - log_level_replica :-1
[2023-07-21 15:55:31,261] [ INFO] - log_on_each_node :True
[2023-07-21 15:55:31,261] [ INFO] - logging_dir :./checkpoint/model_best/runs/Jul21_15-55-25_jupyter-2631487-6518069
[2023-07-21 15:55:31,262] [ INFO] - logging_first_step :False
[2023-07-21 15:55:31,262] [ INFO] - logging_steps :500
[2023-07-21 15:55:31,262] [ INFO] - logging_strategy :IntervalStrategy.STEPS
[2023-07-21 15:55:31,262] [ INFO] - lr_scheduler_type :SchedulerType.LINEAR
[2023-07-21 15:55:31,262] [ INFO] - max_grad_norm :1.0
[2023-07-21 15:55:31,262] [ INFO] - max_steps :-1
[2023-07-21 15:55:31,262] [ INFO] - metric_for_best_model :None
[2023-07-21 15:55:31,262] [ INFO] - minimum_eval_times :None
[2023-07-21 15:55:31,262] [ INFO] - no_cuda :False
[2023-07-21 15:55:31,262] [ INFO] - num_train_epochs :3.0
[2023-07-21 15:55:31,262] [ INFO] - optim :OptimizerNames.ADAMW
[2023-07-21 15:55:31,262] [ INFO] - output_dir :./checkpoint/model_best
[2023-07-21 15:55:31,262] [ INFO] - overwrite_output_dir :False
[2023-07-21 15:55:31,262] [ INFO] - past_index :-1
[2023-07-21 15:55:31,262] [ INFO] - per_device_eval_batch_size :16
[2023-07-21 15:55:31,262] [ INFO] - per_device_train_batch_size :8
[2023-07-21 15:55:31,262] [ INFO] - prediction_loss_only :False
[2023-07-21 15:55:31,262] [ INFO] - process_index :0
[2023-07-21 15:55:31,262] [ INFO] - recompute :False
[2023-07-21 15:55:31,262] [ INFO] - remove_unused_columns :True
[2023-07-21 15:55:31,262] [ INFO] - report_to :['visualdl']
[2023-07-21 15:55:31,262] [ INFO] - resume_from_checkpoint :None
[2023-07-21 15:55:31,262] [ INFO] - run_name :./checkpoint/model_best
[2023-07-21 15:55:31,262] [ INFO] - save_on_each_node :False
[2023-07-21 15:55:31,262] [ INFO] - save_steps :500
[2023-07-21 15:55:31,263] [ INFO] - save_strategy :IntervalStrategy.STEPS
[2023-07-21 15:55:31,263] [ INFO] - save_total_limit :None
[2023-07-21 15:55:31,263] [ INFO] - scale_loss :32768
[2023-07-21 15:55:31,263] [ INFO] - seed :42
[2023-07-21 15:55:31,263] [ INFO] - sharding :[]
[2023-07-21 15:55:31,263] [ INFO] - sharding_degree :-1
[2023-07-21 15:55:31,263] [ INFO] - should_log :True
[2023-07-21 15:55:31,263] [ INFO] - should_save :True
[2023-07-21 15:55:31,263] [ INFO] - skip_memory_metrics :True
[2023-07-21 15:55:31,263] [ INFO] - train_batch_size :8
[2023-07-21 15:55:31,263] [ INFO] - warmup_ratio :0.0
[2023-07-21 15:55:31,263] [ INFO] - warmup_steps :0
[2023-07-21 15:55:31,263] [ INFO] - weight_decay :0.0
[2023-07-21 15:55:31,263] [ INFO] - world_size :1
[2023-07-21 15:55:31,263] [ INFO] -
[2023-07-21 15:55:31,263] [ INFO] - ***** Running Evaluation *****
[2023-07-21 15:55:31,263] [ INFO] - Num examples = 35
[2023-07-21 15:55:31,263] [ INFO] - Total prediction steps = 3
[2023-07-21 15:55:31,263] [ INFO] - Pre device batch size = 16
[2023-07-21 15:55:31,264] [ INFO] - Total Batch size = 16
100%|█████████████████████████████████████████████| 3/3 [00:03<00:00, 1.31s/it]
[2023-07-21 15:55:41,222] [ INFO] - -----Evaluate model-------
[2023-07-21 15:55:41,222] [ INFO] - Class Name: ALL CLASSES
[2023-07-21 15:55:41,222] [ INFO] - Evaluation Precision: 0.95082 | Recall: 0.92063 | F1: 0.93548
[2023-07-21 15:55:41,222] [ INFO] - -----------------------------
7.Taskflow一键部署
from pprint import pprint
from paddlenlp import Taskflow
schema = {
'项目名称': [
'结果',
'单位',
'参考范围'
]
}
my_ie = Taskflow("information_extraction", model="uie-x-base", schema=schema, task_path='./checkpoint/model_best')
pprint(my_ie({"doc": "test.jpg"}))
[{'项目名称': [{'bbox': [[417, 598, 764, 653]],
'end': 161,
'probability': 0.9931185709767476,
'relations': {'单位': [{'bbox': [[1383, 603, 1475, 653]],
'end': 170,
'probability': 0.9982062669088805,
'start': 166,
'text': 'ng/L'}],
'参考范围': [{'bbox': [[1603, 603, 1717, 650]],
'end': 175,
'probability': 0.994915152253455,
'start': 170,
'text': '0-0.2'}],
'结果': [{'bbox': [[1055, 608, 1161, 647]],
'end': 166,
'probability': 0.9779773840612904,
'start': 161,
'text': '0.000'}]},
'start': 150,
'text': '乙肝表面抗原HBsAg'},
{'bbox': [[420, 803, 807, 850]],
'end': 263,
'probability': 0.9839514684545492,
'relations': {'单位': [{'bbox': [[1382, 800, 1481, 856]],
'end': 272,
'probability': 0.9902134016753692,
'start': 268,
'text': 'U/mL'}],
'参考范围': [{'bbox': [[1609, 806, 1717, 845]],
'end': 277,
'probability': 0.9948578061238109,
'start': 272,
'text': '0-0.2'}],
'结果': [{'bbox': [[1055, 806, 1163, 853]],
'end': 268,
'probability': 0.9997722031372689,
'start': 263,
'text': '0.081'}]},
'start': 248,
'text': '乙肝e抗体Anti-HBeAB'},
{'bbox': [[417, 671, 863, 718]],
'end': 197,
'probability': 0.9933030680080606,
'relations': {'单位': [{'bbox': [[1383, 671, 1512, 717]],
'end': 208,
'probability': 0.993252639775573,
'start': 202,
'text': 'MIU/mL'}],
'参考范围': [{'bbox': [[1603, 671, 1697, 717]],
'end': 212,
'probability': 0.9968451209051636,
'start': 208,
'text': '0-10'}],
'结果': [{'bbox': [[1055, 676, 1163, 715]],
'end': 202,
'probability': 0.9627551951018489,
'start': 197,
'text': '0.000'}]},
'start': 181,
'text': '乙肝表面抗体Anti-HBsAB'},
{'bbox': [[420, 735, 706, 785]],
'end': 228,
'probability': 0.9925530039269148,
'relations': {'单位': [{'bbox': [[1383, 738, 1475, 785]],
'end': 237,
'probability': 0.9953925121749307,
'start': 233,
'text': 'U/mL'}],
'参考范围': [{'bbox': [[1606, 741, 1715, 780]],
'end': 242,
'probability': 0.9982005347972311,
'start': 237,
'text': '0-0.5'}],
'结果': [{'bbox': [[1057, 743, 1163, 782]],
'end': 233,
'probability': 0.9943726871306069,
'start': 228,
'text': '0.000'}]},
'start': 218,
'text': '乙肝e抗原HBeAg'},
{'bbox': [[420, 871, 870, 918]],
'end': 299,
'probability': 0.9931226228703274,
'relations': {'单位': [{'bbox': [[1389, 871, 1477, 918]],
'end': 308,
'probability': 0.9990609045893919,
'start': 304,
'text': 'U/mL'}],
'参考范围': [{'bbox': [[1611, 873, 1717, 912]],
'end': 313,
'probability': 0.9937555165322465,
'start': 308,
'text': '0-0.9'}],
'结果': [{'bbox': [[1054, 867, 1169, 921]],
'end': 304,
'probability': 0.9996564084931308,
'start': 299,
'text': '1.053'}]},
'start': 283,
'text': '乙肝核心抗体Anti-HBcAB'},
{'bbox': [[415, 536, 794, 580]],
'end': 130,
'probability': 0.9905078246100985,
'relations': {'单位': [{'bbox': [[1383, 536, 1475, 585]],
'end': 139,
'probability': 0.9996564019316949,
'start': 135,
'text': 's/co'}],
'参考范围': [{'bbox': [[1603, 533, 1745, 588]],
'end': 144,
'probability': 0.9937541085628041,
'start': 139,
'text': '阴性（-)'}],
'结果': [{'bbox': [[1055, 536, 1194, 582]],
'end': 135,
'probability': 0.9912728416351548,
'start': 130,
'text': '阴性（-）'}]},
'start': 118,
'text': '乙肝病毒前S1抗原HBV'}]}]
图像展示
import matplotlib.pyplot as plt
from paddlenlp.utils.doc_parser import DocParser
results = my_ie({"doc": "test.jpg"})
img_show = DocParser.write_image_with_results(
"test.jpg",
result=results[0],
return_image=True)
plt.figure(figsize=(15,15))
plt.imshow(img_show)
plt.show()
项目地址：https://aistudio.baidu.com/aistudio/projectdetail/6518069?sUid=2631487&shared=1&ts=1690163802670
AI识别检验报告 -PaddleNLP UIE-X 在医疗领域的实战的更多相关文章
- 开发者说 | 云+AI赋能心电医疗领域的应用
以"医工汇聚 智竞心电"为主题的首届中国心电智能大赛自2019年1月1日启动全球招募起,共吸引总计545支来自世界各地的医工结合团队,308支团队近780名选手通过初赛资格审查,经 ...
- 用AI识别内部人威胁面临的道德规范
用AI识别内部人威胁面临的道德规范 还记得汤姆·克鲁斯的<少数派报告>吗?人工智能可识别昭示未来风险的员工行为.该如何有效且有道德地使用这一数据呢? 为保护公司网络不受恶意软件.数据渗漏和 ...
- 什么？小程序实时语音识别你还在痛苦的对接科大讯飞？百度Ai识别？
前言 微信小程序,说不上大火,但是需求还是不少的.各大企业都想插一足 于是前端同学就有事情做了. 需求 我需要录音 我边说话边识别,我要同声传译,我要文字转语音,还要萝莉音 我:??? 正文 一开始, ...
- 从Chat-GPT看爆火技术概念及医疗领域科技与应用场景
作者:京东健康 陈刚 一.前言 最近OpenAI在官网上宣告了多模态大模型 GPT-4 的诞生,它可能是迄今为止最好的多模态模型. 主要更新内容如下: 1. 逻辑分析能力更加全面.「考试」能力大幅提升 ...
- Python通过百度Ai识别图片中的文字
版本:python3.7 工作中有需要识别图片中的汗字,查看了半天大神们的博客,但没找到完全可以用的源码,经过自己的实践,以下源码可以实现: 创建应用 首先你需要登录百度AI,选择文字识别,创建一个应 ...
- 如何利用AI识别未知——加入未知类（不太靠谱），检测待识别数据和已知样本数据的匹配程度（例如使用CNN降维，再用knn类似距离来实现），将问题转化为特征搜索问题而非决策问题，使用HTM算法（记忆+模式匹配预测就是智能），GAN异常检测，RBF
https://www.researchgate.net/post/How_to_determine_unknown_class_using_neural_network 里面有讨论,说是用rbf神经 ...
- 【AI in 美团】深度学习在文本领域的应用
背景 近几年以深度学习技术为核心的人工智能得到广泛的关注,无论是学术界还是工业界,它们都把深度学习作为研究应用的焦点.而深度学习技术突飞猛进的发展离不开海量数据的积累.计算能力的提升和算法模型的改进. ...
- 【立即报名】解码AI大杀器：华为云GPU+Tensorflow 容器实战
导语: 人工智能的火热,带来了一波学习TensorFlow深度学习框架的热潮.聊深度学习免不了要用GPU,但目前GPU费用较高,对于个人学习者和创业公司来讲的话,按需配置的云GPU服务器是一个不错的选 ...
- 基于ERNIELayout&pdfplumber-UIE的多方案学术论文信息抽取
本项目链接:https://aistudio.baidu.com/aistudio/projectdetail/5196032?contributionType=1 基于ERNIELayout& ...
- 数据量不足，MedicalNet 如何助力医疗影像 AI 突破瓶颈？
导读 |近日,云+社区技术沙龙“腾讯开源技术”圆满落幕.本次沙龙邀请了多位腾讯技术专家,深度揭秘了腾讯开源项目TencentOS tiny.TubeMQ.Kona JDK.TARS以及Medical ...
随机推荐
- 【leetcode】258. 各位相加
[leetcode]258. 各位相加 C++解法: class Solution { public: int addDigits(int num) { string s;//用来将num转换成字符串 ...
- Python-HwTestReport的简单使用
一.工具包下载 https://github.com/hongweifuture/HwTestReport(出自此大神) 二.使用示例(直接上代码) 1.将 HwTestReport.py 导入项目 ...
- Node + Express 后台开发 —— 登录标识
登录标识 系统通常只有登录成功后才能访问,而 http 是无状态的.倘若直接请求需要登录才可访问的接口,假如后端反复查询数据库,而且每个请求还得带上用户名和密码,这都是不很好. 作为前端,我们听过 c ...
- 2022-10-09：我们给出了一个（轴对齐的）二维矩形列表 rectangles 。 对于 rectangle[i] = [x1, y1, x2, y2]，其中（x1，y1）是矩形 i 左下角的坐
2022-10-09:我们给出了一个(轴对齐的)二维矩形列表 rectangles . 对于 rectangle[i] = [x1, y1, x2, y2],其中(x1,y1)是矩形 i 左下角的坐标 ...
- 2022-07-16：以下go语言代码输出什么？A：[]；B：[5]；C：[5 0 0 0 0]；D：[0 0 0 0 0]。 package main import ( “fmt“ )
2022-07-16:以下go语言代码输出什么?A:[]:B:[5]:C:[5 0 0 0 0]:D:[0 0 0 0 0]. package main import ( "fmt" ...
- 2022-06-24：golang选择题，以下golang代码输出什么？A：1；B：3；C：4；D：编译失败。 package main import ( “fmt“ ) func mai
2022-06-24:golang选择题,以下golang代码输出什么?A:1:B:3:C:4:D:编译失败. package main import ( "fmt" ) func ...
- 2021-01-21：java中，HashMap的读流程是什么？
福哥答案2020-01-21: jdk1.7读流程:1.key是否为空值null,如果为空,直接遍历table[0]链表,寻找key==null键.调用的是getForNullKey()方法.如下:1 ...
- vue入门第一坑：Eslint
Eslint是语法检查插件,它会严格要求你的代码,就你本身代码没错,但是一运行,Eslint就跳出来报错了.它会自动检查你的代码是否符合规范.所以,建议新手入门Vue创建项目的时候可以关掉Eslint ...
- .cur 图片加载提示 You may need an appropriate loader to handle this file type
最近一个gis 项目需要加载一个.cur的图标,但是编译时提示 You may need an appropriate loader to handle this file type, current ...
- openlayers Text字体大小设置
今做一个app版的ol地图,发现区域太小显示拥挤,于是想把字体改小,看起来匀称点,于是盯紧了font属性使劲改 老是不听咋整呢?网上找大佬 于是找到了得到了各路大神鼎力相助 如: 赶紧跑去试试,原来这 ...