实例分割-Mask R-CNN 模型

进入ModelArts

点击如下链接：https://www.huaweicloud.com/product/modelarts.html ， 进入ModelArts主页。点击“进入控制台”按钮，输入用户名和密码登录，进入ModelArts使用页面

创建ModelArts notebook

下面，我们在ModelArts中创建一个notebook开发环境，ModelArts notebook提供网页版的Python开发环境，可以方便的编写、运行代码，并查看运行结果。

第一步：在ModelArts服务主界面依次点击“开发环境”、“创建”

第二步：填写notebook所需的参数：

第三步：配置好notebook参数后，点击下一步，进入notebook信息预览。确认无误后，点击“立即创建”

第四步：创建完成后，返回开发环境主界面，等待Notebook创建完毕后，打开Notebook，进行下一步操作

在ModelArts中创建开发环境

接下来，我们创建一个实际的开发环境，用于后续的实验步骤。

第一步：点击下图所示的“打开”按钮，进入刚刚创建的Notebook第二步：创建一个Python3环境的的Notebook。点击右上角的\"New\"，然后选择Tensorflow-1.13.1开发环境。 第三步：点击左上方的文件名\"Untitled\"，并输入一个与本实验相关的名称

在Notebook中编写并执行代码

在Notebook中，我们输入一个简单的打印语句，然后点击上方的运行按钮，可以查看语句执行的结果：开发环境准备好啦，接下来可以愉快地写代码啦！

首先进行包的安装与引用

In [1]:

!pip install pycocotools==2.0.0

Collecting pycocotools==2.0.0 Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/96/84/9a07b1095fd8555ba3f3d519517c8743c2554a245f9476e5e39869f948d2/pycocotools-2.0.0.tar.gz (1.5MB) 100% |████████████████████████████████| 1.5MB 65.6MB/s ta 0:00:01 Building wheels for collected packages: pycocotools Running setup.py bdist_wheel for pycocotools ... done Stored in directory: /home/ma-user/.cache/pip/wheels/63/72/9e/bac3d3e23f6b04351d200fa892351da57f0e68c7aeec0b1b08 Successfully built pycocotools Installing collected packages: pycocotools Successfully installed pycocotools-2.0.0 You are using pip version 9.0.1, however version 20.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

In [2]:

!pip install imgaug==0.2.9

Collecting imgaug==0.2.9 Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/17/a9/36de8c0e1ffb2d86f871cac60e5caa910cbbdb5f4741df5ef856c47f4445/imgaug-0.2.9-py2.py3-none-any.whl (753kB) 100% |████████████████████████████████| 757kB 71.1MB/s ta 0:00:011 Requirement already satisfied: Pillow in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from imgaug==0.2.9) Requirement already satisfied: numpy>=1.15.0 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from imgaug==0.2.9) Requirement already satisfied: six in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from imgaug==0.2.9) Requirement already satisfied: imageio in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from imgaug==0.2.9) Requirement already satisfied: opencv-python in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from imgaug==0.2.9) Collecting Shapely (from imgaug==0.2.9) Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/20/fa/c96d3461fda99ed8e82ff0b219ac2c8384694b4e640a611a1a8390ecd415/Shapely-1.7.0-cp36-cp36m-manylinux1_x86_64.whl (1.8MB) 100% |████████████████████████████████| 1.8MB 124.2MB/s ta 0:00:01 | 245kB 72.7MB/s eta 0:00:01 Requirement already satisfied: scikit-image>=0.11.0 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from imgaug==0.2.9) Requirement already satisfied: matplotlib in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from imgaug==0.2.9) Requirement already satisfied: scipy in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from imgaug==0.2.9) Requirement already satisfied: cloudpickle>=0.2.1 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from scikit-image>=0.11.0->imgaug==0.2.9) Requirement already satisfied: networkx>=1.8 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from scikit-image>=0.11.0->imgaug==0.2.9) Requirement already satisfied: PyWavelets>=0.4.0 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from scikit-image>=0.11.0->imgaug==0.2.9) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from matplotlib->imgaug==0.2.9) Requirement already satisfied: cycler>=0.10 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from matplotlib->imgaug==0.2.9) Requirement already satisfied: kiwisolver>=1.0.1 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from matplotlib->imgaug==0.2.9) Requirement already satisfied: pytz in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from matplotlib->imgaug==0.2.9) Requirement already satisfied: python-dateutil>=2.1 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from matplotlib->imgaug==0.2.9) Requirement already satisfied: decorator>=4.1.0 in /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages (from networkx>=1.8->scikit-image>=0.11.0->imgaug==0.2.9) Installing collected packages: Shapely, imgaug Found existing installation: imgaug 0.2.6 Uninstalling imgaug-0.2.6: Successfully uninstalled imgaug-0.2.6 Successfully installed Shapely-1.7.0 imgaug-0.2.9 You are using pip version 9.0.1, however version 20.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

如果使用kernel tf-1.8，则需要取消下面两行代码的注释，并执行。

如果使用kernel tf-1.13，则无需执行下面两行代码。

本脚本右上角显示了当前使用的kernel是tf-1.8还是tf-1.13，如您需切换kernel，请点击本脚本顶部菜单栏Kernel->Change kernel

In [3]:

#!pip uninstall numpy -y #!pip install numpy==1.15.4

对需要的代码和数据进行下载

In [4]:

import os from modelarts.session import Session session = Session() if session.region_name == 'cn-north-1': bucket_path="modelarts-labs/end2end/mask_rcnn/instance_segmentation.tar.gz" elif session.region_name == 'cn-north-4': bucket_path="modelarts-labs-bj4/end2end/mask_rcnn/instance_segmentation.tar.gz" else: print("请更换地区到北京一或北京四") if not os.path.exists('./src/mrcnn'): session.download_data(bucket_path=bucket_path, path='./instance_segmentation.tar.gz') if os.path.exists('./instance_segmentation.tar.gz'): # 使用tar命令解压资源包 os.system("tar zxf ./instance_segmentation.tar.gz") # 清理压缩包 os.system("rm ./instance_segmentation.tar.gz")

Successfully download file modelarts-labs/end2end/mask_rcnn/instance_segmentation.tar.gz from OBS to local ./instance_segmentation.tar.gz

Mask R-CNN模型训练部分

第一步：导入相应的Python库，准备预训练模型

In [5]:

import sys import random import math import re import time import numpy as np import cv2 import matplotlib import matplotlib.pyplot as plt from src.mrcnn.config import Config from src.mrcnn import utils import src.mrcnn.model as modellib from src.mrcnn import visualize from src.mrcnn.model import log %matplotlib inline # Directory to save logs and trained model MODEL_DIR = "logs" # Local path to trained weights file COCO_MODEL_PATH = "data/mask_rcnn_coco.h5"

Using TensorFlow backend.

第二步：生成相关配置项

我们定义Config类的子类MyTrainConfig，指定相关的参数，较为关键的参数有：

NAME： Config的唯一名称

NUM_CLASSES： 分类的数量，COCO中共有80种物体+背景

IMAGE_MIN_DIM和IMAGE_MAX_DIM： 图片的最大和最小尺寸，我们生成固定的128x128的图片，因此都设置为128

TRAIN_ROIS_PER_IMAGE： 每张图片上训练的RoI个数

STEPS_PER_EPOCH和VALIDATION_STEPS： 训练和验证时，每轮的step数量，减少step的数量可以加速训练，但是检测精度降低

In [6]:

class MyTrainConfig(Config): # 可辨识的名称 NAME = "my_train" # GPU的数量和每个GPU处理的图片数量，可以根据实际情况进行调整，参考为Nvidia Tesla P100 GPU_COUNT = 1 IMAGES_PER_GPU = 1 # 物体的分类个数，COCO中共有80种物体+背景 NUM_CLASSES = 1 + 80 # background + 80 shapes # 图片尺寸统一处理为1024，可以根据实际情况再进一步调小 IMAGE_MIN_DIM = 1024 IMAGE_MAX_DIM = 1024 # 因为我们生成的形状图片较小，这里可以使用较小的Anchor进行RoI检测 # RPN_ANCHOR_SCALES = (8, 16, 32, 64, 128) # anchor side in pixels # 每张图片上训练的RoI个数，因为我们生成的图片较小，而且每张图片上的形状较少 #　因此可以适当调小该参数，用较少的Anchor即可覆盖大致的物体信息 TRAIN_ROIS_PER_IMAGE = 200 # 每轮训练的step数量 STEPS_PER_EPOCH = 100 # 每轮验证的step数量 VALIDATION_STEPS = 20 config = MyTrainConfig() config.display()

Configurations: BACKBONE resnet101 BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] COMPUTE_BACKBONE_SHAPE None DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.7 DETECTION_NMS_THRESHOLD 0.3 FPN_CLASSIF_FC_LAYERS_SIZE 1024 GPU_COUNT 1 GRADIENT_CLIP_NORM 5.0 IMAGES_PER_GPU 1 IMAGE_CHANNEL_COUNT 3 IMAGE_MAX_DIM 1024 IMAGE_META_SIZE 93 IMAGE_MIN_DIM 1024 IMAGE_MIN_SCALE 0 IMAGE_RESIZE_MODE square IMAGE_SHAPE [1024 1024 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME my_train NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 PRE_NMS_LIMIT 6000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (32, 64, 128, 256, 512) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 100 TOP_DOWN_PYRAMID_SIZE 256 TRAIN_BN False TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 20 WEIGHT_DECAY 0.0001

第三步：准备数据集

我们使用封装好的CocoDataset类，生成训练集和验证集。

In [7]:

from src.mrcnn.coco import CocoDataset COCO_DIR = 'data' # 生成训练集 dataset_train = CocoDataset() dataset_train.load_coco(COCO_DIR, "train") #　加载训练数据集 dataset_train.prepare()

loading annotations into memory... Done (t=0.04s) creating index... index created!

In [8]:

# 生成验证集 dataset_val = CocoDataset() dataset_val.load_coco(COCO_DIR, "val") #　加载验证数据集 dataset_val.prepare()

loading annotations into memory... Done (t=0.04s) creating index... index created!

创建模型

第一步：用"training"模式创建模型对象，用于形状数据集的训练

In [9]:

model = modellib.MaskRCNN(mode="training", config=config, model_dir=MODEL_DIR)

[DEBUG] <__main__.MyTrainConfig object at 0x7f50e6344518> [DEBUG] Tensor("rpn_class/concat:0", shape=(?, ?, 2), dtype=float32) Tensor("rpn_bbox_1/concat:0", shape=(?, ?, 4), dtype=float32)

第二步：加载预训练模型的权重

In [10]:

model.load_weights(COCO_MODEL_PATH, by_name=True)

接下来，我们使用预训练的模型，结合Shapes数据集，对模型进行训练

训练模型

Keras中的模型可以按照制定的层进行构建，在模型的train方法中，我们可以通过layers参数来指定特定的层进行训练。layers参数有以下几种预设值：

heads:只训练head网络中的分类、mask和bbox回归

all: 所有的layer

3+: 训练ResNet Stage3和后续Stage

4+: 训练ResNet Stage4和后续Stage

5+: 训练ResNet Stage5和后续Stage

此外，layers参数还支持正则表达式，按照匹配规则指定layer，可以调用model.keras_model.summary()查看各个层的名称，然后按照需要指定要训练的层。

In [11]:

model.train(dataset_train, dataset_val, learning_rate=config.LEARNING_RATE, epochs=1, layers='all') model_savepath = 'my_mrcnn_model.h5' model.keras_model.save_weights(model_savepath)

Starting at epoch 0. LR=0.001 Checkpoint Path: logs/my_train20190827T1655/mask_rcnn_my_train_{epoch:04d}.h5 Selecting layers to train conv1 (Conv2D) bn_conv1 (BatchNorm) res2a_branch2a (Conv2D) bn2a_branch2a (BatchNorm) res2a_branch2b (Conv2D) bn2a_branch2b (BatchNorm) res2a_branch2c (Conv2D) res2a_branch1 (Conv2D) bn2a_branch2c (BatchNorm) bn2a_branch1 (BatchNorm) res2b_branch2a (Conv2D) bn2b_branch2a (BatchNorm) res2b_branch2b (Conv2D) bn2b_branch2b (BatchNorm) res2b_branch2c (Conv2D) bn2b_branch2c (BatchNorm) res2c_branch2a (Conv2D) bn2c_branch2a (BatchNorm) res2c_branch2b (Conv2D) bn2c_branch2b (BatchNorm) res2c_branch2c (Conv2D) bn2c_branch2c (BatchNorm) res3a_branch2a (Conv2D) bn3a_branch2a (BatchNorm) res3a_branch2b (Conv2D) bn3a_branch2b (BatchNorm) res3a_branch2c (Conv2D) res3a_branch1 (Conv2D) bn3a_branch2c (BatchNorm) bn3a_branch1 (BatchNorm) res3b_branch2a (Conv2D) bn3b_branch2a (BatchNorm) res3b_branch2b (Conv2D) bn3b_branch2b (BatchNorm) res3b_branch2c (Conv2D) bn3b_branch2c (BatchNorm) res3c_branch2a (Conv2D) bn3c_branch2a (BatchNorm) res3c_branch2b (Conv2D) bn3c_branch2b (BatchNorm) res3c_branch2c (Conv2D) bn3c_branch2c (BatchNorm) res3d_branch2a (Conv2D) bn3d_branch2a (BatchNorm) res3d_branch2b (Conv2D) bn3d_branch2b (BatchNorm) res3d_branch2c (Conv2D) bn3d_branch2c (BatchNorm) res4a_branch2a (Conv2D) bn4a_branch2a (BatchNorm) res4a_branch2b (Conv2D) bn4a_branch2b (BatchNorm) res4a_branch2c (Conv2D) res4a_branch1 (Conv2D) bn4a_branch2c (BatchNorm) bn4a_branch1 (BatchNorm) res4b_branch2a (Conv2D) bn4b_branch2a (BatchNorm) res4b_branch2b (Conv2D) bn4b_branch2b (BatchNorm) res4b_branch2c (Conv2D) bn4b_branch2c (BatchNorm) res4c_branch2a (Conv2D) bn4c_branch2a (BatchNorm) res4c_branch2b (Conv2D) bn4c_branch2b (BatchNorm) res4c_branch2c (Conv2D) bn4c_branch2c (BatchNorm) res4d_branch2a (Conv2D) bn4d_branch2a (BatchNorm) res4d_branch2b (Conv2D) bn4d_branch2b (BatchNorm) res4d_branch2c (Conv2D) bn4d_branch2c (BatchNorm) res4e_branch2a (Conv2D) bn4e_branch2a (BatchNorm) res4e_branch2b (Conv2D) bn4e_branch2b (BatchNorm) res4e_branch2c (Conv2D) bn4e_branch2c (BatchNorm) res4f_branch2a (Conv2D) bn4f_branch2a (BatchNorm) res4f_branch2b (Conv2D) bn4f_branch2b (BatchNorm) res4f_branch2c (Conv2D) bn4f_branch2c (BatchNorm) res4g_branch2a (Conv2D) bn4g_branch2a (BatchNorm) res4g_branch2b (Conv2D) bn4g_branch2b (BatchNorm) res4g_branch2c (Conv2D) bn4g_branch2c (BatchNorm) res4h_branch2a (Conv2D) bn4h_branch2a (BatchNorm) res4h_branch2b (Conv2D) bn4h_branch2b (BatchNorm) res4h_branch2c (Conv2D) bn4h_branch2c (BatchNorm) res4i_branch2a (Conv2D) bn4i_branch2a (BatchNorm) res4i_branch2b (Conv2D) bn4i_branch2b (BatchNorm) res4i_branch2c (Conv2D) bn4i_branch2c (BatchNorm) res4j_branch2a (Conv2D) bn4j_branch2a (BatchNorm) res4j_branch2b (Conv2D) bn4j_branch2b (BatchNorm) res4j_branch2c (Conv2D) bn4j_branch2c (BatchNorm) res4k_branch2a (Conv2D) bn4k_branch2a (BatchNorm) res4k_branch2b (Conv2D) bn4k_branch2b (BatchNorm) res4k_branch2c (Conv2D) bn4k_branch2c (BatchNorm) res4l_branch2a (Conv2D) bn4l_branch2a (BatchNorm) res4l_branch2b (Conv2D) bn4l_branch2b (BatchNorm) res4l_branch2c (Conv2D) bn4l_branch2c (BatchNorm) res4m_branch2a (Conv2D) bn4m_branch2a (BatchNorm) res4m_branch2b (Conv2D) bn4m_branch2b (BatchNorm) res4m_branch2c (Conv2D) bn4m_branch2c (BatchNorm) res4n_branch2a (Conv2D) bn4n_branch2a (BatchNorm) res4n_branch2b (Conv2D) bn4n_branch2b (BatchNorm) res4n_branch2c (Conv2D) bn4n_branch2c (BatchNorm) res4o_branch2a (Conv2D) bn4o_branch2a (BatchNorm) res4o_branch2b (Conv2D) bn4o_branch2b (BatchNorm) res4o_branch2c (Conv2D) bn4o_branch2c (BatchNorm) res4p_branch2a (Conv2D) bn4p_branch2a (BatchNorm) res4p_branch2b (Conv2D) bn4p_branch2b (BatchNorm) res4p_branch2c (Conv2D) bn4p_branch2c (BatchNorm) res4q_branch2a (Conv2D) bn4q_branch2a (BatchNorm) res4q_branch2b (Conv2D) bn4q_branch2b (BatchNorm) res4q_branch2c (Conv2D) bn4q_branch2c (BatchNorm) res4r_branch2a (Conv2D) bn4r_branch2a (BatchNorm) res4r_branch2b (Conv2D) bn4r_branch2b (BatchNorm) res4r_branch2c (Conv2D) bn4r_branch2c (BatchNorm) res4s_branch2a (Conv2D) bn4s_branch2a (BatchNorm) res4s_branch2b (Conv2D) bn4s_branch2b (BatchNorm) res4s_branch2c (Conv2D) bn4s_branch2c (BatchNorm) res4t_branch2a (Conv2D) bn4t_branch2a (BatchNorm) res4t_branch2b (Conv2D) bn4t_branch2b (BatchNorm) res4t_branch2c (Conv2D) bn4t_branch2c (BatchNorm) res4u_branch2a (Conv2D) bn4u_branch2a (BatchNorm) res4u_branch2b (Conv2D) bn4u_branch2b (BatchNorm) res4u_branch2c (Conv2D) bn4u_branch2c (BatchNorm) res4v_branch2a (Conv2D) bn4v_branch2a (BatchNorm) res4v_branch2b (Conv2D) bn4v_branch2b (BatchNorm) res4v_branch2c (Conv2D) bn4v_branch2c (BatchNorm) res4w_branch2a (Conv2D) bn4w_branch2a (BatchNorm) res4w_branch2b (Conv2D) bn4w_branch2b (BatchNorm) res4w_branch2c (Conv2D) bn4w_branch2c (BatchNorm) res5a_branch2a (Conv2D) bn5a_branch2a (BatchNorm) res5a_branch2b (Conv2D) bn5a_branch2b (BatchNorm) res5a_branch2c (Conv2D) res5a_branch1 (Conv2D) bn5a_branch2c (BatchNorm) bn5a_branch1 (BatchNorm) res5b_branch2a (Conv2D) bn5b_branch2a (BatchNorm) res5b_branch2b (Conv2D) bn5b_branch2b (BatchNorm) res5b_branch2c (Conv2D) bn5b_branch2c (BatchNorm) res5c_branch2a (Conv2D) bn5c_branch2a (BatchNorm) res5c_branch2b (Conv2D) bn5c_branch2b (BatchNorm) res5c_branch2c (Conv2D) bn5c_branch2c (BatchNorm) fpn_c5p5 (Conv2D) fpn_c4p4 (Conv2D) fpn_c3p3 (Conv2D) fpn_c2p2 (Conv2D) fpn_p5 (Conv2D) fpn_p2 (Conv2D) fpn_p3 (Conv2D) fpn_p4 (Conv2D) In model: rpn_model rpn_conv_shared (Conv2D) rpn_class_raw (Conv2D) rpn_bbox_pred (Conv2D) mrcnn_mask_conv1 (TimeDistributed) mrcnn_mask_bn1 (TimeDistributed) mrcnn_mask_conv2 (TimeDistributed) mrcnn_mask_bn2 (TimeDistributed) mrcnn_class_conv1 (TimeDistributed) mrcnn_class_bn1 (TimeDistributed) mrcnn_mask_conv3 (TimeDistributed) mrcnn_mask_bn3 (TimeDistributed) mrcnn_class_conv2 (TimeDistributed) mrcnn_class_bn2 (TimeDistributed) mrcnn_mask_conv4 (TimeDistributed) mrcnn_mask_bn4 (TimeDistributed) mrcnn_bbox_fc (TimeDistributed) mrcnn_mask_deconv (TimeDistributed) mrcnn_class_logits (TimeDistributed) mrcnn_mask (TimeDistributed)

/home/ma-user/anaconda3/envs/TensorFlow-1.8/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:100: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " /home/ma-user/anaconda3/envs/TensorFlow-1.8/lib/python3.6/site-packages/keras/engine/training_generator.py:44: UserWarning: Using a generator with `use_multiprocessing=True` and multiple workers may duplicate your data. Please consider using the`keras.utils.Sequence class. UserWarning('Using a generator with `use_multiprocessing=True`'

Epoch 1/1 99/100 [============================>.] - ETA: 1s - loss: 0.7529 - rpn_class_loss: 0.0156 - rpn_bbox_loss: 0.1811 - mrcnn_class_loss: 0.1799 - mrcnn_bbox_loss: 0.1343 - mrcnn_mask_loss: 0.2420

/home/ma-user/anaconda3/envs/TensorFlow-1.8/lib/python3.6/site-packages/keras/engine/training_generator.py:272: UserWarning: Using a generator with `use_multiprocessing=True` and multiple workers may duplicate your data. Please consider using the`keras.utils.Sequence class. UserWarning('Using a generator with `use_multiprocessing=True`'

100/100 [==============================] - 120s 1s/step - loss: 0.7607 - rpn_class_loss: 0.0155 - rpn_bbox_loss: 0.1823 - mrcnn_class_loss: 0.1846 - mrcnn_bbox_loss: 0.1363 - mrcnn_mask_loss: 0.2419 - val_loss: 0.8847 - val_rpn_class_loss: 0.0096 - val_rpn_bbox_loss: 0.1637 - val_mrcnn_class_loss: 0.2418 - val_mrcnn_bbox_loss: 0.1997 - val_mrcnn_mask_loss: 0.2698

使用Mask R-CNN 检测图片物体

第一步：定义InferenceConfig，并创建"Inference"模式的模型对象

In [12]:

class InferenceConfig(MyTrainConfig): GPU_COUNT = 1 IMAGES_PER_GPU = 1 inference_config = InferenceConfig() inference_model = modellib.MaskRCNN(mode="inference", config=inference_config, model_dir=MODEL_DIR)

[DEBUG] <__main__.InferenceConfig object at 0x7f4c1bd20470> [DEBUG] Tensor("rpn_class_1/concat:0", shape=(?, ?, 2), dtype=float32) Tensor("rpn_bbox_3/concat:0", shape=(?, ?, 4), dtype=float32) Tensor("input_anchors:0", shape=(?, ?, 4), dtype=float32)

将我们生成的模型权重信息加载进来

In [13]:

# 加载我们自己训练出的形状模型文件的权重信息 print("Loading weights from ", model_savepath) inference_model.load_weights(model_savepath, by_name=True)

Loading weights from my_mrcnn_model.h5

第二步：从验证数据集中随机选出一张图片进行预测，并显示结果

In [14]:

# 随机选出图片进行测试 image_id = random.choice(dataset_val.image_ids) original_image, image_meta, gt_class_id, gt_bbox, gt_mask =\ modellib.load_image_gt(dataset_val, inference_config, image_id, use_mini_mask=False) log("original_image", original_image) log("image_meta", image_meta) log("gt_class_id", gt_class_id) log("gt_bbox", gt_bbox) log("gt_mask", gt_mask) det_instances_savepath = 'random.det_instances.jpg' visualize.display_instances(original_image, gt_bbox, gt_mask, gt_class_id, dataset_train.class_names, figsize=(8, 8), save_path=det_instances_savepath)

original_image shape: (1024, 1024, 3) min: 0.00000 max: 255.00000 uint8 image_meta shape: (93,) min: 0.00000 max: 1024.00000 float64 gt_class_id shape: (2,) min: 16.00000 max: 26.00000 int32 gt_bbox shape: (2, 4) min: 5.00000 max: 1024.00000 int32 gt_mask shape: (1024, 1024, 2) min: 0.00000 max: 1.00000 bool

In [15]:

# 定义助手函数用于设置matplot中的子绘制区域所在的行和列 def get_ax(rows=1, cols=1, size=8): _, ax = plt.subplots(rows, cols, figsize=(size*cols, size*rows)) return ax results = inference_model.detect([original_image], verbose=1) r = results[0] prediction_savepath = 'random.prediction.jpg' visualize.display_instances(original_image, r['rois'], r['masks'], r['class_ids'], dataset_val.class_names, r['scores'], ax=get_ax(), save_path=prediction_savepath)

Processing 1 images image shape: (1024, 1024, 3) min: 0.00000 max: 255.00000 uint8 molded_images shape: (1, 1024, 1024, 3) min: -123.70000 max: 151.10000 float64 image_metas shape: (1, 93) min: 0.00000 max: 1024.00000 int64 anchors shape: (1, 261888, 4) min: -0.35390 max: 1.29134 float32

第三步：也可以通过上传自己的图片进行预测。

上传图片方式：点击网页的’upload‘按钮，将本地图片文件上传，上传后图片所在的路径存储为test_path。

In [16]:

test_path = './test.jpg'

In [17]:

import skimage.io image = skimage.io.imread(test_path) results = inference_model.detect([image], verbose=1) r = results[0] prediction_savepath = 'self.prediction.jpg' visualize.display_instances(image, r['rois'], r['masks'], r['class_ids'], dataset_val.class_names, r['scores'], ax=get_ax(), save_path=prediction_savepath)

Processing 1 images image shape: (1067, 1600, 3) min: 0.00000 max: 255.00000 uint8 molded_images shape: (1, 1024, 1024, 3) min: -123.70000 max: 151.10000 float64 image_metas shape: (1, 93) min: 0.00000 max: 1600.00000 float64 anchors shape: (1, 261888, 4) min: -0.35390 max: 1.29134 float32

评估模型

这一步我们对自己训练出的模型进行一个简单的评估。计算模型的平均精度mAP（mean Average Precision)

In [18]:

# 计算VOC类型的 mAP，条件是 IoU=0.5 # 下面的示例中只选出10张图片进行评估，增加图片数量可以增加模型评估的准确性 image_ids = np.random.choice(dataset_val.image_ids, 10) APs = [] for image_id in image_ids: # Load image and ground truth data image, image_meta, gt_class_id, gt_bbox, gt_mask =\ modellib.load_image_gt(dataset_val, inference_config, image_id, use_mini_mask=False) molded_images = np.expand_dims(modellib.mold_image(image, inference_config), 0) # Run object detection results = inference_model.detect([image], verbose=0) r = results[0] # Compute AP AP, precisions, recalls, overlaps =\ utils.compute_ap(gt_bbox, gt_class_id, gt_mask, r["rois"], r["class_ids"], r["scores"], r['masks']) APs.append(AP) print("mAP: ", np.mean(APs))

mAP: 0.6920385847279131

机器学习 神经网络

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。